Video encoding method and apparatus, electronic device, and storage medium

Abstract

The application provides a video coding method and device, an electronic device and a storage medium. The method comprises: obtaining original video data to be coded, and obtaining resource state data of each candidate computing node in a target node cluster; determining an effective key frame in the original video data, and dividing the original video data according to the effective key frame to obtain video fragments; calculating video complexity of the video fragments, and calling a neural network model to generate a predicted coding cost of each video fragment performing a video coding task on a corresponding candidate computing node based on the video complexity and the resource state data; determining a target computing node corresponding to each video fragment in the candidate computing nodes according to the predicted coding cost and a quality of experience index; calling the target computing node to perform video coding on the video fragments, and determining a target video coding result based on video coding results of all the video fragments. The application can accurately and reasonably allocate computing nodes for different video contents.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a video encoding method and apparatus, electronic device and storage medium. Background Technology

[0002] With the development of internet technology and the upgrading of information infrastructure, online multimedia applications have become an indispensable part of people's daily lives, promoting an explosive growth in video data. From high-definition video-on-demand and high-definition live streaming to artificial intelligence video streaming, diverse application scenarios have placed extremely high demands on network transmission bandwidth and computing power.

[0003] To cope with the processing pressure of large-scale videos, related technologies usually adopt a distributed video coding architecture to improve the timeliness of video transcoding. This distributed processing mechanism usually requires pre-cutting the video stream to be processed into multiple independent video segments according to fixed rules (such as fixed duration), and distributing these segments evenly to multiple computing nodes for parallel encoding.

[0004] However, the relevant technologies cannot accurately assess the actual computational overhead caused by differences in image features between different video content, resulting in the inability to allocate accurate and reasonable computational nodes to different video content. Summary of the Invention

[0005] The main objective of this application is to propose a video encoding method, apparatus, electronic device, and storage medium, which aims to accurately assess the actual computational overhead caused by differences in image features between different video contents, thereby allocating accurate and reasonable computational nodes to different video contents.

[0006] To achieve the above objectives, a first aspect of this application proposes a video coding method, the method comprising: Obtain the raw video data to be encoded, and obtain the resource status data of each candidate computing node in the target node cluster; Valid keyframes are identified in the original video data, and the original video data is divided according to the valid keyframes to obtain video segments; Calculate the video complexity corresponding to each video segment, and call a preset neural network model to generate the predicted encoding cost of each video segment performing video encoding tasks on the corresponding candidate computing node based on the video complexity and the resource status data; Obtain the user experience quality index for each video segment, and determine the target computing node corresponding to each video segment from the candidate computing nodes based on the predicted encoding cost and the user experience quality index. The target computing node is invoked to perform video encoding on the corresponding video segments to obtain the corresponding video encoding results, and the target video encoding result corresponding to the original video data is determined based on the video encoding results of all the video segments.

[0007] To achieve the above objectives, a second aspect of this application provides a video encoding apparatus, the apparatus comprising: The acquisition unit is used to acquire the raw video data to be encoded and to acquire the resource status data of each candidate computing node in the target node cluster. A segmentation unit is used to determine valid keyframes in the original video data and to segment the original video data according to the valid keyframes to obtain video segments. The generation unit is used to calculate the video complexity corresponding to each video segment, and call a preset neural network model to generate the predicted encoding cost of each video segment performing video encoding tasks on the corresponding candidate computing node based on the video complexity and the resource status data. The first determining unit is used to obtain the user experience quality index of each video segment, and determine the target computing node corresponding to each video segment from the candidate computing nodes according to the predicted coding cost and the user experience quality index. The second determining unit is used to call the target computing node to perform video encoding on the corresponding video segments to obtain the corresponding video encoding results, and to determine the target video encoding result corresponding to the original video data based on the video encoding results of all the video segments.

[0008] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0009] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0010] The video encoding method, apparatus, electronic device, and storage medium proposed in this application divide the original video data using effective keyframes, thus solving the technical problem of missing decoding dependencies or splicing anomalies caused by non-keyframe slicing. Then, the video complexity of each video segment is calculated, and the resource status data of each candidate computing node is obtained. Further, a neural network model is invoked to generate the predicted encoding cost for each video segment to perform video encoding tasks on the corresponding candidate computing node based on the video complexity and resource status data. In this way, this embodiment solves the problem in related technologies where task scheduling relies solely on fixed durations or single hardware loads, leading to inaccurate and unreasonable allocation of computing nodes, and achieves deep modeling and matching of video content features and underlying resource profiles of nodes. Subsequently, the user experience quality index of each video segment is obtained, and the optimal computing node corresponding to each video segment is determined from the candidate computing nodes based on a comprehensive score of the predicted encoding cost and the user experience quality index. Therefore, this embodiment can accurately assess the actual computational overhead caused by differences in image features between different video content by combining multi-dimensional video content features, the actual processing capabilities of computing nodes, and user viewing experience needs, thereby allocating accurate and reasonable computing nodes for different video content. Attached Figure Description

[0011] Figure 1 This is a flowchart of the video encoding method provided in the embodiments of this application; Figure 2 This is an architecture diagram of the video encoding system provided in the embodiments of this application; Figure 3 yes Figure 1 The flowchart of step S102 in the document; Figure 4 yes Figure 1 The flowchart of step S102 in the document; Figure 5 yes Figure 1 The flowchart of step S103 in the process; Figure 6 This is another flowchart of the video encoding method provided in the embodiments of this application; Figure 7 This is another flowchart of the video encoding method provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the video encoding device provided in the embodiments of this application; Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0013] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0014] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0015] With the development of internet technology and the upgrading of information infrastructure, online multimedia applications have become an indispensable part of people's daily lives, promoting the growth of video data volume. In particular, the development of high-definition video, ultra-high-definition video, live streaming, and artificial intelligence (AI) video generation technologies has placed extremely high demands on network transmission bandwidth and computing power in diversified application scenarios.

[0016] Traditional stand-alone video encoding schemes often suffer from long encoding times and low resource utilization when processing 4K, 8K, high frame rate, and large-scale video tasks due to limited computing resources. To cope with the processing pressure of large-scale video, related technologies generally adopt distributed video encoding architectures to improve the timeliness of video transcoding. However, this distributed processing mechanism usually requires pre-cutting the video stream to be processed into multiple independent video segments according to fixed rules, then distributing these segments evenly across multiple computing nodes to perform encoding tasks in parallel, and finally concatenating the encoded results for output. For example, related technologies typically employ Kubernetes-based containerized video transcoding clusters, FFmpeg-based video slicing and concatenation schemes, and HLS / DASH-based video segmentation encoding schemes.

[0017] However, related technologies cannot accurately assess the actual computational overhead caused by differences in image features between different video content, making it impossible to allocate accurate and reasonable computational nodes to different video content. These technologies typically employ a fixed-length video slicing strategy, failing to fully consider the image group structure, scene complexity, and motion intensity of the video, which easily leads to uneven encoding time consumption and degraded stitching quality.

[0018] Furthermore, at the task scheduling level, existing systems primarily allocate tasks based on static metrics such as CPU utilization, GPU utilization, memory usage, and node idle status, lacking the ability to perceive video content. Because there are significant differences in encoding complexity between different video segments (e.g., static conference footage has lower encoding complexity, while high-motion footage such as sports events and game streams has higher encoding complexity), even if the segment length is the same, their actual encoding time can still vary considerably. This mechanism easily leads to uneven node load, waste of heterogeneous GPU resources, and blocking of tasks at the end of the encoding process, ultimately increasing the overall transcoding completion time.

[0019] Furthermore, the distributed scheduling systems in these technologies lack a Quality of Experience (QoE)-driven mechanism and a dynamic feedback loop. These systems often focus solely on system throughput and encoding speed, neglecting to adequately consider performance metrics such as video quality, user viewing experience, and bitrate fluctuations. Simultaneously, due to the use of static scheduling models, these technologies cannot update the computational cost prediction model based on actual encoding results, nor can they effectively correct the true capabilities of each computing node, resulting in a lack of adaptive optimization of scheduling strategies.

[0020] Therefore, there is an urgent need in this field for an intelligent video distributed coding and scheduling scheme that can combine video content complexity, node resource capabilities, video experience quality, and dynamic feedback mechanisms to accurately predict coding costs, improve the utilization of heterogeneous resources, and continuously optimize task scheduling efficiency.

[0021] Based on this, embodiments of this application provide a video encoding method and apparatus, electronic device and storage medium, which aim to accurately assess the actual computational overhead caused by differences in image features between different video content, thereby allocating accurate and reasonable computational nodes to different video content.

[0022] The video encoding method, apparatus, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the video encoding method in this application is described.

[0023] The video encoding method provided in this application relates to the field of artificial intelligence technology. The video encoding method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the video encoding method, but is not limited to the above forms.

[0024] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0025] Figure 1 This is an optional flowchart of the video encoding method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S105.

[0026] Step S101: Obtain the raw video data to be encoded and obtain the resource status data of each candidate computing node in the target node cluster; Step S102: Determine the valid keyframes in the original video data, and divide the original video data according to the valid keyframes to obtain video segments; Step S103: Calculate the video complexity corresponding to each video segment, and call the preset neural network model to generate the predicted encoding cost of each video segment performing video encoding tasks on the corresponding candidate computing nodes based on the video complexity and resource status data. Step S104: Obtain the user experience quality index for each video segment, and determine the target computing node corresponding to each video segment from the candidate computing nodes based on the predicted encoding cost and the user experience quality index. Step S105: Call the target computing node to perform video encoding on the corresponding video segments to obtain the corresponding video encoding results, and determine the target video encoding result corresponding to the original video data based on the video encoding results of all video segments.

[0027] Steps S101 to S105 of this embodiment, which divide the original video data using effective keyframes, can solve the technical problem of missing decoding dependencies or splicing anomalies caused by non-keyframe slicing. Then, the video complexity of the video segments is calculated, and the resource status data of each candidate computing node is obtained. Further, a neural network model is invoked to generate the predicted encoding cost for each video segment to perform video encoding tasks on the corresponding candidate computing node based on the video complexity and resource status data. Thus, this embodiment solves the problem in related technologies where task scheduling relies solely on fixed durations or single hardware loads, leading to inaccurate and unreasonable allocation of computing nodes, and achieves deep modeling and matching of video content features and underlying resource profiles of nodes. Subsequently, the user experience quality index of each video segment is obtained, and based on the comprehensive score of the predicted encoding cost and the user experience quality index, the optimal computing node corresponding to each video segment is determined from the candidate computing nodes. Therefore, this embodiment can accurately assess the actual computational overhead caused by differences in image features between different video content by combining multi-dimensional video content features, the actual processing capabilities of computing nodes, and user viewing experience needs, thereby allocating accurate and reasonable computing nodes for different video content.

[0028] Before introducing the video coding method provided in this application, the video coding architecture provided in this application will be introduced first. Please refer to... Figure 2 , Figure 2 This is an architecture diagram of the video encoding system provided in this application, such as... Figure 2As shown, the video encoding system is a distributed encoding scheduling system. It includes a video parsing module, an adaptive slicing module, a video complexity analysis module, a node resource awareness module, an AI prediction module, a scheduling optimization module, a distributed encoding module, a stitching output module, and a dynamic feedback module. First, the input raw video data undergoes metadata extraction and structural analysis by the video parsing module to obtain the video parsing result. This result is then sent to the adaptive slicing module, which cuts the video into multiple independent video segments at keyframe positions. Subsequently, the video complexity analysis module extracts content features from each video segment and calculates the corresponding video complexity based on these features. Simultaneously, the node resource awareness module collects resource status data from each computing node within the target node cluster to generate a node capability profile. Then, the video complexity and node capability profiles are fed into the AI ​​prediction module, which uses a neural network model to predict the encoding cost of a specific video segment on each candidate computing node. The scheduling optimization module then generates an optimal video encoding task distribution strategy based on a preset strategy. Finally, the distributed encoding module distributes the encoding tasks to the corresponding nodes for parallel processing according to the distribution strategy. After the encoding task is completed, on the one hand, the splicing output module performs time-series correction and merging and encapsulation on the discrete video encoding results to output the target video file. On the other hand, the dynamic feedback module collects real node operation indicators and image quality evaluation results, and updates the parameters of the AI ​​prediction model and the node capability profile by calculating the prediction deviation, thereby forming a continuous iterative closed-loop system with adaptive learning capabilities.

[0029] In step S101, the original video data to be encoded is first obtained through the video parsing module. This original video data is the video stream to be processed. The module performs container format parsing, audio / video track identification, encoding parameter reading, and frame-level structure analysis on the original video data to obtain video attribute information. The container format can include, but is not limited to, MP4, TS, FLV, or MKV. Specifically, the video parsing module obtains basic metadata corresponding to the original video data, such as resolution, frame rate, bit rate, encoding format, Group of Pictures (GOP) structure, I-frame (intra-coded frame) position, Presentation Time Stamp (PTS), and Decoding Time Stamp (DTS).

[0030] Simultaneously, the resource status data of each candidate computing node in the target node cluster is periodically collected through the node resource awareness module. The target node cluster consists of multiple heterogeneous computing nodes, each with different computing resource capabilities. Specifically, the resource status data includes the number of CPU cores, CPU frequency, CPU real-time utilization, GPU model, GPU memory capacity, GPU encoder capability, GPU real-time load, memory capacity and bandwidth, network bandwidth, storage I / O capability, current task queue length, and historical encoding completion status of the candidate computing node. Then, a corresponding node capability vector is generated for each candidate computing node based on the resource status data. The node capability vector is used to characterize the comprehensive resource capability level of the candidate computing node at the current moment. For example, in the target node cluster, the first... The node capability vector of each candidate computing node can be represented as: ,in This indicates the CPU processing power of the candidate computing nodes. This indicates the GPU coding capability of the candidate computing nodes. This indicates the memory capacity of the candidate computing nodes. This indicates the network transmission capability of the candidate computing nodes. This represents the storage I / O capability of candidate computing nodes. For example, if a candidate computing node is equipped with an 8-core CPU, an NVIDIA A100 GPU, 40GB of video memory, 100Gbps network bandwidth, and high-speed NVMe storage, then the components in its node capability vector will be relatively high, indicating that the node has strong video encoding capabilities. On the other hand, if another candidate computing node is only equipped with a 4-core CPU, no dedicated GPU, 16GB of memory, and 1Gbps network bandwidth, then the GPU-related components in its node capability vector will be zero or extremely low, indicating that the node is more suitable for handling low-complexity encoding tasks.

[0031] In step S102, valid keyframes can be identified in the original video data, and the original video data can be divided according to the valid keyframes to obtain multiple independent and decodable video segments. Specifically, firstly, frame-level parsing is performed on the original video data to identify I-frames in the original video data. Since I-frames can be decoded independently without relying on preceding frames, I-frames can be used as the starting point of video slicing. Therefore, in this embodiment, the identified I-frames are used as initial candidate keyframes.

[0032] Then, the validity of the initial candidate keyframes is determined by combining the GOP structure of the original video data, thus filtering the initial candidate keyframes. Specifically, for I-frames located at the beginning of a closed GOP, they are marked as candidate keyframes that can be decoded independently. If an I-frame is located in an open GOP and there are cross-GOP frame references, the search can continue to find the nearest closed GOP boundary to prevent the problem of missing reference frames after video slicing. Then, based on the scene transition position, the search window is used to find the I-frames that are closest to the scene transition position and meet the GOP integrity requirements, and these are finally determined as the set of valid keyframes. .

[0033] In some embodiments, please refer to Figure 3 Determine valid keyframes in the raw video data, including: Step S301: Determine the candidate internal encoded frames and the frame group structure parameters in the original video data, and determine the scene switching positions in the original video data. Step S302: Within the first preset search time window corresponding to the scene switching position, the target internal encoded frame is filtered from the candidate internal encoded frames based on the integrity constraints of the screen group structure parameters, and the effective key frame is determined based on the target internal encoded frame.

[0034] In step S301, as described above, the original video data is analyzed at the frame level by the video parsing module. The encoding type of each frame is read, and the I-frame (Intra-coded frame), P-frame (Predicted frame), and B-frame (Bidirectional predicted frame) are identified. The P-frame encoding process refers to the preceding I-frame or P-frame and uses inter-frame temporal redundancy for compression. The compression efficiency is higher than that of the I-frame, but it cannot be decoded independently. The B-frame is a bidirectional predicted frame. Its encoding process refers to both the preceding and following frames. It has the highest compression efficiency, but the decoding dependency is complex.

[0035] Then, based on the position of the I-frame on the video timeline, the corresponding position of the candidate internal coded frame is determined. Finally, based on the positions of all I-frames on the video timeline, a set of candidate internal coded frames is formed, which can be represented as follows: Among them, candidate intra-coded frames, or I-frames, represent intra-coded frames. Candidate intra-coded frames are video frames that can be decoded independently without relying on preceding or subsequent frames.

[0036] Simultaneously, the frame group structure in the original video data can be parsed, and frame group parameters can be extracted. These parameters include the distance between adjacent internal coded frames, the length of the frame group (the total number of frames contained in a GOP), whether the frame group is closed (i.e., whether it is a closed GOP, indicating whether the B-frames of the current GOP only reference frames within the same GOP; if the B-frames do not reference frames from other GOPs, then the GOP is a closed GOP, otherwise it is an open GOP), and information such as inter-frame reference relationships.

[0037] Furthermore, this application can also detect and determine the temporal locations where significant changes occur in the image content of the original video data, such as a switch from a conference room scene to an outdoor scene, by analyzing histogram differences between adjacent frames, pixel differences between frames, edge changes, optical flow abrupt changes, image entropy changes, or scene switch markers output by the encoder. Based on these abrupt changes, the scene switch locations are determined, forming a set of scene switch locations, which can be represented as follows: In this regard, slicing video near the scene transition location can effectively avoid the inclusion of semantically different scene content within a single video slice, thereby improving subsequent encoding efficiency and splicing quality.

[0038] In step S302, specifically, when a scene switching location is detected... At that time, the first preset search time window is set based on the scene switching position. For example, by scene transition point Centered on the target, extend forward and backward by a preset number of video frames or a certain number of milliseconds. Within the first preset search time window. Within the framework, target internally encoded frames are selected from candidate internally encoded frames based on integrity constraints of the frame group structure parameters. These integrity constraints include: the selected target internally encoded frames must be able to serve as independent decoding starting points; they must be located at the beginning of a complete frame group; and their corresponding frame group must be closed, meaning there are no reference frame dependencies crossing segment boundaries or frame groups. Furthermore, the time distance between the target internally encoded frame and the scene transition position must be less than a preset threshold; the time interval between the target internally encoded frame and the previous determined valid keyframe must meet the minimum segment duration; and the time interval between the target internally encoded frame and the previous determined valid keyframe must not exceed the maximum segment duration. Thus, target internally encoded frames can be selected from the candidate internally encoded frame set based on these integrity constraints.

[0039] If an internally encoded frame is the starting frame of a closed frame group, but there is no obvious scene transition location nearby, it is marked as a normal candidate keyframe. Normal candidate keyframes are used to perform fallback slicing when the duration of the current video segment reaches the preset maximum segment length, to avoid excessively long processing times for a single video segment due to a lack of scene transitions. If the scene transition location itself corresponds to an internally encoded frame and simultaneously satisfies the integrity constraint of the closed frame group, this embodiment can determine the corresponding target internally encoded frame as a valid keyframe (i.e., a strong keyframe). If the scene transition location itself is not an internally encoded frame, then within the first preset search time window, a target internally encoded frame that is closest to the scene transition location and satisfies the integrity constraint is found as a replacement slice point, and thus established as a valid keyframe.

[0040] If a target internally encoded frame that meets the conditions exists in both the forward and backward search directions, this embodiment prioritizes selecting the target internally encoded frame closer to the scene transition position as the valid keyframe, while ensuring that the time interval between this frame and the previously determined valid keyframe meets the aforementioned duration constraint requirements. For internally encoded frames in open GOP structures with cross-scene group reference dependencies, the video coding system will not treat them as valid keyframes, but will continue searching for the next target internally encoded frame that meets the integrity constraint, or, if the coding parameters allow, re-encode the GOP near that position to form a closed GOP boundary that can be independently decoded. Thus, through the above filtering steps, the video coding system finally generates an accurate set of valid keyframes, which can be represented, for example, as follows: .

[0041] To achieve adaptive video slicing, this embodiment introduces a segment length classification mechanism. Specifically, video segments can be divided into three types—shorter segments, standard segments, and longer segments—based on set length boundaries or comprehensive complexity scores. In the first embodiment, a minimum segment length (e.g., 2 seconds), a standard segment length (e.g., 5 or 6 seconds), and a maximum segment length (e.g., 10 or 15 seconds) are preset. When the calculated target segment length is between the minimum and standard lengths, it is defined as a shorter segment, mainly used in high-motion-intensity, high-image-entropy, or high-scene-complexity coding regions. When the target segment length is close to the standard segment length, it is defined as a standard segment suitable for medium-complexity ordinary scenes. When the target segment length is between the standard and maximum lengths, it is defined as a longer segment, used in low-complexity scenarios such as static meetings or monitoring. In another implementation, the division can also be based directly on the normalized comprehensive complexity score. For example, a threshold can be set: when the score is greater than or equal to 0.7, it is judged as a high complexity region and a shorter segment is used; when the score is between 0.3 and 0.7, a standard segment is used; and when the score is less than 0.3, it is judged as a low complexity region and a longer segment is used. In this way, the balance between the video coding load and the efficiency of distributed scheduling can be flexibly taken into account.

[0042] This embodiment achieves intelligent and precise positioning of video segmentation boundaries in the original video data by combining the integrity constraints of the frame group structure parameters with the scene switching positions. Since each valid keyframe in the final established set of valid keyframes meets the requirements for independent decoding and the integrity of the frame group structure, and is as close as possible to the actual scene change points on the timeline, the video segments obtained by subsequent video cutting based on valid keyframes all possess completely independent encoding and decoding capabilities. Thus, compared to related technologies that slice the original video data with a fixed duration, this embodiment effectively avoids decoding errors caused by cutting at non-keyframe locations, reduces problems such as abrupt scene changes, missing reference frames, and audio-visual asynchrony at the video stitching boundaries after distributed parallel processing, and improves the continuity and visual experience of the final video.

[0043] In some embodiments, please refer to Figure 4 The original video data is divided based on valid keyframes to obtain video segments, including: Step S401: Obtain motion intensity data, image entropy data, and scene change feature data of the original video data, and calculate the comprehensive complexity score based on the motion intensity data, image entropy data, and scene change feature data. Step S402: Calculate the target slice length based on the comprehensive complexity score, and determine the corresponding target slice time based on the current starting time and the target slice length; Step S403: Determine the target slice boundary based on the valid keyframes within the second preset search time window near the target slice time, and divide the original video data according to the target slice boundary to obtain video segments.

[0044] In step S401, the original video data is first pre-analyzed according to a set time window. For each time window, motion intensity data, image entropy data, and scene change feature data corresponding to the original video data are acquired. The preset time window can be 1 second, 2 seconds, or one GOP cycle, or it can be a video segment between adjacent valid keyframes as the analysis window. Specifically, the motion intensity data (set as a parameter) Image entropy data is used to characterize the degree of change in a video frame over time. It can be calculated through inter-frame pixel differences, motion vector magnitude, optical flow changes, or changes in the structural similarity of adjacent frames. For example, a large average pixel difference between adjacent frames (such as in sports events or live game broadcasts) indicates rapid motion or camera movement within that time window, suggesting high motion intensity. Image entropy data (set as a parameter) Scene complexity is used to characterize the spatial texture complexity and information content of a video image. It can calculate the information entropy of the luminance or color components of a video frame. For example, if an image has many textures, edges, and noise details (such as landscapes, animations, or high-definition videos), the pixel distribution is more complex, resulting in higher image entropy. This means the encoder needs to process more spatial details, increasing the compression difficulty. Scene change feature data is the scene complexity (set as a parameter). The scene change index (FSI) is used to characterize the degree of semantic and structural change in video content. It can be calculated based on features such as scene switching frequency, number of camera changes, brightness abrupt changes, or edge density changes. For example, if camera switching or targets rapidly entering and exiting occur frequently within a certain time window, the scene changes rapidly; conversely, if a static meeting or fixed monitoring screen remains stable for a long time, the scene changes slowly.

[0045] After acquiring the above three types of feature data, they are normalized, and a comprehensive complexity score of the video content within the corresponding time window is calculated based on motion intensity data, image entropy data, and scene change feature data (set as parameter). Its corresponding mathematical description is: ,in, , , Let represent the weight coefficients of the three parameters mentioned above, and satisfy . In practical applications, it can improve performance in scenarios where motion factors have a significant impact. Weighting can be increased in scenes that significantly affect image details. The weighting can be increased for videos with frequent camera transitions. The weight.

[0046] In step S402, the target slice length is calculated based on the comprehensive complexity score, and the corresponding target slice time is determined based on the current starting time and the target slice length. In this embodiment, for high-complexity video segments with intense motion, rich texture details, or frequent scene changes, the target slice length is set to be shorter to reduce the encoding complexity and processing time of subsequent individual video segments; while for low-complexity video segments with gentle motion, simple image content, or few scene changes, the target slice length is set to be longer to reduce the total number of video segments and reduce task scheduling overhead.

[0047] Let the target slice length be a parameter. The maximum allowed fragment length in this embodiment is (e.g., 10 seconds), the minimum allowed fragment length is (e.g., 2 seconds), minimum chunk length indicates the shortest allowed video chunk length, such as 2 seconds or a full GOP period, less than... Fragmentation increases task scheduling overhead, file management overhead, and concatenation complexity; therefore, this embodiment constrains the generation of fragments shorter than [a certain length]. The maximum fragment length This indicates the maximum allowed fragment length, such as 10 seconds, 12 seconds, or 15 seconds. (Exceeding this limit...) Fragmentation can lead to excessively long individual encoding tasks, causing uneven node load or tail task blocking. Therefore, this embodiment constrains the generation of tasks longer than a certain time. The length of the target slice can be calculated using the following formula: The closer the overall complexity score is to 1, the more complex the original video data, and the closer the calculated target slice length is to the minimum slice length; conversely, the closer the overall complexity score is to 0, the closer the target slice length is to the maximum slice length. After obtaining the target slice length, it is assumed that the starting point of the current video slice is a valid keyframe. Its corresponding timestamp is the current starting time (set to ). By adding the current starting time and the target slice length on the time axis, the theoretical target slice time can be calculated (i.e., ).

[0048] In step S403, the target slice boundary is determined based on valid keyframes within a second preset search time window near the target slice time, and the original video data is divided according to the target slice boundary to obtain video segments. This embodiment only allows slicing at keyframe positions to ensure that each video segment has independent encoding and decoding capabilities after slicing. Therefore, this embodiment does not directly slice at any theoretical time point corresponding to the target slice time determined in step S402. Instead, a second preset search time window is set before and after the theoretical target slice time, and valid keyframes (denoted as ) that satisfy constraints such as independent decoding and GOP structure integrity are searched within the second preset search time window. If multiple candidate valid keyframes exist within the second preset search time window, this embodiment prioritizes selecting the keyframe that is closest in time to the target slice and will not cause the current slice length to exceed the minimum or maximum slice length constraints, and establishes it as the actual target slice boundary. Finally, the original video data is accurately divided according to the determined target slice boundary, and multiple video slice data and their corresponding slice description information (including slice number, start timestamp, end timestamp, and duration, etc.) are output, thereby completing the adaptive slicing process.

[0049] This embodiment divides the raw video data of complex scenes into shorter video segments, effectively reducing the time consumption of a single encoding task under high complexity; while simple scenes are divided into longer video segments, reducing the number of tasks and scheduling communication overhead, thereby achieving a more reasonable encoding load balancing in a distributed system. Simultaneously, since the actual target slice boundary is limited by the effective keyframes within the second preset search time window, this embodiment not only avoids image dependency loss and decoding errors caused by non-keyframe segmentation, but also improves the image quality and audio-visual synchronization effect at the video stitching boundary after multi-node concurrent encoding.

[0050] In step S103, there are significant differences in encoding complexity among different video segments. Even if the segment lengths are the same, their actual encoding time may still vary considerably. For example, static meeting footage has low encoding complexity due to its smooth motion, simple textures, and stable scenes, allowing the encoder to complete compression with less bitrate and computational resources. Conversely, high-motion footage such as sports events and game live streams has higher encoding complexity due to its drastic inter-frame changes, high motion vector amplitudes, and frequent scene transitions, requiring the encoder to consume more computational resources and encoding time to ensure image quality. Therefore, the video complexity corresponding to each video segment can be calculated first.

[0051] In some embodiments, please refer to Figure 5 Calculate the video complexity for each video segment, including: Step S501: Obtain the space complexity feature of each video segment and the time complexity feature of each video segment. Step S502: Obtain the coding structure feature data of each video segment; Step S503: Generate the corresponding video complexity based on the spatial complexity features, time complexity features, and coding structure feature data.

[0052] In step S501, spatial complexity features are used to measure the richness of detail and pixel distribution of video frames in the spatial dimension, including parameters such as image entropy, edge density, noise intensity, and color variation. For example, in landscape scenes or high-definition film clips containing a large amount of texture detail, the image entropy and edge density are high, indicating that the spatial features of the video segments are more complex, and the encoder needs to process more spatial details. Simultaneously, the temporal complexity features of each video segment are obtained, including parameters such as inter-frame differences, motion vector intensity, scene switching frequency, and optical flow changes. Through the comprehensive extraction of spatial and temporal complexity features, this embodiment can form a preliminary quantitative assessment of the encoding difficulty of video segments in terms of spatial detail and temporal evolution.

[0053] In step S502, the coding structure feature data of each video segment is obtained. This data represents the compression organization and data size of the video segment, including parameters such as GOP length, I-frame density, B-frame ratio, and input bitrate. A GOP typically begins with an I-frame, the GOP length represents the distance between two independent decodeable I-frames, and the I-frame density and B-frame ratio determine the distribution of intra-frame compression and bidirectional predictive inter-frame compression. For example, if a video segment has a high input bitrate and contains a large number of short GOP structures, it indicates that the segment contains more independent decoding reference frames, requiring more computational resources to maintain the high-density keyframe structure during subsequent processing.

[0054] In step S503, the corresponding video complexity is generated based on the spatial complexity features, temporal complexity features, and coding structure features. Specifically, the spatial complexity features, temporal complexity features, and coding structure features of each video segment are integrated to construct a comprehensive representation of the current video segment. The complexity feature vector of video segmentation encoding difficulty (set as parameter) ),in ,in Indicates resolution characteristics, The intensity of motion in the time complexity is represented by... Image entropy represents the space complexity. This represents the GOP structural features in the encoded structural feature data. This represents the code rate feature in the coded structure feature data, while This represents the scene complexity within the time complexity feature. Video complexity involves reducing and fusing the video's content attributes, motion patterns, and bitstream structure, and using these as inputs for subsequent distributed task scheduling and AI encoding cost prediction.

[0055] This embodiment overcomes the shortcomings of traditional video processing solutions that do not fully consider the differences in video content complexity. It can accurately and comprehensively quantify and evaluate the true encoding difficulty of each video segment from three dimensions: spatial texture, temporal motion, and underlying coding structure. Thus, this embodiment provides high-quality, multi-dimensional feature vector input for subsequent AI prediction models, enabling the video encoding system to accurately predict the encoding time and resource overhead of video segments on different computing nodes. It also provides a data foundation for achieving optimal matching of heterogeneous computing resources and cluster load balancing, improving the overall throughput efficiency and resource utilization of the distributed encoding system when processing complex video streams.

[0056] Furthermore, a pre-defined neural network model is invoked to generate the predicted encoding cost for each video segment to perform video encoding tasks on the corresponding candidate computing nodes, based on video complexity and resource status data. Specifically, the complexity feature vector is... With candidate node capability vector The inputs are fed into a pre-defined neural network model for combined prediction and evaluation. This neural network model includes convolutional neural networks (CNNs), transformer models, graph neural networks (GNNs), reinforcement learning models, or combinations thereof. The neural network model constructs a prediction function... The model matches the content complexity of a video segment with the hardware resource capabilities of a candidate computing node. The predicted coding cost output by the model includes the first... The first fragment in the... Predictive coding time on each node It also includes cost metrics such as predicted CPU / GPU resource usage, memory usage, predicted output bitrate, predicted image quality loss, and predicted bitrate fluctuation risk.

[0057] In step S104, the Quality of User Experience (QoE) metric for each video segment is obtained. This QoE metric characterizes the impact of video encoding results on the end-user's viewing experience. The QoE metric includes objective image quality evaluation standards such as Video Multi-method Assessment Fusion (VMAF), Structural Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR), as well as bitrate fluctuation, stuttering risk, and scene importance. Regarding bitrate fluctuation and stuttering risk, this embodiment quantifies and defines the QoE benefit value for each video segment, and its mathematical model is as follows: ,in Represents predicted image quality. Represents exercise intensity. The importance of the representative content This represents the risk of bitrate fluctuation. , , , These are all preset coefficients used to adjust the weight of various experience parameters.

[0058] When finally determining the target computing node, the scheduling optimization module comprehensively predicts the encoding cost, resource transmission overhead, and the aforementioned QoE benefit value to construct a scheduling objective function for optimization: In this formula, Indicates the time taken for prediction coding. Indicates computational resource consumption. Indicates data transmission cost. This represents the quality of experience gains that a video slice can obtain on the corresponding candidate computing node. to These represent the weight parameters for each dimension. In other optional embodiments, the scheduling objective function can also be configured, depending on actual business needs, as a single objective such as minimizing coding completion time, maximizing user experience quality, minimizing resource costs, or any combination of the above objectives.

[0059] Furthermore, based on the objective function, the matching relationship between each video segment and each candidate node is scored, and the selected segment is determined. The smallest candidate computing node is selected as the optimal target computing node for executing the encoding task of the corresponding video segment. Based on this, a task scheduling plan is output, which includes detailed information such as the number of the video segment, the routing information of the target computing node, the task priority, the input and output file paths, the expected completion time, and the failure retry policy.

[0060] In step S105, the target computing nodes are invoked to perform video encoding on the corresponding video segments to obtain the corresponding video encoding results. Specifically, the distributed encoding module of the video encoding system distributes each video segment to the corresponding target computing nodes according to the task scheduling plan generated in the preceding steps. Multiple target computing nodes then perform parallel encoding operations, including transcoding, bitrate control, and resolution conversion, and output the encoded video segment files, thus obtaining the video encoding results. Furthermore, the target video encoding results corresponding to the original video data can be determined based on the video encoding results of all video segments.

[0061] In some embodiments, determining the target video encoding result corresponding to the original video data based on the video encoding results of all video segments includes: Obtain the start display time, end decoding time, and frame group boundary identifier corresponding to each video encoding result, and obtain the segment number sequence corresponding to the video segment; All video encoding results are time-offset corrected according to the segment number order, start display time, end decoding time, and screen group boundary markers, and the target video encoding result is generated based on all video encoding results after time-offset correction.

[0062] In this embodiment, the multiple encoded video segment files (i.e., video encoding results) output by the distributed encoding module are parsed to obtain the start display time, end decoding time, and picture group boundary identifier corresponding to each video encoding result, and the segment number order corresponding to the video segment is obtained. The start display time represents the start presentation timestamp (PTS), used to indicate the specific playback presentation time of the first frame in the video segment; the end decoding time represents the end decoding timestamp (DTS), used to indicate the reference time when the last frame in the video segment enters the decoder for decoding. In the encoding structure storing B-frames, since the decoding of B-frames needs to refer to subsequent frames, the decoding timestamp usually precedes the display timestamp, meaning the decoding order is inconsistent with the display order; the picture group boundary identifier corresponds to GOP boundary information, used to indicate whether the beginning and end of the video segment are complete (whether it starts at the beginning frame of a complete picture group or ends at the end frame of a picture group) and are independently decodeable structural intervals.

[0063] Meanwhile, since the video is assigned segment description information arranged chronologically during the adaptive slicing stage, the segment number order corresponding to the video segments can be obtained to provide a basis for subsequent correct sorting. For example, for the original video data divided into 5 video segments, the segment number order is 1, 2, 3, 4, 5. Five corresponding video encoding results are obtained. Each video encoding result carries its own start display time, end decoding time, and frame group boundary identifier. The start display time of the video encoding result with segment number 1 is usually 0, and its frame group boundary identifier indicates that the segment starts at the first I-frame of the complete frame group.

[0064] Then, the splicing output module sorts all video encoding results according to the segment number order, ensuring that each segment is restored to its state before slicing in terms of order. Furthermore, combining the start display time, end decoding time, and frame group boundary markers of each video segment, time offset correction is performed on all video encoding results. Since each computing node executes the encoding task independently, and due to differences in encoder implementation, encoding parameters, and internal clocks among computing nodes during distributed encoding, the encoded segments output by different nodes may have slight offsets in actual timestamps. First, the encoded segments are sorted according to their segment numbers. Then, using the starting PTS of segment number 1 as the reference time point, the theoretical time offset of each subsequent segment relative to the reference time point is calculated sequentially, and the actual starting PTS and ending DTS of each segment are compared with the theoretical time offset. When the difference between the actual timestamp and the theoretical time offset of a certain encoded segment is detected to exceed a preset threshold (e.g., more than 2 frame periods or 40ms), the timestamp of the segment is corrected, and its start PTS and end DTS are uniformly offset to the time point corresponding to the theoretical time offset, thereby eliminating the timestamp deviation introduced by distributed coding.

[0065] While performing time offset correction, this embodiment also verifies the GOP structure of each video segment based on the GOP boundary marker to ensure that each corrected video segment still maintains a closed GOP structure at the splicing boundary. That is, the last frame of the previous GOP does not reference the first frame of the next GOP, and the first frame (I-frame) of the next GOP can be decoded independently, thereby ensuring that the spliced ​​video stream can be decoded normally at any position.

[0066] After completing the time offset correction, this embodiment can map and correct the relative timestamps of each isolated video segment to a unified global timeline, thereby correcting the timestamp offset caused by distributed coding. The video encoding results are written to the target video stream file in sequence according to the corrected timestamp order, and audio and video synchronization processing is performed during the writing process. That is, the PTS of the video track is fine-tuned according to the PTS information of the audio track to ensure the continuity of the audio track and the video track at the splicing point, and to ensure that the audio frame and the corresponding video frame remain synchronized during playback.

[0067] After completing sequence verification and time offset correction, the splicing output module merges all video encoding results that have undergone time axis alignment processing, encapsulates them to generate a target video stream file in the target format, and outputs the target video encoding result.

[0068] This embodiment obtains the start display time, end decoding time, frame group boundary identifier, and segment number sequence of each encoded video segment, and systematically corrects the timestamp offset introduced by distributed coding during the splicing stage. At the same time, it combines GOP boundary integrity verification and audio-video synchronization processing to effectively solve the technical problems of timestamp inconsistency, splicing boundary quality degradation, and audio-video asynchrony caused by multi-node parallel processing in distributed video coding. This ensures that the final generated target video stream file is consistent with the original video in terms of timing logic, decoding compatibility, and playback experience, thereby improving the output quality and reliability of the video distributed coding system.

[0069] In some embodiments, please refer to Figure 6 After determining the target video encoding result corresponding to the original video data based on the video encoding results of all video segments, the method also includes: Step S601: Obtain the actual operation index data of each video segment obtained by video encoding on the corresponding target computing node and the actual quality detection results corresponding to the video encoding results; Step S602: Calculate the prediction deviation data based on the actual operating indicator data, the actual quality inspection results, and the predicted coding cost; Step S603: Update the model parameters of the neural network model based on the prediction deviation data.

[0070] In step S601, after the distributed encoding module completes its encoding task, the dynamic feedback module obtains the actual operational metrics data of each video segment encoded on its corresponding target computing node, as well as the actual quality detection results corresponding to the video encoding results. The actual operational metrics data represent the real operational feedback from the video encoding system, which continuously monitors and records the consumption of underlying hardware resources and task time during the video transcoding task on the computing node. This includes actual encoding time, actual GPU usage, actual memory usage, actual output bitrate, encoding logs, task status, and anomaly information. The actual quality detection results represent the numerical values ​​used by objective video quality evaluation algorithms to measure the image quality of the final encoded video segments, including actual VMAF, SSIM, or PSNR, and other experience quality metrics.

[0071] In some embodiments, obtaining the actual operational metrics data of each video segment obtained by video encoding on the corresponding target computing node and the actual quality detection results corresponding to the video encoding results includes: The actual video encoding time, actual resource consumption, and actual output bitrate of the video segments are obtained by performing video encoding on the target computing node, and actual operation index data are generated based on the actual video encoding time, actual resource consumption, and actual output bitrate. The video encoding results are evaluated for video quality to obtain a video quality score, and the corresponding actual quality detection result is determined based on the video quality score.

[0072] In this embodiment, the dynamic feedback module acquires the actual video encoding time, actual resource consumption, and actual output bitrate of the video segments encoded on the target computing node. Specifically, after a computing node in the distributed encoding cluster completes a specific segment encoding task, the video encoding system can collect the actual execution status of the video task. The actual video encoding time represents the time taken to encode the segment. The video segment in the first The actual encoding time on a target computing node refers to the time elapsed from when the target computing node receives and begins processing the video segment until the corresponding encoded result is generated. Actual resource usage data represents the statistical value of the actual consumption of underlying hardware resources by the video encoding task during its execution on the target computing node. This can include hardware load parameters such as actual GPU usage and actual memory usage during processing. Simultaneously, the actual output bitrate represents the actual data volume of the video segment file output by the video encoder per unit time, reflecting the actual data output rate of the compressed video segment. Auxiliary information such as encoding logs, task status, and exception information can also be obtained. This information is used in subsequent analysis to determine whether there are abnormal interruptions, resource contention, or hardware failures during the encoding process. Thus, the above information can be integrated to obtain actual operational performance data.

[0073] The dynamic feedback module evaluates the video encoding results output by the distributed coding module, obtaining a video quality score, and determines the corresponding actual quality detection result based on the video quality score. The video quality evaluation means that after completing hardware-level indicator collection, the video encoding system can use specific image quality evaluation algorithms to objectively measure the visual fidelity of the transcoded and compressed video encoding results. To accurately reflect the true performance of the encoded image, the video encoding system typically performs evaluations based on actual indicators such as VMAF, SSIM, or PSNR. By executing these evaluation algorithms, the video encoding system can calculate specific quantitative values ​​reflecting the current image's sharpness, distortion level, or structural retention, obtaining a video quality score. Subsequently, the video encoding system can jointly determine the actual quality detection result for the corresponding video segment based on the video quality score and visual experience. Thus, the video encoding system is no longer limited to a single consideration of computing speed and hardware resources, but also incorporates the actual visual performance of the final delivered image into the feedback system.

[0074] In step S602, the dynamic feedback module calculates the prediction deviation data based on actual operational indicator data, actual quality inspection results, and predictive coding cost. The predictive coding cost represents the estimated cost output by the AI ​​prediction module before the video coding task is issued, based on the content complexity characteristics of video segments and the resource capabilities of candidate nodes. The video coding system compares the actual consumption and image quality performance with the previously estimated values ​​item by item to quantify the prediction error. Taking predictive coding time as an example, the video coding system calculates the video segment time using the following formula. At the target computing node Prediction time bias: ,in, This represents the prediction error calculated by the video coding system. This represents the actual encoding time of video segments on the computing nodes. This indicates the time taken for the AI ​​prediction module to output the prediction code in advance.

[0075] when When the value is positive, it indicates that the AI ​​prediction model underestimates the encoding time of the task, meaning the actual time is longer than the predicted value; when... A negative value indicates an overestimation bias, meaning the actual time taken is shorter than the predicted time. Similarly, video coding systems can calculate the GPU resource prediction bias between actual and predicted GPU usage, the memory resource prediction bias between actual and predicted memory usage, the bitrate prediction bias between actual and predicted output bitrate, and the image quality prediction bias between actual VMAF, SSIM, PSNR metrics and predicted image quality loss.

[0076] By calculating the prediction deviation data from the above multi-dimensional perspective, the video coding system can quantify the prediction accuracy of the AI ​​prediction model under different video complexity features and different node capability features, providing a quantitative basis for subsequent model parameter updates.

[0077] In step S603, the dynamic feedback module can update the model parameters of the neural network model based on the calculated prediction deviation data. The neural network model has been introduced in the previous steps and will not be repeated here. The video coding system uses the prediction deviation data (such as time error, resource estimation error, etc.) as error feedback signals for model training and optimization. Through machine learning update algorithms such as gradient descent and backpropagation, the weight parameters and bias parameters of the neural network model are adjusted, so that the model can reduce the prediction deviation of similar video segments on similar target computing nodes in subsequent predictions.

[0078] Furthermore, the video coding system also uses prediction deviation data to synchronously update the node capability profiles and scheduling weights of computing nodes. For example, when the actual performance of a computing node is found to be consistently lower than the theoretical estimate, the video coding system will revise the capability rating of the corresponding computing node. When a target computing node consistently exhibits prediction deviations in the same direction across multiple video segment coding tasks, such as when the actual coding time of a computing node is consistently longer than the predicted value, it indicates that the node's true coding capability is lower than the capability estimate in the current profile. The dynamic feedback module accordingly lowers the value of the GPU coding capability component in the node's capability vector, making the node capability profile more accurately reflect the actual processing capability of the computing node. In addition, the dynamic feedback module can also fine-tune the weight parameters of the scheduling objective function in the scheduling optimization module based on the distribution characteristics of the prediction deviation data. For example, when the overall prediction time deviation is large, the weight of the predicted coding time term can be appropriately increased to strengthen the impact of coding time on the scheduling score, thereby making the scheduling strategy more adaptable to the actual coding environment. Through the joint update of the above model parameters, node capability profiles, and scheduling weights, the system forms a dynamic feedback closed loop, ensuring that the subsequent video segment coding cost prediction and task scheduling results continuously approach the true optimal solution.

[0079] This embodiment solves the problem of traditional static scheduling schemes failing to perceive actual execution effects and causing scheduling drift by continuously collecting underlying real-world operational data and objective video quality evaluation results, and then comparing them with previously generated predicted cost data for error correction. Thus, this embodiment endows the video coding system with adaptive evolution and learning optimization capabilities, enabling it to continuously optimize internal model parameters and node resource profiles based on real coding results. This allows it to continuously improve prediction accuracy and scheduling rationality when facing constantly changing heterogeneous computing environments and complex and diverse video content, ultimately achieving the core objectives of improving overall computing resource utilization and ensuring a high-quality viewing experience for users.

[0080] In some embodiments, please refer to Figure 7 , Figure 7This demonstrates the process of video distributed coding scheduling and its internal feedback closed-loop control mechanism. The video coding system first receives the input raw video data, then divides it into multiple independent video segments through keyframe detection and adaptive slicing. Subsequently, content features are extracted from each video segment, and the real-time resource status of the computing nodes is simultaneously acquired. Based on the video content features and resource status data, an AI prediction module predicts the processing cost of the coding task, thereby generating the optimal scheduling decision and distributing the task to the corresponding target computing node for distributed coding. Finally, all encoded segments are merged and spliced ​​together for output. Simultaneously, a real-time detection mechanism is introduced in the distributed coding stage. The video coding system continuously determines whether there are any unfinished video segments: if all segments have been processed (i.e., the determination is no), the process ends; if the coding task is not yet fully completed (i.e., the determination is yes), the video system sends the actual execution feedback data of the currently completed video segments to the result feedback and model update module (i.e., the aforementioned dynamic feedback module). Figure 7 As shown by the dashed line, the result feedback and the updated data generated by the model update module are fed back in reverse, dynamically correcting the parameters and strategies of the preceding stages such as keyframe detection, adaptive slicing, AI prediction, and scheduling decisions in real time. In this way, the video encoding system can continuously optimize the scheduling and slicing accuracy of subsequent video segments by utilizing the latest generated real feedback throughout the lifecycle of the same video processing task, thereby achieving adaptive intelligent encoding scheduling.

[0081] Please see Figure 8 This application also provides a video encoding apparatus 800 that can implement the above-described video encoding method. The apparatus includes: The acquisition unit 810 is used to acquire the raw video data to be encoded and to acquire the resource status data of each candidate computing node in the target node cluster. The segmentation unit 820 is used to determine the valid keyframes in the original video data and to segment the original video data according to the valid keyframes to obtain video segments. The generation unit 830 is used to calculate the video complexity corresponding to each video segment, and call the preset neural network model to generate the predicted coding cost of each video segment to perform video coding tasks on the corresponding candidate computing nodes based on the video complexity and resource status data. The first determining unit 840 is used to obtain the user experience quality index of each video segment, and determine the target computing node corresponding to each video segment from the candidate computing nodes based on the predicted coding cost and the user experience quality index. The second determining unit 850 is used to call the target computing node to perform video encoding on the corresponding video segments, obtain the corresponding video encoding results, and determine the target video encoding result corresponding to the original video data based on the video encoding results of all video segments.

[0082] The specific implementation of this video encoding device is basically the same as the specific implementation of the video encoding method described above, and will not be repeated here.

[0083] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described video encoding method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0084] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the video encoding method of the embodiments of this application. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0085] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described video encoding method.

[0086] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0087] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0088] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0089] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0090] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0091] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0092] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0094] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0095] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0096] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A video encoding method, characterized in that, The method includes: Obtain the raw video data to be encoded, and obtain the resource status data of each candidate computing node in the target node cluster; Valid keyframes are identified in the original video data, and the original video data is divided according to the valid keyframes to obtain video segments; Calculate the video complexity corresponding to each video segment, and call a preset neural network model to generate the predicted encoding cost of each video segment performing video encoding tasks on the corresponding candidate computing node based on the video complexity and the resource status data; Obtain the user experience quality index for each video segment, and determine the target computing node corresponding to each video segment from the candidate computing nodes based on the predicted encoding cost and the user experience quality index. The target computing node is invoked to perform video encoding on the corresponding video segments to obtain the corresponding video encoding results, and the target video encoding result corresponding to the original video data is determined based on the video encoding results of all the video segments.

2. The method according to claim 1, characterized in that, After determining the target video encoding result corresponding to the original video data based on the video encoding results of all the video segments, the method further includes: Obtain the actual operational index data of each video segment obtained by video encoding on the corresponding target computing node, and the actual quality detection results corresponding to the video encoding results; The prediction deviation data is calculated based on the actual operating indicator data, the actual quality inspection results, and the prediction coding cost. The model parameters of the neural network model are updated based on the prediction bias data.

3. The method according to claim 2, characterized in that, The step of obtaining the actual operational index data of each video segment obtained by video encoding on the corresponding target computing node and the actual quality detection results corresponding to the video encoding results includes: The actual video encoding time, actual resource consumption data, and actual output bitrate of the video segment are obtained by performing video encoding on the target computing node, and actual operation index data are generated based on the actual video encoding time, the actual resource consumption data, and the actual output bitrate. The video encoding results are evaluated for video quality to obtain a video quality score, and the corresponding actual quality detection result is determined based on the video quality score.

4. The method according to claim 1, characterized in that, The step of determining valid keyframes in the original video data includes: Candidate internal coded frames and frame group structure parameters are determined in the original video data, and scene switching positions in the original video data are determined. Within the first preset search time window corresponding to the scene switching position, target internal encoded frames are filtered from the candidate internal encoded frames based on the integrity constraints of the screen group structure parameters, and valid keyframes are determined based on the target internal encoded frames.

5. The method according to claim 4, characterized in that, The step of dividing the original video data according to the effective keyframes to obtain video segments includes: The motion intensity data, image entropy data, and scene change feature data of the original video data are obtained, and a comprehensive complexity score is calculated based on the motion intensity data, the image entropy data, and the scene change feature data. The target slice length is calculated based on the comprehensive complexity score, and the corresponding target slice time is determined based on the current starting time and the target slice length. The target slice boundary is determined based on the valid keyframes within a second preset search time window near the target slice time, and the original video data is divided according to the target slice boundary to obtain video segments.

6. The method according to claim 1, characterized in that, The calculation of the video complexity corresponding to each video segment includes: Obtain the space complexity feature of each video segment and the time complexity feature of each video segment; Obtain the encoding structure feature data of each video segment; The corresponding video complexity is generated based on the spatial complexity feature, the time complexity feature, and the coding structure feature data.

7. The method according to claim 1, characterized in that, The step of determining the target video encoding result corresponding to the original video data based on the video encoding results of all the video segments includes: Obtain the start display time, end decoding time, and screen group boundary identifier corresponding to each video encoding result, and obtain the segment number sequence corresponding to the video segment; Based on the segment numbering order, the start display time, the end decoding time, and the screen group boundary identifier, time offset correction is performed on all the video encoding results, and a target video encoding result is generated based on all the time offset corrected video encoding results.

8. A video encoding device, characterized in that, The device includes: The acquisition unit is used to acquire the raw video data to be encoded and to acquire the resource status data of each candidate computing node in the target node cluster. A segmentation unit is used to determine valid keyframes in the original video data and to segment the original video data according to the valid keyframes to obtain video segments. The generation unit is used to calculate the video complexity corresponding to each video segment, and call a preset neural network model to generate the predicted encoding cost of each video segment performing video encoding tasks on the corresponding candidate computing node based on the video complexity and the resource status data. The first determining unit is used to obtain the user experience quality index of each video segment, and determine the target computing node corresponding to each video segment from the candidate computing nodes according to the predicted coding cost and the user experience quality index. The second determining unit is used to call the target computing node to perform video encoding on the corresponding video segments to obtain the corresponding video encoding results, and to determine the target video encoding result corresponding to the original video data based on the video encoding results of all the video segments.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the video encoding method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the video encoding method according to any one of claims 1 to 7.

Images