Distributed video decoding output method and system

Abstract

The invention provides a distributed video decoding output method and system. The method comprises the following steps: collecting video stream data, and obtaining a decoded video clip; splicing the decoded video clips according to a preset splicing rule to generate a first video output model; determining a time axis position corresponding to at least one video clip matched with the features of the video clip to obtain a matching result, and establishing a corresponding synchronization model between the video clip and the first video output model; determining a time transformation matrix of the video clip relative to the first video output model according to the synchronization model; generating a distributed video decoding output model according to the time transformation matrix; and decoding the target video stream data through the distributed video decoding output model, and outputting a corresponding video clip, decoding data corresponding to the video clip and a splicing structure generated according to the video clip. According to the technical scheme provided by the invention, the video decoding efficiency can be remarkably improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video processing technology, and in particular to a distributed video decoding output method and system. Background Art

[0002] With the rapid development of Internet technology and multimedia content, video streaming services have become an indispensable part of people's daily lives. Whether it is online education, remote conferencing or entertainment live broadcasting, efficient and stable video decoding technology is required to support high-definition and smooth video playback experience. In the above application scenarios, the real-time, stability and picture quality of video streaming are key factors in user experience. Especially in the case of large-scale concurrent viewing or poor network conditions, how to ensure the efficiency and quality of video decoding has become an important technical challenge. In addition, in distributed systems, due to factors such as network delay and server load changes, traditional centralized decoding methods are difficult to meet the needs of modern video services.

[0003] At present, common video decoding solutions mainly rely on a single decoder to complete the decoding of the entire video stream. Although this method is simple and easy to implement, it has obvious shortcomings when facing high traffic and high concurrency. Summary of the invention

[0004] The embodiments of the present application provide a distributed video decoding output method and system to solve the problem of poor video decoding efficiency in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a distributed video decoding output method, including:

[0006] Collecting video stream data, and dividing the video stream data into multiple data blocks;

[0007] Allocating the multiple data blocks to multiple decoding nodes for parallel decoding to obtain decoded video segments, wherein the transmission order of the multiple data blocks can be dynamically adjusted according to the decoding progress of the multiple decoding nodes;

[0008] After the multiple decoding nodes complete decoding, splicing the decoded video segments according to a preset splicing rule to generate a first video output model;

[0009] In the first video output model, determining a time axis position corresponding to at least one video segment matching the features of the video segment, obtaining a matching result, and establishing a synchronization model corresponding to the video segment and the first video output model based on the matching result, wherein the synchronization model is used to indicate a nonlinear time offset relationship between the video segment and the first video output model;

[0010] determining a time transformation matrix of the video segment relative to the first video output model according to the synchronization model;

[0011] According to the time transformation matrix, transform the corresponding time coordinates in the video segment to the time axis position corresponding to the video segment in the first video output model to generate a distributed video decoding output model, wherein the distributed video decoding output model at least includes the video segment, the decoding data corresponding to the video segment, and the splicing structure generated according to the video segment;

[0012] The acquired target video stream data is input into the distributed video decoding output model so as to decode the target video stream data through the distributed video decoding output model, and output the corresponding video segments, the decoding data corresponding to the video segments and the splicing structure generated according to the video segments.

[0013] In a second aspect, an embodiment of the present application provides a distributed video decoding output system, including:

[0014] An acquisition module, used for acquiring video stream data and dividing the video stream data into multiple data blocks;

[0015] An adjustment module, configured to distribute the plurality of data blocks to a plurality of decoding nodes for parallel decoding to obtain decoded video segments, wherein the transmission order of the plurality of data blocks can be dynamically adjusted according to the decoding progress of the plurality of decoding nodes;

[0016] A generating module, configured to, after the plurality of decoding nodes complete decoding, splice the decoded video segments according to a preset splicing rule to generate a first video output model;

[0017] a synchronization module, configured to determine, in the first video output model, a time axis position corresponding to at least one video segment matching the features of the video segment, obtain a matching result, and establish, based on the matching result, a synchronization model corresponding to the video segment and the first video output model, wherein the synchronization model is used to indicate a nonlinear time offset relationship between the video segment and the first video output model;

[0018] a determination module, configured to determine a time transformation matrix of a video segment relative to the first video output model according to the synchronization model;

[0019] an output module, configured to transform the time coordinates corresponding to the video segment to the time axis position corresponding to the video segment in the first video output model according to the time transformation matrix, so as to generate a distributed video decoding output model, wherein the distributed video decoding output model at least includes the video segment, the decoding data corresponding to the video segment, and the splicing structure generated according to the video segment;

[0020] A processing module is used to input the acquired target video stream data into the distributed video decoding output model so as to decode the target video stream data through the distributed video decoding output model, and output corresponding video segments, decoding data corresponding to the video segments, and a splicing structure generated according to the video segments.

[0021] In a third aspect, a computing device is provided in an embodiment of the present application, comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a distributed video decoding output method as described in any one of the first aspects above.

[0022] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a distributed video decoding and output method as described in any one of the first aspects above.

[0023] In an embodiment of the present application, video stream data is collected and divided into multiple data blocks; the multiple data blocks are distributed to multiple decoding nodes for parallel decoding to obtain decoded video segments, wherein the transmission order of the multiple data blocks can be dynamically adjusted according to the decoding progress of the multiple decoding nodes; after the multiple decoding nodes complete decoding, the decoded video segments are spliced ​​according to a pre-set splicing rule to generate a first video output model; in the first video output model, the time axis position corresponding to at least one video segment matching the features of the video segment is determined to obtain a matching result, and based on the matching result, a corresponding synchronization model between the video segment and the first video output model is established, wherein the synchronization model is used to indicate the time axis position of the video segment and the first video output model. The nonlinear time offset relationship between the video output models is determined according to the synchronization model; the time transformation matrix of the video segment relative to the first video output model is determined according to the synchronization model; according to the time transformation matrix, the corresponding time coordinates in the video segment are transformed to the time axis position corresponding to the video segment in the first video output model to generate a distributed video decoding output model, and the distributed video decoding output model at least includes a video segment, decoding data corresponding to the video segment, and a splicing structure generated according to the video segment; the obtained target video stream data is input into the distributed video decoding output model to decode the target video stream data through the distributed video decoding output model, and the corresponding video segment, the decoding data corresponding to the video segment, and the splicing structure generated according to the video segment are output. The technical solution provided by the present application can significantly improve the efficiency of video decoding and the stability of the system.

[0024] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A flowchart of a distributed video decoding output method provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of the structure of a distributed video decoding output system provided in an embodiment of the present application;

[0028] Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0030] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to different types.

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0032] Figure 1 A flowchart of a distributed video decoding output method is provided for an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0033] 101. Collect video stream data, and divide the video stream data into multiple data blocks;

[0034] Optionally, the step of dividing the video stream data into a plurality of data blocks in step 101 includes: identifying key frames and non-key frames in the video stream data; and dividing the video stream data into a plurality of data blocks according to position information of the key frames.

[0035] In this step, video stream data refers to a digital signal composed of a series of continuous image frames, which is usually used to achieve real-time or on-demand video playback. A key frame (I frame) refers to a data frame that is completely independent of other frames during the video encoding process. It contains all the information required to display the frame. A non-key frame (P frame / B frame) is a frame type that relies on the previous and next key frames or other non-key frames for decoding. A P frame (Predictive Frame) only contains change information relative to the previous frame, while a B frame (Bi-predictive Frame) may refer to information from both the previous and next frames.

[0036] In order to perform video decoding more efficiently, especially for distributed video decoding, it is very important to split the video stream data into data blocks containing key frames. This ensures that each decoding node can start decoding from the key frame, thereby avoiding dependency problems in the decoding process and increasing the possibility of parallel decoding.

[0037] In the embodiment of the present application, it is assumed that we have a video stream data from a network camera, which contains 30 frames per second (FPS), wherein every 10 frames contain a key frame. The length of the video stream data is 3 minutes. First, we need to identify the key frame positions in the video stream. For example, the first key frame appears in the first frame (frame 1) of the 1st second, the second key frame appears in the 11th frame of the 11th second, and so on.

[0038] Next, based on the position information of these key frames, we divide the video stream data into multiple data blocks. Each data block contains a key frame and its subsequent non-key frames until the next key frame. For example, the first data block will contain frames 1 to 10, the second data block contains frames 11 to 20, and so on. The advantage of this is that each data block can be decoded independently because each data block contains a complete image information (ie, a key frame) and other frames that depend on the key frame (ie, non-key frames).

[0039] To further illustrate, assuming the total size of the video stream data is 30MB, the approximate size of each data chunk will be 1MB (considering that the size of the data chunk will vary depending on the number of keyframes and non-keyframes). If we have three decoding nodes, we can split the video stream data into three data chunks and send them to different decoding nodes for parallel processing.

[0040] For example, the segmentation and distribution of video stream data is as follows:

[0041] Data block 1: Frames 1 to 10 (about 1MB), allocated to decoding node A;

[0042] Data block 2: Frames 11 to 20 (about 1MB), allocated to decoding node B;

[0043] Data block 3: Frames 21 to 30 (about 1MB), allocated to decoding node C;

[0044] In this way, we can achieve effective segmentation of video stream data and utilize multiple decoding nodes to speed up the video decoding process.

[0045] 102. Allocate the multiple data blocks to multiple decoding nodes for parallel decoding to obtain decoded video segments;

[0046] Wherein, the transmission order of the multiple data blocks can be dynamically adjusted according to the decoding progress of the multiple decoding nodes;

[0047] In this step, multiple data blocks refer to independent segments segmented from the video stream according to key frames, and each data block contains at least one key frame and subsequent non-key frames. Decoding nodes refer to computing units that can receive data blocks and decode them. These nodes can be processes on physical servers or virtual machine instances in a cloud computing environment. Parallel decoding refers to sending video data blocks to multiple decoding nodes for processing at the same time to speed up the overall video decoding. Dynamic adjustment of the transmission order means that the system can change the data block allocation strategy in real time according to the workload of each decoding node (i.e., decoding progress) to optimize resource utilization, reduce waiting time and improve efficiency.

[0048] In the embodiment of the present application, it is assumed that there are four decoding nodes (decoding nodes A, B, C, D) and ten data blocks (data blocks 1 to 10), each of which contains a key frame and several non-key frames. The video stream data has been divided into multiple data blocks in the above manner and is ready for parallel decoding.

[0049] First, during the system initialization phase, the status of each decoding node is set to available, and all data blocks can be allocated. The system starts to allocate data block 1 to decoding node A, data block 2 to decoding node B, data block 3 to decoding node C, and data block 4 to decoding node D. Since the decoding speed may vary depending on the performance of the node, the system will continuously monitor the decoding progress of each decoding node.

[0050] Assume that after a period of time, decoding node A has completed the decoding of data block 1, while other nodes are still processing their own data blocks. At this time, the system will detect that decoding node A is idle, and according to the dynamic adjustment strategy, the system will assign the next unassigned data block 5 to decoding node A instead of waiting for other nodes to complete before continuing to assign tasks. This approach ensures that decoding node A will not be idle, thereby maximizing resource utilization.

[0051] As time goes by, other decoding nodes also complete the decoding of their own data blocks. The system will dynamically adjust the allocation of remaining data blocks according to the latest decoding progress to ensure that all decoding nodes can stay as busy as possible. This dynamic adjustment mechanism helps balance the load between nodes and improve the decoding efficiency of the entire system.

[0052] For example, if decoding node B completes decoding of data block 2 faster than expected, the data blocks originally allocated to other slower nodes may be reallocated to decoding node B to take advantage of its faster processing capabilities. In this way, the system can flexibly adjust the data block allocation strategy according to the actual decoding progress, ensuring an efficient and balanced video decoding process.

[0053] 103. After the multiple decoding nodes complete decoding, splice the decoded video segments according to a preset splicing rule to generate a first video output model;

[0054] Optionally, before splicing the decoded video segments according to the preset splicing rules to generate the first video output model in step 103, the steps include: discretizing each of the video segments into a plurality of video frames, wherein the plurality of video frames include corresponding frame numbers and timestamps; applying time synchronization and / or frame rate adjustment to the plurality of video frames to obtain a splicing order of each of the video frames; and generating a splicing rule of the video segments relative to the first video output model according to the splicing order of the plurality of video frames. In this step, "a plurality of decoding nodes" refers to decoder instances distributed at different physical locations or running on different computing resources, which process different parts of the video stream in parallel. "Preset splicing rules" define how the decoded video segments are reassembled in sequence to form a continuous video output. "The first video output model" refers to a reassembled video stream model, which should maintain the temporal continuity and visual fluency of the video. "Discretizing a video segment into a plurality of video frames" means splitting the video segment into a series of separate image frames, each of which has its own unique frame number and timestamp for subsequent processing. Time Synchronization ensures that all frames are in the correct time sequence, even in a multi-node environment. Frame Rate Adjustment may need to adjust the frame rate according to the requirements of the video playback device or network conditions to ensure smooth video playback. Stitching Order specifies the order in which video frames are reassembled according to their position on the timeline.

[0055] In the embodiment of the present application, consider a scenario where a video stream is divided into several segments, each of which is assigned to a different decoding node for decoding. After decoding, the video segments need to be spliced ​​together to form a complete video output. Assume that there are three video segments, which are decoded by three decoding nodes respectively, and each segment contains multiple video frames.

[0056] Assume that the first video clip contains frames numbered 1 to 10, the second video clip contains frames numbered 11 to 20, and the third video clip contains frames numbered 21 to 30. Each frame has a corresponding timestamp, for example, the timestamp of frame 1 is T1, the timestamp of frame 2 is T2, and so on.

[0057] Before stitching the video segments together, the system further processes each video segment. First, each video segment is discretized into multiple video frames. Then, time synchronization is applied to these video frames to ensure that the video frames can be arranged in their original time order even in the case of network delays or differences in decoding speeds. If it is found that some frames arrive late due to network reasons, the system can compensate for the time difference through interpolation or other methods.

[0058] In addition, you may need to adjust the frame rate to ensure that the frame rate of the output video meets the requirements of the playback device. For example, if the maximum frame rate supported by the target playback device is 30fps (Frames Per Second), you need to ensure that the frame rate of the spliced ​​video does not exceed this value. If the frame rate of the decoded video clip is higher than 30fps, you can adjust the frame rate by discarding some frames or repeating some frames.

[0059] Finally, the first video output model is generated according to the splicing order of the video frames. For example, for the frames in the above example, the splicing order will be frame 1 to frame 30 in ascending order. In this way, it can be ensured that the final output video is a continuous and smooth video stream. If frame loss is found during the splicing process, the system should be able to automatically make up for the missing frames to maintain the integrity of the video.

[0060] Optionally, the step 103 of applying time synchronization and / or frame rate adjustment to the multiple video frames to obtain the splicing order of each of the video frames includes: calculating the relative displacement vector and instantaneous speed between the multiple video frames; adjusting the speed or time position of each of the video frames based on the relative displacement vector and the instantaneous speed to ensure that all the video frames are aligned on the time axis; and determining the splicing order of each of the video frames according to each of the adjusted video frames;

[0061] The step of calculating the relative displacement vector and instantaneous speed between the plurality of video frames includes:

[0062] Set video frame i at time t i The position of the i ), video frame j at time t j The position of the j ), the relative displacement between video frames i and j can be calculated as follows:

[0063] ΔP ij =P(i, t i )-P(j,t j )

[0064] Among them, ΔP ij represents the relative displacement vector between frame i and frame j;

[0065] Define the instantaneous velocity v(i) of video frame i as:

[0066]

[0067] Among them, Δt is the difference between adjacent timestamps, P(i, t i +1) is the video frame i at time t i +1 for the location.

[0068] In this step, "relative displacement vector" refers to the relative movement distance and direction between two video frames in space. "Instantaneous speed" refers to the instantaneous movement rate of the video frame at a certain moment, that is, the change in position per unit time. "Time synchronization" here means adjusting the speed or time position of the video frame so that the video frames from different decoding nodes can be aligned on the time axis, thereby ensuring the continuity and smoothness of the video playback. The "splicing order" is determined based on the adjusted video frame position, ensuring that the video frames can be spliced ​​together in the correct time order.

[0069] In the embodiment of the present application, it is assumed that we have a video stream divided into several segments, each of which contains multiple video frames. For simplicity, we assume that the video segment contains two video frames, namely video frame i and video frame j. We need to calculate the relative displacement vector and instantaneous speed between the two video frames, and adjust the speed or time position of the video frames accordingly to ensure that they are aligned on the timeline.

[0070] Assume that video frame i is at time t i The position of the i ), video frame j at time t j The position of the j ), where the position can be a position identifier of the video frame in the video stream, such as a frame number, etc. We calculate the relative displacement vector between video frames i and j

[0071] ΔP ij =P(i, t i )-P(j,t j )

[0072] In this example, if we assume that P(i, t i )=(2,3) and P(j,t j )=(4,1), then ΔP ij =(2-4,3-1)=(-2,2).

[0073] Next, define the instantaneous velocity v(i) of video frame i as:

[0074]

[0075] Assume that the difference between adjacent timestamps Δt=1, and video frame i is at time t i +1 position:

[0076] P(i,t i +1)=(3,4), then

[0077] Based on the above calculations, we can adjust the speed or time position of video frames i and j to ensure that they are aligned on the timeline. For example, if the speed of video frame i is faster than video frame j, we may slightly delay the display time of video frame i, and vice versa. Once the adjustment is completed, we can determine the splicing order of video frames i and j so that they are displayed in the correct time order.

[0078] It should be noted that the examples here are simplified for the purpose of illustrating the concept. In actual applications, the position P may be multi-dimensional, and the adjustment of the video frame may be more complicated, involving more video frames and more accurate timestamps. However, the basic principle is the same, that is, by calculating the relative displacement and instantaneous velocity, and then adjusting the position or velocity of the video frame to achieve the purpose of time synchronization.

[0079] Optionally, the step 103 of generating a splicing rule of the video clip relative to the first video output model according to the splicing order of the plurality of video frames includes: calculating the time interval and visual consistency between each pair of video frames in the plurality of video frames, and determining a matching degree function between each pair of video frames according to the time interval and the visual consistency; calculating the instantaneous speed between each pair of video frames in the plurality of video frames, and evaluating the smooth transition between each pair of video frames by a preset weighted average speed; determining a splicing score between each pair of video frames based on the matching degree function and the smooth transition; sorting based on the splicing scores, and constructing a splicing sequence to generate a splicing rule;

[0080] The step of calculating the time interval and visual consistency between each pair of video frames in the plurality of video frames, and determining a matching degree function between each pair of video frames according to the time interval and the visual consistency, comprises:

[0081] Based on the inter-frame correlation and temporal continuity of the video frames, a matching function M(i, j) between video frames i and j is defined, and the matching function is used to indicate the time interval Δt between the two frames. ij and visual consistency ΔV ij , wherein the matching degree function is expressed as:

[0082] M(i, j) = w t ·Δt ij +w v ΔV ij

[0083] Among them, w t and w v are the weight coefficients of time interval and visual consistency, respectively;

[0084] The calculating the instantaneous speed between each pair of video frames in the plurality of video frames and evaluating the smooth transition between each pair of video frames by a preset weighted average speed comprises:

[0085] By introducing the weighted average velocity v avg (i, j) is used to measure the smooth transition between video frames i and j, where the weighted average speed is expressed as:

[0086] In this step, "time interval" refers to the distance between two video frames on the timeline, usually expressed in time units (such as seconds). "Visual consistency" refers to the similarity of the visual content between two adjacent video frames, which can be quantified by comparing the difference in pixel values ​​or other features (such as edges, textures, etc.) between the two frames. The "matching degree function" is a function that combines time interval and visual consistency, and is used to evaluate whether two frames of video should be spliced ​​together. "Smooth transition" refers to the visual continuity when switching between video frames, which affects the quality of the final video. "Weighted average speed" is an indicator used to evaluate the smooth transition between video frames, which reflects the average movement speed of two frames of video in time.

[0087] In the present embodiment, assume that we have two video frames i and j, and we need to calculate the matching function M(i, j) between the two video frames and evaluate the smooth transition between them. First, we calculate the time interval Δt between video frames i and j ij and visual consistency ΔV ij Assuming Δt ij is the time difference between video frames i and j, such as Δt ij = 0.033 seconds (assuming the frame rate is 30fps). Visual consistency ΔV ij It can be obtained through some image similarity algorithm, such as ΔV ij =0.85 (assuming that the closer this value is to 1, the more similar the two frames are).

[0088] The matching function M(i, j) is defined as

[0089] M(i, j) = w t ·Δt ij +w v ΔV ij

[0090] Among them, w t and w v are the weight coefficients of time interval and visual consistency respectively. If we set w t =0.6 and w v=0.4, then M(i, j)=0.6·0.033+0.4·0.85=0.3418. This represents the degree of match between video frames i and j.

[0091] Next, we calculate the instantaneous velocities v(i) and v(j) between video frames i and j and calculate the weighted average velocity v by avg (i, j) to evaluate the smooth transition between them. Assuming v(i) = 2px / s, v(j) = 3px / s, then This value represents the average speed at which video frames i and j move in time.

[0092] Finally, we use the matching function M(i, j) and the smooth transition v avg (i, j) to determine the splicing score between each pair of video frames. The splicing score can be simply understood as a comprehensive evaluation, which may combine M(i, j) and the effect of smooth transition to determine the priority of video frame splicing. Sorting based on the splicing score can construct the optimal splicing sequence, thereby generating splicing rules.

[0093] For example, if we have two video frames i and j, and another pair of video frames k and l, we compute M(i, j) and M(k, l), respectively, and the corresponding v avg (i, j) and v avg (k, l), and then determine the splicing score of each pair of video frames based on these values. Finally, the video frame pair with the highest splicing score is selected as part of the splicing sequence, and so on, until all video frames are reasonably sorted. In this way, we get the splicing rules of video clips relative to the first video output model.

[0094] Optionally, the calculating the time interval and visual consistency between each pair of video frames in the plurality of video frames, and determining a matching degree function between each pair of video frames according to the time interval and the visual consistency, further includes:

[0095] Calculate the motion vector MV of each video frame ij ;

[0096] According to the time interval Δt ij , the visual consistency ΔV ij , the motion vector MV ij , determine the matching degree function M′(i, j) between each pair of video frames;

[0097] Wherein, the matching degree function M′(i, j) is expressed as:

[0098] M′(i, j)=w t ·Δtij +w v ΔV ij +w m ·∥∥MV ij ∥∥+w s ∥v avg (i, j)∥

[0099] Among them, w m is the weight coefficient of the motion vector, ∥∥MV ij ∥∥ represents the average motion vector magnitude between video frames i and j; w s is the weight coefficient of average speed, ∥v avg (i, j) / / represents the average speed amplitude between video frames i and j.

[0100] In this step, "motion vector" refers to the vector that describes the direction and distance of movement of an object in a video frame. It is a commonly used concept in video encoding and analysis. "Motion vector magnitude" refers to the length of the vector, that is, the actual distance the object moves between frames. "Average speed magnitude" refers to the average speed of the object moving between two frames, which can help evaluate the smooth transition between video frames. The extended version of the "matching degree function" M'(i, j) is added by adding the motion vector MV ij and the average velocity amplitude ∥v avg (i, j)∥ to more comprehensively evaluate the matching degree between two frames.

[0101] In the present embodiment, assume that there are two video frames i and j, and we need to calculate the matching function M′(i, j) between them and evaluate the smooth transition between them. First, we calculate the time interval Δt between video frames i and j ij and visual consistency ΔV ij .

[0102] Assumptions:

[0103] Time interval Δt ij = 0.033 seconds (assuming the frame rate is 30fps)

[0104] Visual consistency ΔV ij =0.85 (assuming that the closer this value is to 1, the more similar the two frames are)

[0105] Next, we calculate the motion vector MV for each video frame ij Assuming MV ij Represents the average moving distance of the object from frame i to frame j, such as MV ij =(10px, 15px), its amplitude is

[0106] We then calculate the instantaneous velocities v(i) and v(j) between video frames i and j and calculate the weighted average velocity v by avg (i, j) to evaluate the smooth transition between them. Assume:

[0107] v(i)=2px / s

[0108] v(j)=3px / s

[0109] So:

[0110]

[0111] Average speed amplitude ∥∥v avg (i, j) ∥∥ = 2.5 px / s;

[0112] The matching degree function M′(i, j) is defined as:

[0113] M′(i, j)=w t ·Δt ij +w v ΔV ij +w m ·∥∥MV ij ∥∥+w s ·∥∥v avg (i, j)∥∥

[0114] Among them, w t is the weight coefficient of the time interval, w v is the weight coefficient of visual consistency, w m is the weight coefficient of the motion vector magnitude, w s is the weight coefficient of the average velocity amplitude.

[0115] Assume that the weight coefficients are:

[0116] w t =0.

[0117] w v =0.

[0118] w m =0.3

[0119] w s =0.

[0120] So:

[0121] M'(i, j)=0.2·0.033+0.3·0.85+0.3·18.03+0.2·2

[0122] M'(i, j)=0.0066+0.255+5.409+0.5

[0123] M′(i, j)=6.170

[0124] This represents the enhanced match between video frames i and j.

[0125] In this way, we can more comprehensively evaluate the matching degree between video frames and make more reasonable decisions in the video stitching process. In practical applications, the selection of weight coefficients may be adjusted according to different specific application scenarios.

[0126] Optionally, before the step 103 of splicing the decoded video segments according to a preset splicing rule to generate the first video output model, it further includes: determining whether the decoded video segments can be successfully spliced; if so, splicing the decoded video segments according to the preset splicing rule to generate the first video output model;

[0127] The step of determining whether the decoded video segments can be smoothly spliced ​​includes: introducing a nonlinear interpolation function NLF(i, j), wherein the nonlinear interpolation function is used to evaluate the visual discontinuity between the video frame i and the video frame j due to the time offset, and the nonlinear interpolation function is expressed as:

[0128] NLF(i, j) = exp(-α·(Δt ij ) 2 )

[0129] Among them, α is the regularization parameter;

[0130] If the result of multiplying the matching degree M′(i, j) of the video frame i and the video frame j by the nonlinear interpolation function NLF(i, j) is greater than the set threshold, it is confirmed that the video frames i and j can be successfully spliced.

[0131] In this step, we discuss a key link in the video splicing process - evaluating whether the decoded video segments can be smoothly spliced. In order to ensure that the video segments still maintain visual continuity and smoothness after splicing, it is necessary to evaluate the visual discontinuity caused by the time offset between adjacent video frames. For this purpose, a nonlinear interpolation function NLF(i, j) is introduced, which is used to quantify the degree of visual discontinuity caused by the time difference between video frame i and video frame j.

[0132] Concept explanation:

[0133] Nonlinear interpolation function NLF(i, j): This is a function used to measure the visual discontinuity caused by the time offset between video frames i and j. The "nonlinear" here means that the function is not a simple linear relationship, but uses an exponential form to simulate the perception characteristics of the human visual system for time delay. The form of the function is NLF(i, j) = exp(-α·(αt ij ) 2 ), where α is the regularization parameter, which determines the influence of the time difference on the function value, αt ij Represents the time interval between frame i and frame j.

[0134] Matching function M′(i, j): This is an indicator used to evaluate the similarity between video frame i and video frame j. It takes into account multiple factors, such as time interval, visual consistency, etc., to determine whether two frames are similar enough to be spliced.

[0135] Setting a threshold: In order to decide whether to splice video frames i and j, a threshold needs to be set as a judgment standard. If the value of M′(i, j)·NLF(i, j) is greater than this threshold, it is considered that the two frames can be successfully spliced; otherwise, it is considered that the splicing effect is not good and further processing or selection of other frames for splicing may be required.

[0136] In the present application embodiment

[0137] Suppose we have two video frames i and j, and we need to evaluate whether these two frames can be smoothly spliced ​​together. First, we calculate the time interval Δt between the two frames ij .

[0138] For example, assuming that the time interval between frame i and frame j is 0.033 seconds (assuming the frame rate is 30fps), and the regularization parameter α is set to 0.5, then the nonlinear interpolation function NLF(i, j) can be calculated as follows:

[0139] NLF(i, j) = exp(-α·(Δt ij ) 2 )=exp(-0.5·(0.033) 2 )≈exp(-0.0005445)

[0140] ≈0.999456

[0141] Next, assume that we have obtained the matching degree M′(i, j) between video frames i and j by other means (such as the formula mentioned above). Assume that M′(i, j)=6.1706 (refer to the previous example).

[0142] Now, we combine the matching degree function M′(i, j) with the nonlinear interpolation function NLF(i, j):

[0143] M′(i,j)·NLF(i,j)=6.1706·0.999456≈6.1702

[0144] If the threshold value is set to 5.0, then because 6.1702 is greater than 5.0, we can confirm that video frames i and j can be successfully spliced.

[0145] Through such steps, we can effectively evaluate and ensure the continuity and quality of the final output video before video stitching.

[0146] 104. In the first video output model, determine a time axis position corresponding to at least one video segment that matches the feature of the video segment, obtain a matching result, and establish a synchronization model corresponding to the video segment and the first video output model based on the matching result, wherein the synchronization model is used to indicate a nonlinear time offset relationship between the video segment and the first video output model;

[0147] In this step, the characteristics of the video clip usually refer to the content characteristics of the video, such as color, brightness, texture, motion vector, etc., and the timeline position refers to the starting point and the ending point of the video clip in time. The process of determining the matching of the characteristics of the video clip with the corresponding timeline position in the first video output model is actually to ensure that the video clip can be accurately placed at the correct position on the timeline when it is finally output, so as to ensure the coherence and integrity of the video content.

[0148] The synchronization model here refers to a mathematical model or algorithm that describes the nonlinear time offset relationship between the video clips and the first video output model. The nonlinear time offset relationship may be caused by different frame rates, encoding methods or transmission delays between the video clips. The role of the synchronization model is to eliminate this time deviation so that the video clips can be seamlessly connected in the final output model.

[0149] In the embodiment of the present application, it is assumed that we have a video project that contains multiple video clips obtained from different sources, each of which has its own characteristics, such as resolution, frame rate, color space, etc. Our goal is to synthesize these clips into a coherent first video output model. To achieve this goal, we need to first determine the timeline position of each video clip in the final output model.

[0150] First, we collect information about all video segments, including their features and timestamps. Suppose we have two video segments A and B, the timestamp range of segment A is from T1 to T2, and the timestamp range of segment B is from T3 to T4.

[0151] Next, we analyze the features of the video clips, such as motion vector analysis to know that clip A mainly shows an object moving from left to right, while clip B shows a static background. Then, we compare these features with the requirements of the first video output model and determine that clip A should start playing from the 5th second of the output model, while clip B should start playing from the 10th second.

[0152] In this process, we found that the actual playback time of segment A was a little faster than expected, while segment B was a little slower. Therefore, we need to build a synchronization model to adjust the time offset of the two segments to ensure that they can start playing accurately at the specified time point in the final output. Here, a simple linear interpolation function can be used to adjust the time offset, but considering the possible nonlinear time offset, we chose a more complex model.

[0153] Assume that the playback speed of video segment A is Va, and the playback speed of segment B is Vb. The speeds we need to adjust are ΔVa and ΔVb respectively. We can define a nonlinear time offset function to describe this relationship, such as:

[0154] T adjust (t) = α·t β

[0155] Among them, α and β are parameters adjusted according to the deviation between the actual playback time of the video segment and the expected playback time. By adjusting these parameters, the time axis of the video segment can be aligned with the time axis of the first video output model.

[0156] For example, if we find that the timeline of segment A is 1 second earlier than expected, and segment B is 1 second later, we can compensate for this difference by adjusting the parameters in the above formula. For example, set α = 1, β = 1.1 for segment A, and set α = 1, β = 0.9 for segment B, so that the synchronization effect can be achieved through nonlinear time stretching or compression.

[0157] In this way, we can ensure that each video clip can find the correct timeline position in the first video output model, and eliminate the impact of non-linear time offset through the synchronization model, making the video content more coherent and harmonious.

[0158] 105. Determine a time transformation matrix of the video segment relative to the first video output model according to the synchronization model;

[0159] In this step, the "time transformation matrix" is a mathematical tool used to describe the adjustment of video clips on the time axis. Through this matrix, the time transformation of video clips can be achieved, including but not limited to time stretching, compression, translation and other operations. The purpose is to make the time axis position of the video clips in the first video output model consistent with the expectation, so as to ensure the natural and smooth connection between video clips.

[0160] The temporal transformation matrix usually contains a series of parameters that determine how the video clips are adjusted. For example, a simple temporal transformation matrix can be represented as a linear transformation that can adjust the time of the video clip by a scaling factor and a time offset. In more complex cases, the temporal transformation matrix may involve nonlinear transformations to cope with more complex timing adjustment requirements.

[0161] In the embodiment of the present application, suppose we have a video segment A whose original time is T1 to T2, and we want to adjust the time of this video segment to T1' to T2' in the first video output model. To achieve this goal, we need to calculate a time transformation matrix based on the synchronization model to transform the time of video segment A from T1 to T2 to T1' to T2'.

[0162] First, we define the form of the time transformation matrix. Assuming that the time transformation of video clip A can be approximated by a linear transformation, we can define a time transformation matrix M, which contains two main parameters: scaling factor s and time offset d. The specific form is as follows:

[0163] M=(sd)

[0164] Where s is the scaling factor and d is the time offset.

[0165] Assume that we want to transform the time of video segment A from T1 to T2 to T1′ to T2′, and it is known that T1′ and T2′ are the new time start and end points determined according to the synchronization model. We can calculate the scaling factor s and the time offset d by the following formula:

[0166]

[0167] d = T1′-T1×s

[0168] Here, s represents the ratio by which video segment A needs to be stretched or compressed, and d represents the distance by which video segment A needs to be translated on the time axis. With these two parameters, we can transform the time of video segment A from T1 to T2 to T1′ to T2′.

[0169] For example, suppose the original time of video segment A is from the 10th second (T1=10) to the 20th second (T2=20), and we hope to adjust the time of this video segment to from the 15th second (T1′=15) to the 25th second (T2′=25) in the first video output model. The parameters calculated according to the above formula are:

[0170]

[0171] d = 15-10 × 1 = 5

[0172] This means that video segment A does not need to be stretched or compressed (s=1), but only needs to be shifted backward by 5 seconds (d=5). Therefore, the time transformation matrix M is:

[0173] M=(1 5)

[0174] By applying this time transformation matrix, video segment A will be transformed from the 10th to the 20th second to the 15th to the 25th second, and thus accurately placed at the correct timeline position in the first video output model.

[0175] The application of this time transformation matrix is ​​not limited to simple linear transformation, but can also be extended to nonlinear transformation, such as polynomial transformation, to adapt to more complex time offset relationships. In this way, it can be ensured that the time axis position of the video clip in the final output model is consistent with the expectation, thereby achieving seamless connection of video content.

[0176] 106. According to the time transformation matrix, transform the time coordinates corresponding to the video segment to the time axis position corresponding to the video segment in the first video output model to generate a distributed video decoding output model, wherein the distributed video decoding output model at least includes the video segment, the decoding data corresponding to the video segment, and the splicing structure generated according to the video segment;

[0177] In this step, "time coordinate transformation" refers to adjusting the time information (such as frame number, timestamp, etc.) in the video clip according to the previously calculated time transformation matrix so that the video clip can be correctly positioned in the first video output model. This step is to ensure that the video clip can be played in the expected time sequence, thereby ensuring the continuity and consistency of the video content. "Distributed video decoding output model" refers to the process of completing the video decoding task through the collaboration of multiple decoding nodes in a distributed system environment, and recombining the decoded video clips into a complete video stream in time sequence. This model includes at least the video clip itself, the decoded video data, and the splicing structure generated based on the video clip. The latter refers to the order and connection method of the video clips in the final output.

[0178] In the embodiments of the present application, assume that we have a video clip B with an original time from T1 to T2. After decoding, we need to transform its time coordinates to the correct time positions T1' to T2' in the first video output model according to the previously calculated time transformation matrix. Our goal is to create a distributed video decoding output model that can correctly splice the video clip B with other video clips in chronological order.

[0179] First, we need to ensure that each video clip has been decoded by different nodes in the distributed system. Each node is responsible for decoding a specific part of the video stream and transmitting the decoded data back to the master node. The master node receives all the decoded video clips and uses the previously mentioned time transformation matrix to adjust the time coordinates of each clip to ensure that they can be arranged in the correct chronological order in the final video output.

[0180] Assume that the original time of video clip B is from the 1st minute (T1 = 60 seconds) to the 2nd minute (T2 = 120 seconds), and according to the synchronization model and the time transformation matrix, we need to transform its time coordinates to the 2nd minute (T1' = 120 seconds) to the 3rd minute (T2' = 180 seconds). This means that the time of video clip B needs to be shifted backward by 1 minute (60 seconds). In actual operation, this may involve updating the metadata of video clip B to reflect the new start time and end time.

[0181] Next, the master node splices the video clip B with adjusted time coordinates with other video clips that have been decoded and had their time coordinates adjusted in chronological order. Assume that we also have another video clip C with adjusted time coordinates from T3' to T4', and T4' < T1', then clip C will be located before clip B.

[0182] Finally, the generated distributed video decoding output model will include all the video clips and their decoded data, as well as the splicing structure generated based on these video clips. In this way, a complete video output model is formed, in which all the video clips are correctly arranged according to their positions on the time axis in the first video output model.

[0183] For example, if we have three video clips A, B, and C, which are decoded and adjusted in time and spliced together in chronological order T1', T2', T3', then the final distributed video decoding output model will present as a coherent video stream, with each clip appearing at the correct time point, thus providing a seamless viewing experience for users.

[0184] 107. Input the acquired target video stream data into the distributed video decoding output model to decode the target video stream data through the distributed video decoding output model, and output corresponding video segments, decoding data corresponding to the video segments, and a splicing structure generated according to the video segments.

[0185] In this step, "target video stream data" refers to the video data stream to be decoded received from the source, which is usually transmitted in a compressed format, such as H.264 or H.265 (also known as HEVC, High Efficiency Video Coding). "Distributed video decoding output model" refers to a system consisting of multiple decoding nodes that work together to complete the decoding task of the video stream. The decoded video clips will contain decoded data, that is, the pixel information of the video and other auxiliary information, such as audio tracks, subtitles, etc. "Splicing structure" refers to the way the decoded video clips are reassembled in chronological order to ensure that the video clips can be played in the correct order to form a coherent video output.

[0186] In the embodiment of the present application, it is assumed that we have a video conferencing application that needs to decode the video stream from the remote user in real time. The video stream is transmitted in the H.265 (HEVC) format, which allows efficient data compression but also requires powerful decoding capabilities.

[0187] First, the client application receives the target video stream data, which is transmitted in real time from the remote user via the Internet or other network connections. The video stream data is divided into multiple small data packets, each of which represents a segment of the video.

[0188] These packets are then sent to the various decoding nodes in the distributed video decoding output model. To simplify the example, we assume that the model contains two decoding nodes: Node A and Node B. Each node is responsible for decoding a portion of the video stream. For example, Node A decodes the first half of the video clip, while Node B decodes the second half of the video clip.

[0189] The decoding process involves converting compressed video data into uncompressed video data so that it can be displayed by the terminal device. The decoded video clip contains the pixel information of the video, that is, the decoded data. In this process, some decoding algorithms and formulas may be involved, such as the hierarchical structure of CTU (Coding Tree Unit) and CU (Coding Unit) for decoding HEVC.

[0190] Once the video segments are decoded, they are reassembled in the time order they appeared in the original video stream to form a spliced ​​structure. For example, if Node A decoded the video segment from 1 to 5 seconds, and Node B decoded the video segment from 6 to 10 seconds, then during the reassembly phase, the two segments will be spliced ​​together in time order.

[0191] Finally, the distributed video decoding output model outputs the decoded video segments to the display component of the video player or video conferencing application in the correct order, so that users can watch smooth and coherent video content. In this process, the system also needs to ensure that the decoding and splicing speeds are fast enough to support the needs of real-time video communication. If there are any delays or errors, the system needs to be able to adjust in time to ensure that the user experience is not affected.

[0192] Figure 2 A structural diagram of a distributed video decoding output system is provided for an embodiment of the present application, such as Figure 2 As shown, the device comprises:

[0193] The acquisition module 21 is used to acquire video stream data and divide the video stream data into multiple data blocks;

[0194] An adjustment module 22, configured to distribute the plurality of data blocks to a plurality of decoding nodes for parallel decoding to obtain decoded video segments, wherein the transmission order of the plurality of data blocks can be dynamically adjusted according to the decoding progress of the plurality of decoding nodes;

[0195] A generating module 23, configured to splice the decoded video segments according to a preset splicing rule after the plurality of decoding nodes complete decoding, to generate a first video output model;

[0196] a synchronization module 24, configured to determine, in the first video output model, a time axis position corresponding to at least one video segment that matches the features of the video segment, obtain a matching result, and establish a synchronization model corresponding to the video segment and the first video output model based on the matching result, wherein the synchronization model is used to indicate a nonlinear time offset relationship between the video segment and the first video output model;

[0197] A determination module 25, configured to determine a time transformation matrix of a video segment relative to the first video output model according to the synchronization model;

[0198] An output module 26, configured to transform the time coordinates corresponding to the video segment to the time axis position corresponding to the video segment in the first video output model according to the time transformation matrix, so as to generate a distributed video decoding output model, wherein the distributed video decoding output model at least includes the video segment, the decoding data corresponding to the video segment, and the splicing structure generated according to the video segment;

[0199] The processing module 27 is used to input the acquired target video stream data into the distributed video decoding output model, so as to decode the target video stream data through the distributed video decoding output model, and output the corresponding video segment, the decoded data corresponding to the video segment, and the splicing structure generated according to the video segment.

[0200] Figure 2 The distributed video decoding and output system can be executed Figure 1 The implementation principle and technical effect of the distributed video decoding output method described in the embodiment are not described in detail. The specific way in which each module and unit performs operations in the distributed video decoding output system in the above embodiment has been described in detail in the embodiment of the method, and will not be described in detail here.

[0201] In one possible design, Figure 2 A distributed video decoding and output system of the embodiment shown can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0202] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0203] The processing component 32 is used to: collect video stream data and divide the video stream data into multiple data blocks; distribute the multiple data blocks to multiple decoding nodes for parallel decoding to obtain decoded video segments, wherein the transmission order of the multiple data blocks can be dynamically adjusted according to the decoding progress of the multiple decoding nodes; after the multiple decoding nodes complete decoding, splice the decoded video segments according to a pre-set splicing rule to generate a first video output model; in the first video output model, determine the time axis position corresponding to at least one video segment that matches the characteristics of the video segment to obtain a matching result, and based on the matching result, establish a corresponding synchronization model between the video segment and the first video output model, wherein the synchronization model is used to indicate the time axis position of the video segment and the first video output model. A nonlinear time offset relationship between video output models; determining a time transformation matrix of a video segment relative to the first video output model according to the synchronization model; transforming the corresponding time coordinates in the video segment to the time axis position corresponding to the video segment in the first video output model according to the time transformation matrix to generate a distributed video decoding output model, wherein the distributed video decoding output model at least includes a video segment, decoding data corresponding to the video segment, and a splicing structure generated according to the video segment; inputting the acquired target video stream data into the distributed video decoding output model to decode the target video stream data through the distributed video decoding output model, and outputting the corresponding video segment, decoding data corresponding to the video segment, and a splicing structure generated according to the video segment.

[0204] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0205] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0206] Of course, the computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0207] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc.

[0208] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0209] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0210] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 A distributed video decoding and output method of the illustrated embodiment.

[0211] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0212] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0213] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0214] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A distributed video decoding output method, characterized in that: include: Collecting video stream data, and dividing the video stream data into multiple data blocks; Allocating the multiple data blocks to multiple decoding nodes for parallel decoding to obtain decoded video segments, wherein the transmission order of the multiple data blocks can be dynamically adjusted according to the decoding progress of the multiple decoding nodes; After the multiple decoding nodes complete decoding, splicing the decoded video segments according to a preset splicing rule to generate a first video output model; In the first video output model, determining a time axis position corresponding to at least one video segment matching the features of the video segment, obtaining a matching result, and establishing a synchronization model corresponding to the video segment and the first video output model based on the matching result, wherein the synchronization model is used to indicate a nonlinear time offset relationship between the video segment and the first video output model; determining a time transformation matrix of the video segment relative to the first video output model according to the synchronization model; According to the time transformation matrix, transform the corresponding time coordinates in the video segment to the time axis position corresponding to the video segment in the first video output model to generate a distributed video decoding output model, wherein the distributed video decoding output model at least includes the video segment, the decoding data corresponding to the video segment, and the splicing structure generated according to the video segment; The acquired target video stream data is input into the distributed video decoding output model so as to decode the target video stream data through the distributed video decoding output model, and output the corresponding video segments, the decoding data corresponding to the video segments and the splicing structure generated according to the video segments.

2. The method according to claim 1, characterized in that The step of dividing the video stream data into a plurality of data blocks comprises: Identify key frames and non-key frames in the video stream data; The video stream data is divided into a plurality of data blocks according to the position information of the key frame.

3. The method according to claim 1, characterized in that Before splicing the decoded video segments according to the preset splicing rule to generate the first video output model, the method includes: Discretize each of the video clips into a plurality of video frames, wherein the plurality of video frames include corresponding frame numbers and timestamps; Obtaining a splicing order of each of the video frames by applying time synchronization and / or frame rate adjustment to the multiple video frames; A splicing rule of the video segment relative to the first video output model is generated according to the splicing order of the plurality of video frames.

4. The method according to claim 3, characterized in that The step of applying time synchronization and / or frame rate adjustment to the multiple video frames to obtain a splicing order of each of the video frames includes: Calculating the relative displacement vector and instantaneous speed between the plurality of video frames; Based on the relative displacement vector and the instantaneous speed, adjusting the speed or time position of each of the video frames to ensure that all the video frames are aligned on the time axis; Determining a splicing order of each of the video frames according to each of the adjusted video frames; The step of calculating the relative displacement vector and instantaneous speed between the plurality of video frames includes: Set video frame i at time t i The position of the i ), video frame j at time t j The position of the j ), the relative displacement between video frames i and j can be calculated as follows: ΔP ij =P(i,t i )-P(j,t j ) Where ΔPi j represents the relative displacement vector between frame i and frame j; Define the instantaneous velocity v(i) of video frame i as: Among them, Δt is the difference between adjacent timestamps, P(i, t i +1) is the video frame i at time t i +1 for the location.

5. The method according to claim 4, characterized in that Generating a splicing rule of the video segment relative to the first video output model according to the splicing order of the plurality of video frames includes: Calculating the time interval and visual consistency between each pair of video frames in the plurality of video frames, and determining a matching degree function between each pair of video frames according to the time interval and the visual consistency; Calculating the instantaneous speed between each pair of video frames in the plurality of video frames, and evaluating the smooth transition between each pair of video frames by a preset weighted average speed; Determine a splicing score between each pair of video frames based on the matching degree function and the smooth transition; Sorting based on the splicing scores and constructing splicing sequences to generate splicing rules; The step of calculating the time interval and visual consistency between each pair of video frames in the plurality of video frames, and determining a matching degree function between each pair of video frames according to the time interval and the visual consistency, comprises: Based on the inter-frame correlation and temporal continuity of the video frames, a matching function M(i, j) between video frames i and j is defined, and the matching function is used to indicate the time interval Δt between the two frames. ij and visual consistency ΔV ij , wherein the matching degree function is expressed as: M(i,j)=w t ·Δt ij +w v ·ΔV ij Among them, w t and w v are the weight coefficients of time interval and visual consistency, respectively; The calculating the instantaneous speed between each pair of video frames in the plurality of video frames and evaluating the smooth transition between each pair of video frames by a preset weighted average speed comprises: By introducing the weighted average velocity v avg (i, j) is used to measure the smooth transition between video frames i and j, where the weighted average speed is expressed as:

6. The method according to claim 5, characterized in that The step of calculating the time interval and visual consistency between each pair of video frames in the plurality of video frames, and determining a matching degree function between each pair of video frames according to the time interval and the visual consistency, further comprises: Calculate the motion vector MV of each video frame ij ; According to the time interval Δt ij , the visual consistency ΔV ij , the motion vector MV ij , determine the matching degree function M′(i, j) between each pair of video frames; Wherein, the matching degree function M′(i, j) is expressed as: M′(i,j)=w t ·Δt ij +w v ·ΔV ij +w m ·||MV ij ||+w s ·||v avg (i,j)|| Among them, w m is the weight coefficient of the motion vector, ||MV ij || represents the average motion vector magnitude between video frames i and j; w s is the weight coefficient of average speed, ||v avg (i, j)|| represents the average velocity magnitude between video frames i and j.

7. The method according to claim 6, characterized in that Before splicing the decoded video segments according to the preset splicing rule to generate the first video output model, the method further includes: Determine whether the decoded video clips can be successfully spliced; If so, splicing the decoded video segments according to a preset splicing rule to generate a first video output model; The step of determining whether the decoded video segments can be successfully spliced ​​includes: A nonlinear interpolation function NLF(i, j) is introduced. The nonlinear interpolation function is used to evaluate the visual discontinuity between video frame i and video frame j due to time offset. The nonlinear interpolation function is expressed as: NLF(i,j)=exp(-α·(Δt ij ) 2 ) Among them, α is the regularization parameter; If the result of multiplying the matching degree M′(i, j) of the video frame i and the video frame j by the nonlinear interpolation function NLF(i, j) is greater than the set threshold, it is confirmed that the video frames i and j can be successfully spliced.

8. A distributed video decoding output system, characterized in that: include: An acquisition module, used for acquiring video stream data and dividing the video stream data into multiple data blocks; An adjustment module, configured to distribute the plurality of data blocks to a plurality of decoding nodes for parallel decoding to obtain decoded video segments, wherein the transmission order of the plurality of data blocks can be dynamically adjusted according to the decoding progress of the plurality of decoding nodes; A generating module, configured to, after the plurality of decoding nodes complete decoding, splice the decoded video segments according to a preset splicing rule to generate a first video output model; a synchronization module, configured to determine, in the first video output model, a time axis position corresponding to at least one video segment that matches the features of the video segment, obtain a matching result, and establish, based on the matching result, a synchronization model corresponding to the video segment and the first video output model, wherein the synchronization model is used to indicate a nonlinear time offset relationship between the video segment and the first video output model; a determination module, configured to determine a time transformation matrix of a video segment relative to the first video output model according to the synchronization model; an output module, configured to transform the time coordinates corresponding to the video segment to the time axis position corresponding to the video segment in the first video output model according to the time transformation matrix, so as to generate a distributed video decoding output model, wherein the distributed video decoding output model at least includes the video segment, the decoding data corresponding to the video segment, and the splicing structure generated according to the video segment; A processing module is used to input the acquired target video stream data into the distributed video decoding output model so as to decode the target video stream data through the distributed video decoding output model, and output corresponding video segments, decoding data corresponding to the video segments, and a splicing structure generated according to the video segments.

9. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a distributed video decoding output method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a distributed video decoding and output method as claimed in any one of claims 1 to 7 is implemented.

Images