Video coding display method and device, electronic equipment and computer readable medium

Abstract

This disclosure provides embodiments of a video encoding / decoding and display method, apparatus, electronic device, and computer-readable medium. One specific implementation of the method includes: splitting each video frame to be encoded to obtain a set of video frame blocks; determining a set of video frame block encoding channels that meet interrupt triggering conditions; performing key video frame block peak shifting processing on each video frame block to be encoded to obtain a set of key video frame encoding blocks and a set of channel delay encoding time differences; performing video encoding on each video frame block to be encoded to obtain a set of video frame block channel bitstreams; determining the decoding and splicing threads for the video frame block channel bitstream set to obtain a set of decoding and splicing threads; performing video frame block decoding and splicing on the video frame block channel bitstream set to obtain a decoded video; and performing video frame display processing on the decoded video. This implementation can improve the balance of the bitstream after video encoding, improve the stability and real-time performance of transmitted video, and reduce video transmission latency.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to video encoding / decoding display methods, apparatuses, electronic devices, and computer-readable media. Background Technology

[0002] With the increasing demand for high-definition video, especially in remote real-time applications (such as online conferencing and telemedicine), higher requirements are being placed on the stability and low latency of video encoding and decoding. The typical approach for video encoding, decoding, and display is as follows: the video to be encoded is encoded frame by frame to obtain a video frame stream. Then, the video frame stream is decoded frame by frame to obtain the decoded video, which is then displayed on the terminal.

[0003] However, in practice, it has been found that when using the above method to encode and decode video for display, the following technical problems often occur: encoding the entire video frame without adjusting key video frames and ordinary frames leads to low stability of the video bitstream when there are bandwidth fluctuations during video transmission. At the same time, the decoding process requires waiting for the key video frames to be decoded before decoding, resulting in a large decoding delay and high video output latency, which prolongs the transmission and display time of video frames.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the present disclosure concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure provide video encoding / decoding display methods, apparatuses, electronic devices, and computer-readable media to address one or more of the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide a video encoding and decoding display method, comprising: in response to determining that an image splitting and encoding transmitting end has received a video to be encoded, splitting each video frame to be encoded included in the video to be encoded according to the encoding device information of the image splitting and encoding transmitting end to generate a video frame block group to be encoded, thereby obtaining a set of video frame block groups to be encoded; in response to detecting that the set of video frame block groups to be encoded satisfies an interrupt triggering condition, determining a set of video frame block encoding channels that satisfies the interrupt triggering condition; and performing key video frame block staggering processing on each video frame block to be encoded included in the set of video frame block encoding channels to obtain a set of key video frame encoding blocks and a channel delay encoding. The video frame block encoding time difference set is used to control the video frame block encoding channel set to encode each corresponding video frame block to be encoded, thereby obtaining a video frame block channel bitstream set. In response to the determination that the decoding and merging receiver has received the video frame block channel bitstream set, the decoding and splicing thread for each video frame block channel bitstream in the video frame block channel bitstream set is determined, resulting in a decoding and splicing thread set. Based on the decoding and splicing thread set, video frame block decoding and splicing processing is performed on the video frame block channel bitstream set to obtain a decoded video. In response to the determination that the video display end has received the decoded video, video frame display processing is performed on the decoded video.

[0008] Secondly, some embodiments of this disclosure provide a video encoding and decoding display apparatus, including: a splitting unit configured to, in response to determining that an image splitting encoding transmitter has received a video to be encoded, split each video frame to be encoded included in the video to be encoded according to encoding device information of the image splitting encoding transmitter to generate a video frame block group to be encoded, thereby obtaining a set of video frame block groups to be encoded; a first determining unit configured to, in response to detecting that the set of video frame block groups to be encoded satisfies an interrupt triggering condition, determine a set of video frame block encoding channels that satisfies the interrupt triggering condition; and a key video frame block staggering unit configured to perform key video frame block staggering processing on each video frame block to be encoded included in the set of video frame block encoding channels to obtain a set of key video frame encoding blocks and a set of channel delay encoding time differences; The control unit is configured to control the video frame block encoding channel set to encode each corresponding video frame block to be encoded according to the key video frame encoding block set and the channel delay encoding time difference set, to obtain a video frame block channel bitstream set; the second determining unit is configured to determine the decoding and splicing thread of each video frame block channel bitstream in the video frame block channel bitstream set in response to the determination that the decoding and merging receiving end has received the video frame block channel bitstream set, to obtain a decoding and splicing thread set; the video frame block decoding and splicing unit is configured to perform video frame block decoding and splicing processing on the video frame block channel bitstream set according to the decoding and splicing thread set, to obtain a decoded video; and the video frame display unit is configured to perform video frame display processing on the decoded video in response to the determination that the video display end has received the decoded video.

[0009] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0010] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0011] The various embodiments of this disclosure have the following beneficial effects: the video encoding and decoding display method of some embodiments of this disclosure can improve the balance of the encoded video stream, improve the stability and real-time performance of the transmitted video, and reduce the latency of video transmission. Specifically, the reason for the low stability of the related video stream, the high latency of video output, and the extended transmission and display time of video frames is that: the entire video frame is encoded without adjustment of key video frames and ordinary frames. When there are bandwidth fluctuations during video transmission, the stability of the video stream is low. At the same time, the decoding process requires waiting for the key video frames to be decoded before decoding, resulting in a large decoding delay, which leads to a high latency of video output and an extended transmission and display time of video frames. Based on this, the video encoding and decoding display method of some embodiments of this disclosure can first, in response to determining that the image splitting and encoding sending end has received the video to be encoded, split each video frame to be encoded in the video to be encoded according to the encoding device information of the image splitting and encoding sending end, to generate a group of video frame blocks to be encoded, and obtain a set of video frame blocks to be encoded. Here, splitting the video frames to be encoded reduces the amount of encoded data during video transmission and provides a data foundation for subsequent key frame block staggering. Secondly, in response to the detection that the set of video frame blocks to be encoded meets the interruption trigger condition, a set of video frame block encoding channels that meet the interruption trigger condition is determined. Here, meeting the terminal trigger condition does not require waiting for the entire video frame to be transmitted before subsequent encoding, reducing transmission encoding latency. Thirdly, key video frame block staggering is performed on each video frame block to be encoded within the aforementioned video frame block encoding channel set, resulting in a key video frame block encoding set and a channel delay encoding time difference set. Here, key video frame block staggering achieves a uniform distribution of key video frame blocks in each video frame block encoding channel, reducing the impact of transmission bandwidth fluctuations on the stability of the encoded bitstream during video transmission, reducing bandwidth occupation during peak transmission periods, and improving the stability of bitstream transmission. Next, based on the aforementioned key video frame block encoding set and the aforementioned channel delay encoding time difference set, the aforementioned video frame block encoding channel set is controlled to encode each corresponding video frame block to be encoded, resulting in a video frame block channel bitstream set. Here, the encoding efficiency of video encoding and the transmission stability of the video frame block channel bitstream set to the decoder-merge receiver can be improved. Subsequently, in response to determining that the decoder-merge receiver has received the aforementioned video frame block channel bitstream set, the decoding and splicing threads for each video frame block channel bitstream in the set are determined, resulting in a decoding and splicing thread set. Here, multi-threaded decoding processing using the decoding and splicing thread set can improve decoding efficiency and shorten decoding time. Then, based on the aforementioned decoding and splicing thread set, video frame block decoding and splicing processing is performed on the aforementioned video frame block channel bitstream set to obtain the decoded video.Here, by using a decoding and splicing thread set to splice multi-threaded video frame blocks into video frames, the efficiency and effectiveness of splicing video frame blocks into video frames can be improved, the splicing time can be shortened, and the splicing resource consumption can be reduced. Finally, in response to the determination that the video display end has received the above-mentioned decoded video, the above-mentioned decoded video is processed for video frame display. Here, the display time of the decoded video can be reduced and the display quality of the decoded video can be improved. Thus, this video encoding and decoding display method can directly perform subsequent encoding processing when the video frame block meets the interrupt triggering condition through video frame splitting processing, without waiting for the entire video frame to be transmitted, which can reduce latency. In addition, the peak-shifting processing of key video frame blocks can evenly distribute the layout of key video frame blocks, improve the balance of the bitstream, reduce the impact of transmission bandwidth fluctuations, improve the efficiency of encoding and transmission, and reduce latency. By using a decoding and splicing thread set for video frame block decoding and splicing processing, decoding efficiency and image synthesis speed can be improved, and the video frame transmission and display time can be shortened. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0013] Figure 1 This is a flowchart of some embodiments of the video encoding / decoding and display method according to the present disclosure;

[0014] Figure 2 This is a schematic diagram illustrating the determination of the video frame block encoding channel set by interrupt triggering in some embodiments of the video encoding and decoding display method according to this disclosure;

[0015] Figure 3 This is a schematic diagram of a video frame block channel bitstream set according to some embodiments of the video encoding and decoding display method of this disclosure;

[0016] Figure 4 This is a flowchart illustrating the entire process of receiving the video to be encoded from the image splitting and encoding sending end, splitting the video, triggering the interruption trigger condition, processing the peak shifting of key video frame blocks, and encoding the video according to some embodiments of the video encoding and decoding display method disclosed herein.

[0017] Figure 5 This is a flowchart illustrating the entire processing flow from determining the decoding and splicing thread set, video frame block decoding and splicing processing, and video frame display processing in some embodiments of the video encoding and decoding display method according to this disclosure.

[0018] Figure 6 These are schematic diagrams illustrating the structure of some embodiments of the video encoding / decoding display apparatus according to the present disclosure;

[0019] Figure 7 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0020] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0021] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0022] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0023] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0024] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0025] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] Figure 1 A flow 100 of some embodiments of a video encoding / decoding display method according to the present disclosure is shown. The video encoding / decoding display method includes the following steps:

[0027] Step 101: In response to the determination that the image splitting and encoding sending end has received the video to be encoded, the encoding device information of the image splitting and encoding sending end is used to split each video frame to be encoded in the video to be encoded to generate a video frame block group, thus obtaining a video frame block group set to be encoded.

[0028] In some embodiments, the execution entity (e.g., an electronic device) of the above-described video encoding and decoding display method may, in response to determining that the image splitting and encoding sending end has received the video to be encoded, perform splitting processing on each video frame to be encoded included in the video to be encoded according to the encoding device information of the image splitting and encoding sending end, to generate a set of video frame blocks to be encoded. The image splitting and encoding sending end may be a terminal used to perform image splitting and encoding on the received video to be encoded. For example, the image splitting and encoding sending end may be a server running VPSS (Video Processing Sub-System) and an Allwinner H728 encoding chip. The video to be encoded may be high-definition video. For example, the video to be encoded may be online conference video, telemedicine video, or remote high-definition video captured by a drone. The encoding device information may be information describing the device of the image splitting and encoding sending end. For example, the encoding device information may include, but is not limited to, at least one of the following: memory information, bandwidth information, interrupt control information, and line counter module information. The video frame blocks to be encoded in the aforementioned group of video frame blocks to be encoded can be image blocks obtained after splitting, with overlapping portions between adjacent video frame blocks to be encoded. In practice, the aforementioned execution entity can, in response to determining that the aforementioned image splitting and encoding sending end is a line counter module that supports 6 independent line counting interrupts, split each video frame to be encoded into 6 video frame blocks of the same size with overlapping portions, thus obtaining a set of video frame blocks to be encoded.

[0029] In addressing the aforementioned technical problems in the application scenario of remote real-time video conferencing, the following technical issues often arise: During video transmission, bandwidth fluctuations and limitations in the computing resources of the encoding / decoding terminal mean that a fixed segmentation strategy cannot adjust to these fluctuations, leading to low bandwidth utilization, reduced video quality and content loss, prolonged transmission time, and a decreased user experience. Based on the characteristics of this application scenario—adaptability to real-time conditions, high video stability, low latency, high-quality video transmission, and significant network fluctuations—we have decided to adopt the following solution:

[0030] In some optional implementations of certain embodiments, the process of splitting each video frame to be encoded into a video frame block group based on the encoding device information of the image splitting encoding transmitter to obtain a set of video frame block groups may include the following steps:

[0031] The first step is to determine the video frame block granularity information set of the video to be encoded. This set of video frame block granularity information can be information on the non-uniformly divisible granularity of each frame of the video to be encoded, determined based on the video content. For example, the video frame block granularity information could include 2... 2 and 3 3. Coarse-grained segmentation. The above 2. 2 can represent dividing a video frame to be encoded into four independent video frame blocks with two rows and two columns. In practice, the above-mentioned execution entity can perform the following block granularity determination steps for each video frame to be encoded: First, use optical flow to perform motion intensity analysis on the video frame to be encoded to generate a motion intensity map. Second, use the local binary mode operator to determine the texture complexity map of the video frame to be encoded. Third, input the video frame to be encoded into an object detection model to obtain a semantic importance map. The object detection model can be a lightweight MobileNet-SSD (MobileNet-Single Shot MultiBox Detector) model. Then, the motion intensity map, texture complexity map, and semantic importance map are weighted and fused to obtain a fused video frame. Next, the fused video frame is decomposed using a quadtree decomposition method to obtain decomposed video frame groups. Finally, the multiple decomposed video frame groups are hierarchically fused to obtain a video frame block granularity information set.

[0032] The second step involves identifying regions of interest (ROIs) within the video to be encoded, resulting in a set of ROIs for each video frame. These ROIs can be key content of interest to the user or regions with significant changes in foreground objects within the video frame. In practice, the executing entity can first use optical flow to determine the motion vector field map of the video to be encoded. Then, a saliency detection algorithm is used to perform visual recognition on the video to be encoded, obtaining a visual saliency map of the video frames. This saliency detection algorithm can be the LTTI visual saliency model. Finally, a weighted fusion is performed on each motion vector field map in the visual saliency map of the video frames and its corresponding motion vector field map to obtain the set of ROIs for each video frame.

[0033] The third step involves generating a video decoding time constraint set and a video encoding time constraint set for the video to be encoded, based on the aforementioned video frame block granularity information set, the aforementioned encoding device information, and the aforementioned video frame region of interest set. The video decoding time constraint information can be used to constrain the decoding time of the aforementioned video frames to be encoded, thereby shortening the decoding time and reducing latency. The video encoding time constraint can be used to constrain the encoding time of the aforementioned video frames to be encoded, thereby shortening the encoding time and reducing latency.

[0034] As an example, the aforementioned execution entity can utilize the video encoding time constraint function and the video decoding time constraint function to generate a video decoding time constraint information set and a video encoding time constraint information set for the video to be encoded, based on the aforementioned video frame block granularity information set, the aforementioned encoding device information, and the aforementioned video frame region of interest set. The aforementioned video decoding time constraint function can be expressed as:

[0035] .

[0036] in, This represents the decoding time of the video decoding time constraint function. Indicates the first The first video frame to be encoded The block and compression level is The computational resources required to decode video frame blocks. Indicates the first The first video frame to be encoded Whether the video frame block is located within the region of interest of the video frame. A value of 1 indicates that it is, and a value of 0 indicates that it is not. Indicate whether to send the data located at the 1st position. The first video frame to be encoded The block and compression level is The video frame block has a value of 1 indicating transmission and a value of 0 indicating no transmission. This indicates the number of processor cores at the decoding and merging receiver. This indicates the conversion efficiency of the decoding and merging receiver when performing decoding in multi-core working mode. This indicates the available computing resources at the decoding and merging receiver. This indicates the position information of the video frame to be encoded within the video to be encoded. Indicates according to the first The video frame block set is obtained by dividing the video frame into blocks based on the block granularity information of each video frame. This indicates the compression level during the decoding process. This indicates the number of frames in the video to be encoded. Indicates the first Information on the granularity of each video frame block.

[0037] The video encoding time constraint function described above can be a function with the same form as the video decoding time constraint function described above, except that the device information corresponding to the decoding and merging receiver is changed to the device information of the image splitting and encoding sender, and the preset maximum decoding time is changed to the preset maximum encoding time.

[0038] Fourth, based on the aforementioned video frame block granularity information set, the aforementioned encoding device information, and the transmission bandwidth status information, a video transmission time constraint information set for the video to be encoded is generated. The aforementioned transmission bandwidth status information refers to the bandwidth status information at the transmission moment. The video transmission time constraint information can be time-related constraint information during the transmission of the video to be encoded. The aforementioned transmission bandwidth status information can be the average bandwidth over the entire transmission period.

[0039] As an example, the aforementioned execution entity can utilize the video transmission time constraint function to generate a video transmission time constraint information set for the video to be encoded, based on the aforementioned video frame block granularity information set, the aforementioned encoding device information, and the transmission bandwidth status information. The aforementioned video transmission time constraint function can be expressed as:

[0040] .

[0041] in, This represents the transmission time corresponding to the video transmission time constraint function. Indicates the first The first video frame to be encoded The block and compression level is The time spent transmitting video frame blocks. This represents the average network bandwidth during the transmission of the aforementioned video to be encoded.

[0042] Fifth, based on the aforementioned video decoding time constraint information set, video encoding time constraint information set, and video transmission time constraint information set, the aforementioned video frame block granularity information set is filtered to obtain a filtered video frame block granularity information set, which serves as the target block granularity information set. The filtered video frame block granularity information in the aforementioned filtered video frame block granularity information set can be block granularity information that satisfies the constraints of the aforementioned video decoding time constraint information, video encoding time constraint information, and video transmission time constraint information.

[0043] As an example, the aforementioned execution entity may first select at least one video frame block granularity information corresponding to at least one video decoding time constraint information less than or equal to a preset decoding time threshold from the aforementioned video decoding time constraint information set, as a first video frame block granularity information set. The preset decoding time threshold may be a pre-set maximum value for decoding time. Secondly, it may select at least one video frame block granularity information corresponding to at least one video encoding time constraint information less than or equal to a preset encoding time threshold from the aforementioned video encoding time constraint information set, as a second video frame block granularity information set. The preset encoding time threshold may be a pre-set maximum value for encoding time. Then, it may select at least one video frame block granularity information corresponding to at least one video transmission time constraint information less than or equal to a preset transmission time threshold from the aforementioned video transmission time constraint information set, as a third video frame block granularity information set. The preset transmission time threshold may be a pre-set maximum value for transmitting the video to be encoded. Finally, it may determine the intersection of the first, second, and third video frame block granularity information sets, as a target block granularity information set.

[0044] Step 6: Determine the region of interest (ROI) transmission loss rate and ROI transmission waste rate for each target block granularity information in the aforementioned target block granularity information set. The ROI transmission loss rate characterizes the proportion of video frame blocks in the video frame block set corresponding to the target block granularity information that are considered to be regions of interest but were not transmitted. The ROI transmission waste rate characterizes the proportion of video frame blocks in the video frame block set corresponding to the target block granularity information that are transmitted but are not regions of interest.

[0045] Step 7: Based on the above-mentioned region of interest transmission missing rate and region of interest transmission wastage rate, the above-mentioned target block granularity information set is filtered again to obtain the filtered target block granularity information.

[0046] As an example, the aforementioned execution entity can minimize the weighted sum of the aforementioned region of interest transmission loss rate and the aforementioned region of interest transmission waste rate, using this as the minimized video viewing quality function. Then, the aforementioned target block granularity information set is input into the minimized video viewing quality function to obtain the video viewing quality function value set. Finally, the target block granularity information corresponding to the video viewing quality function value with the smallest value in the aforementioned video viewing quality function value set is determined as the filtered target block granularity information.

[0047] Step 8: Based on the target block granularity information after the above screening, each video frame to be encoded in the above video to be encoded is split into blocks to obtain a set of video frame blocks to be encoded, and the video frame display is processed after encoding, decoding and transmitting the set of video frame blocks to be encoded.

[0048] The above-described technical solution and its related content, as an inventive point of this disclosure, solve the technical problem mentioned in the background art: "low bandwidth utilization, reduced video quality and content loss, prolonged video transmission time, and reduced user experience." Factors leading to low bandwidth utilization, reduced video quality and content loss, prolonged video transmission time, and reduced user experience are often as follows: During video transmission, due to bandwidth fluctuations and limitations in the computing resources of the encoding / decoding terminal, a fixed block strategy cannot be adjusted with bandwidth fluctuations, resulting in low bandwidth utilization, reduced video quality and content loss, prolonged video transmission time, and reduced user experience. Solving these factors can improve bandwidth utilization, improve video quality, shorten video transmission time, and improve user experience. To achieve this effect, this disclosure first determines the video frame block granularity information set and identifies the region of interest (ROI) of the video to be encoded by analyzing the motion intensity, texture complexity, and semantic importance of the video. This allows for compatibility with different video content and terminal capabilities, improving the accuracy of the generated video frame block granularity information and providing data for subsequent time constraint information. Secondly, based on the generated video decoding time constraint information set, video encoding time constraint information set, and video transmission time constraint information set, the video frame block granularity information set is filtered. By constructing constraints, unsuitable block granularities can be accurately eliminated to ensure the real-time performance, timely response, and shortened encoding / decoding and transmission time of the video to be encoded. Subsequently, by determining the region of interest transmission loss rate and region of interest transmission waste rate, the filtered target block granularity information is determined, quantifying the cost of different block granularities to select the block granularity with the lowest cost. This is suitable for the current bandwidth conditions, improves bandwidth utilization, and effectively avoids video loss and bandwidth resource waste. Finally, using the filtered target block granularity information, the video to be encoded is split into video frames to obtain a set of video frame blocks to be encoded. After encoding, decoding, and transmitting the set of video frame blocks to be encoded, video frame display processing is performed to obtain the video frame blocks most suitable for the current bandwidth, thereby improving encoding, decoding, and transmission efficiency, shortening the time, reducing latency, and improving the stability of video transmission.

[0049] Step 102: In response to detecting that the set of video frame blocks to be encoded meets the interrupt triggering condition, determine the set of video frame block encoding channels that meet the interrupt triggering condition.

[0050] In some embodiments, the execution entity may, in response to detecting that the set of video frame blocks to be encoded meets an interrupt triggering condition, determine a set of video frame block encoding channels that meet the interrupt triggering condition. The interrupt triggering condition may be a condition that the video frame blocks to be encoded meet preset video frame block size information. The preset video frame size information may be the size information of the video frame blocks determined by the encoding device information. The video frame block encoding channels in the set of video frame block encoding channels may be thread channels located at the image split encoding sending end used for encoding the video frame blocks to be encoded. Figure 2 As shown, Figure 2 The diagram illustrates how an interrupt trigger determines the video frame block encoding channel set. Figure 2 In the text, 1920 and 1080 can represent the pixel values ​​of the video frame to be encoded, i.e., the size of the video frame to be encoded. Figure 2 The phrase "line 176, trigger terminal 0" indicates that an interrupt is triggered at line 176 of the video frame to be encoded, and the data is subsequently sent to the first channel for encoding and decoding.

[0051] In some optional implementations of certain embodiments, the process of determining the video frame block encoding channel set that satisfies the interruption triggering condition in response to detecting that the set of video frame blocks to be encoded satisfies the interruption triggering condition may include the following steps:

[0052] The first step involves creating a set of video frame block buffers and a set of buffer fill status monitors, where there is a one-to-one correspondence between the video frame block buffer set and the buffer fill status monitor set. The video frame block buffers in the video frame block buffer set can be storage areas used to store the video frame blocks to be encoded. The buffer fill status monitors in the buffer fill status monitor set can be used to detect whether the video frame block buffers are fully filled.

[0053] The second step is to determine, in response to the existence of at least one buffer filling status monitor in the above-mentioned buffer filling status monitor set that indicates the completion of video frame block buffer filling, to determine at least one set of video frame tiles to be encoded stored in at least one video frame block buffer corresponding to the at least one buffer filling status monitor as the target set of video frame tiles to be encoded.

[0054] The third step involves generating a video frame block filling interruption signal set in response to determining that the target set of video frame blocks to be encoded meets the interruption triggering conditions. The video frame block filling interruption signals in this set can be signals used to interrupt the filling of the video frame blocks to be encoded into the video frame block buffer.

[0055] The fourth step is to determine the set of video frame block encoding channels that meet the interruption triggering conditions based on the aforementioned video frame block filling interruption signal set. For example, the execution entity can determine the set of video frame block buffers corresponding to the aforementioned video frame block filling interruption signal set as the target video frame block buffer set. Then, it determines the set of encoding threads that perform encoding processing on each video frame block to be encoded included in the target video frame block buffer set, as the video frame block encoding channel set.

[0056] Step 103: Perform key video frame block staggering processing on each video frame block to be encoded included in the video frame block coding channel set to obtain the key video frame coding block set and the channel delay coding time difference set.

[0057] In some embodiments, the execution entity may perform key video frame block staggering processing on each video frame block to be encoded included in the video frame block encoding channel set to obtain a key video frame encoding block set and a channel delay encoding time difference set. The key video frame encoding blocks in the key video frame encoding block set may be video frame blocks used for intra-frame encoding. The channel delay encoding time difference in the channel delay encoding time difference set may be the time difference value between the video frame block encoding channel and the corresponding key video frame block for intra-frame encoding.

[0058] In some optional implementations of certain embodiments, the above-mentioned key video frame block staggering processing of each video frame block to be encoded included in the video frame block coding channel set to obtain the key video frame coding block set and the channel delay coding time difference set may include the following steps:

[0059] The first step is to perform padding completion timing sorting on the aforementioned video frame block coding channel set to obtain the video frame block coding channel sequence. This padding completion timing sorting can be based on the chronological order in which the video frame block coding channels receive the video frame blocks to be encoded.

[0060] The second step involves determining the location information of key video frame blocks for each video frame block to be encoded within each video frame block encoding channel in the aforementioned video frame block encoding channel sequence, based on the preset channel key video frame block staggering condition. This results in a key video frame block location information set. The preset channel key video frame block staggering condition can be the time difference between the start times of encoding the first channel key video frame block in the video frame block encoding channel, ensuring a uniform distribution of the channel key video frame block set within the aforementioned video frame block encoding channel set. The key video frame block location information in the key video frame block location information set can be the location information of the channel key video frame block within the aforementioned video frame block encoding channel. As an example, the execution entity can encode the preset channel key video frame block staggering condition to obtain a key video frame block staggering feature vector. Then, this key video frame block staggering feature vector is input to the image splitting and encoding sending end for analysis and processing to determine the location information of key video frame blocks for each video frame block to be encoded within each video frame block encoding channel in the aforementioned video frame block encoding channel sequence, thus obtaining a key video frame block location information set.

[0061] The third step is to determine the channel delay coding time difference set and the key video frame coding block set of the above-mentioned video frame block coding channel sequence based on the above-mentioned channel key video frame block location information set.

[0062] As an example, the aforementioned execution entity can use the aforementioned channel key video frame block location information set to perform keyframe annotation on the video frame block to be encoded for each video frame block encoding channel in the aforementioned video frame block encoding channel sequence, corresponding to the channel key video frame block location information set, thus obtaining a key video frame encoding block set. Then, the encoding time when each video frame block encoding channel located at the initial position in the video frame block encoding channel sequence completes encoding the first key video frame block is determined, and this time is used as the channel delay encoding time difference between adjacent video frame block encoding channels.

[0063] Step 104: Based on the key video frame coding block set and the channel delay coding time difference set, control the video frame block coding channel set to perform video coding on the corresponding video frame blocks to be encoded, and obtain the video frame block channel bitstream set.

[0064] In some embodiments, the execution entity can control the video frame block encoding channel set to encode each corresponding video frame block to be encoded based on the key video frame encoding block set and the channel delay encoding time difference set, thereby obtaining a video frame block channel bitstream set. The video frame block channel bitstream in the video frame block channel bitstream set can be a bitstream obtained by encoding each of the aforementioned video frame blocks to be encoded. For example... Figure 3 As shown, Figure 3The diagram shows the internal structure of the video frame block channel bitstream set obtained by jointly implementing key video frame block peak shifting processing and video encoding in steps 103 and 104. Figure 4 As shown, Figure 4 This shows an internal diagram of the entire process implemented jointly by steps 101 to 104. Figure 4 The split-screen interrupt in the image can be the splitting process performed by the image split encoding sender and the handling of the interrupt triggering conditions. Figure 4 The breakpoints interrupt 0 and [0, 0, 1920, 176] in the code can indicate that channel 0 is interrupted at the video frame to be encoded, [0, 0, 1920, 176].

[0065] In addressing the aforementioned technical problems in the application scenario of real-time video calls (e.g., video conferencing, telemedicine, online education), the following technical issues often arise: Due to the lack of an adjustment mechanism for keyframe and regular frame transmission within the inherent Group of Pictures (GOP) during real-time video calls, the encoded bitstreams of keyframes and regular frames differ significantly. Furthermore, the fixed and single-channel search range for motion estimation during encoding leads to low encoding efficiency, prolonged transmission time, and wasted transmission resources. Based on the characteristics of this application scenario—real-time performance, low latency, high video quality, minimal bitrate fluctuations, and extended battery life for edge mobile devices—we have decided to adopt the following solution:

[0066] In some optional implementations of certain embodiments, controlling the video frame block encoding channel set to encode each corresponding video frame block to be encoded based on the key video frame encoding block set and the channel delay coding time difference set to obtain a video frame block channel bitstream set may include the following steps:

[0067] The first step, for each video frame block coding channel in the above video frame block coding channel set, is to perform the following determination steps:

[0068] Sub-step 1: Extract the key video frame coding blocks included in the above video frame block coding channel as the channel key video frame blocks.

[0069] Sub-step 2: Based on the aforementioned channel delay coding time difference set, control the aforementioned video frame block coding channel to perform intra-block coding on the aforementioned channel key video frame blocks to obtain the channel key video frame block bitstream.

[0070] As an example, the aforementioned execution entity may, in response to determining that the waiting encoding time of the aforementioned video frame block encoding channel reaches the aforementioned channel delay encoding time difference set, use intra-frame coding to control the aforementioned video frame block encoding channel, perform intra-frame coding on the aforementioned channel key video frame blocks, and obtain the channel key video frame block bitstream.

[0071] Sub-step 3: Remove the key video frame blocks of the aforementioned channel from each video frame block to be encoded included in the aforementioned video frame block encoding channel to obtain the remaining video frame block group to be encoded.

[0072] Sub-step 4 involves determining the matching video frame blocks for each remaining video frame block in the aforementioned group of remaining video frame blocks to be encoded, thereby generating a set of matching video frame blocks to be encoded. These matching video frame blocks can be the most matched video frame blocks found by searching from the adjacent channel sets and the target channel set. The adjacent channel sets can be encoded video frame blocks that belong to the same frame as the remaining video frame blocks and have already been encoded. The target channel set can be the most matched video frame block found by searching from the already encoded video frame blocks included in the video block encoding channel. It should be noted that these matching video frame blocks are obtained by searching within the search range of all video frame blocks included in the video block encoding channel, which can capture real motion across boundaries, significantly improve prediction accuracy, and reduce residual energy.

[0073] As an example, the aforementioned execution entity can perform the following search and matching steps for each remaining video frame block to be encoded in the remaining video frame block group: First, establish a first video frame block search grid with the location of the remaining video frame block to be encoded as the search center. This first video frame block search grid can be a search grid whose search range includes all the video frame blocks to be encoded within the video frame block encoding channel, whose search radius is twice the size of the video frame block to be encoded, and whose grid spacing is the ratio of the size of the video frame block to 8. Second, within the first video frame block search grid, using SAD (Sum of Absolute Difference) and the search step size, determine the macroblock of the already encoded video frame block with the smallest sum of absolute errors to each macroblock of the remaining video frame blocks to be encoded, and use this as the initial set of video frame macroblocks. Next, a second video frame block search grid is determined, and within this second video frame block search grid, the initial video frame macroblock set is updated using SATD (Sum of Absolute Transformed Difference) to obtain a first updated video frame macroblock set. The second video frame block search grid can be a search grid with a search range encompassing all video frame blocks to be encoded within the video frame block encoding channel, a search radius half the size of the video frame block to be encoded, and a grid spacing of 2 pixels. Finally, a third video frame block search grid is determined, and within this third video frame block search grid, the initial video frame macroblock set is updated using SATD (Sum of Absolute Transformed Difference) to obtain a second updated video frame macroblock set, which serves as the matching video frame blocks to be encoded. The sub-pixel matching update can include half-pixel search updates and quarter-pixel search updates.

[0074] Sub-step 5: Based on the aforementioned set of video frame blocks to be encoded, perform inter-block coding on the remaining group of video frame blocks to be encoded to obtain the channel video frame block bitstream. The channel video frame block bitstream can be a bitstream obtained by inter-coding within the search range of all video frame blocks to be encoded included in the video frame block coding channel. In practice, the executing entity can utilize inter-frame coding to perform inter-block coding on the remaining group of video frame blocks to be encoded based on the aforementioned set of video frame blocks to be encoded to obtain the channel video frame block bitstream.

[0075] Sub-step 6 involves performing bitstream control on the aforementioned channel key video frame block bitstream and the aforementioned channel inter-frame block bitstream to obtain the bitstream-controlled channel key video frame block bitstream and the bitstream-controlled channel inter-frame block bitstream, which serve as the key video frame block control bitstream and the channel inter-frame block control bitstream, respectively. The executing entity can first determine the video texture complexity of the set of video frame blocks to be encoded corresponding to the aforementioned channel key video frame block bitstream. Secondly, in response to determining that there exists a video texture complexity greater than or equal to a preset texture complexity threshold, the quantization parameter (QP) of the channel key video frame block bitstream is reduced to obtain the adjusted quantization parameter. The preset texture complexity threshold can be a pre-set critical value for whether to adjust the quantization parameter size. Then, the inter-frame residual energy set of the aforementioned channel inter-frame block bitstream is determined. The inter-frame residual energy in the aforementioned inter-frame residual energy set can be calculated by calculating the sum of absolute transform domain errors (SATD) of the matched video frame block to be encoded and the remaining video frame blocks to be encoded. Subsequently, based on the aforementioned inter-frame residual energy set, the quantization parameters corresponding to the inter-block bitstream of the channel video frames are dynamically adjusted to obtain the adjusted residual quantization parameter set. This dynamic adjustment can involve increasing the quantization parameter (e.g., increasing it by 3-5) when the inter-frame residual energy is less than or equal to a first threshold, and decreasing the quantization parameter (e.g., decreasing it by 3-5) when the inter-frame residual energy is greater than a second threshold. The first and second thresholds can be pre-set critical values ​​for increasing and decreasing the quantization parameter. Finally, based on the adjusted quantization parameters and the adjusted residual quantization parameter set, bitstream control is performed on the aforementioned channel key video frame block bitstream and the aforementioned channel inter-block video frame bitstream to obtain the key video frame block control bitstream and the channel inter-block video frame control bitstream.

[0076] Sub-step 7 involves determining the aforementioned key video frame block control stream and the aforementioned inter-video frame block control stream as the video frame block channel stream, and transmitting the aforementioned video frame block channel stream to the video display terminal.

[0077] The above-described technical solution and its related content, as an inventive point of this disclosure, solve the technical problem mentioned in the background art: "leading to low coding efficiency, prolonged transmission time, and waste of a large amount of transmission resources." The factors leading to low coding efficiency, prolonged transmission time, and waste of a large amount of transmission resources are often as follows: Due to the inherent lack of an adjustment mechanism for keyframe and ordinary frame transmission in real-time video calls within a GOP (Group of Pictures), the encoded bitstreams of keyframes and ordinary frames differ significantly. Furthermore, the motion estimation search range during encoding is fixed and single-channel, resulting in low coding efficiency, prolonged transmission time, and waste of a large amount of transmission resources. Solving these factors can improve coding efficiency, shorten transmission time, and reduce the waste of transmission resources. To achieve this effect, this disclosure first extracts key video frame blocks of a channel and performs intra-frame encoding, which can improve the accuracy of key video frame block extraction, thereby improving the video quality of subsequent encoding and reducing the number of intra-frame encoded video frame blocks. Secondly, by performing a multi-stage search for matching video frame blocks to be encoded for each remaining video frame block to be encoded in both the video frame block encoding channel and adjacent video frame block encoding channels, a set of matching video frame blocks to be encoded is obtained and inter-frame coding is performed. This improves the matching accuracy of motion estimation, enhances the load uniformity of the bitstream between channels, captures real motion across boundaries, reduces residual energy and inter-frame coding bitstream, and improves the efficiency of inter-frame coding. Finally, by integrating the bitstreams of key video frame blocks in the channel and the bitstreams between video frame blocks in the channel after bitstream control, the bitstream fluctuation can be reduced, the stability of video transmission can be improved, and the transmission time and transmission resource waste can be reduced.

[0078] Step 105: In response to determining that the decoding and merging receiver has received a set of video frame block channel bitstreams, determine the decoding and splicing thread for each video frame block channel bitstream in the set of video frame block channel bitstreams, and obtain a set of decoding and splicing threads.

[0079] In some embodiments, the execution entity may, in response to determining that the decoding and merging receiving end has received the video frame block channel bitstream set, determine the decoding and splicing thread for each video frame block channel bitstream in the video frame block channel bitstream set, thereby obtaining a decoding and splicing thread set. The decoding and merging receiving end may be a terminal that decodes the received video frame block channel bitstream set to obtain decoded video frame blocks and splices them into a video frame. For example, the decoding and merging receiving end may be a server with an embedded Silicon Motion SM768 chip. The decoding and splicing threads in the decoding and splicing thread set may be threads used to process bitstream decoding and video frame block splicing.

[0080] Step 106: Based on the decoding and splicing thread set, perform video frame block decoding and splicing processing on the video frame block channel bitstream set to obtain the decoded video.

[0081] In some embodiments, the execution entity may perform video frame block decoding and splicing processing on the video frame block channel bitstream set according to the aforementioned decoding and splicing thread set to obtain a decoded video. The decoded video may be a video frame obtained by splicing the decoded and reconstructed video frame block set to obtain a complete video frame for display and playback.

[0082] In some optional implementations of certain embodiments, the process of performing video frame block decoding and splicing on the video frame block channel bitstream set according to the aforementioned decoding and splicing thread set to obtain the decoded video may include the following steps:

[0083] The first step is to control the above-mentioned decoding and splicing thread set to decode the above-mentioned video frame block channel bitstream set to obtain the decoded video frame block set.

[0084] The second step is to clear the display buffer of the aforementioned video display terminal to obtain a cleared display buffer. The aforementioned video display terminal can be a monitor terminal used to display decoded video composed of a set of decoded video frame blocks. For example, the aforementioned video display terminal can be an HDMI (High Definition Multimedia Interface) monitor. The aforementioned display buffer can be a storage area used to store the spliced ​​video frames.

[0085] The third step involves storing the set of non-overlapping decoded video frame blocks, including the aforementioned decoded video frame block set, into the aforementioned zeroed display buffer via direct memory access. The non-overlapping decoded video frame blocks in the aforementioned set can be video frame blocks that do not overlap with other decoded video frame blocks.

[0086] Fourth, for each overlapping decoded video frame block region in the overlapping decoded video frame block region set included in the above-mentioned decoded video frame block set, perform the following pixel blending steps:

[0087] Sub-step 1 involves performing an incremental transparency transformation on the overlapping region of the first decoded video frame block corresponding to the aforementioned overlapping decoded video frame block region to obtain a first overlapping region transparency set. Here, the aforementioned overlapping decoded video frame block region can be the overlapping region between adjacent video frame blocks to be encoded. The aforementioned first decoded video frame block can be the upper overlapping region of the overlapping decoded video frame block region. The first overlapping region transparency in the aforementioned first overlapping region transparency set can be the transparency of each overlapping row when the first decoded video frame block and the second decoded video frame block are merged, i.e., the fusion weight. In practice, the aforementioned execution entity can determine the overlap width of the overlapping decoded video frame block region. Then, a linear transparency transformation from 0 to 1 is performed on each row of the aforementioned overlap width to obtain the first overlapping region transparency set.

[0088] Sub-step 2 involves performing a decreasing transparency transformation on the overlapping region of the second decoded video frame block corresponding to the aforementioned overlapping decoded video frame block region to obtain a second overlapping region transparency set. The second decoded video frame block can be the lower overlapping region of the overlapping decoded video frame block region. The second overlapping region transparency in the aforementioned second overlapping region transparency set can be the transparency of each overlapping row when the first and second decoded video frame blocks are merged, with each first overlapping region transparency in the first overlapping region transparency set and the corresponding second overlapping region transparency in the second overlapping region transparency set having a fusion weight of 1. In practice, the execution entity can determine the overlap width of the overlapping decoded video frame block region. Then, a linear transparency transformation from 1 to 0 is performed on each row of the aforementioned overlap width to obtain the second overlapping region transparency set.

[0089] Sub-step 3 involves pixel blending of the overlapping regions of the first and second decoded video frame blocks based on the first and second overlapping region transparency sets, to obtain a decoded overlapping video frame block region. This decoded overlapping video frame block region can be an overlapping region obtained through an alpha blending algorithm. In practice, the executing entity can perform weighted fusion of the first and second overlapping region transparency sets, the overlapping regions of the first and second decoded video frame blocks, to obtain the decoded overlapping video frame block region.

[0090] The fifth step involves transferring the obtained decoded overlapping video frame block regions to the aforementioned zeroed display buffer via direct memory access.

[0091] The sixth step involves splicing together the sets of overlapping and non-overlapping decoded video frame blocks stored in the zeroed display buffer to obtain the decoded video.

[0092] It should be noted that the decoding and splicing thread set can perform multi-threaded parallel processing, which can improve the processing speed of decoding and splicing, shorten the processing time, and use DMA (Direct Memory Access) to directly transmit the data of decoded video frames without the need for CPU data transmission. This can reduce the resource consumption of the decoding and merging receiver, thereby improving the efficiency of decoding and splicing as well as the real-time performance of subsequent display.

[0093] In addressing the aforementioned technical problems in the application scenario of real-time video streaming decoding, the following issues often arise: numerous dependencies exist between video frame blocks during the decoding process after encoding the video frame block set; serial latency exists between decoding and video frame block concatenation, leading to low efficiency in both processes; thread idleness results in resource waste; and the duration of video decoding and concatenation is prolonged, ultimately reducing the quality of the decoded video. Based on the characteristics of this application scenario—adaptability to video frame blocks obtained from image splitting, overlapping video frame blocks, multi-threaded parallel decoding, low latency, and load balancing—we have decided to adopt the following solution:

[0094] The above-mentioned decoding and splicing process, which involves decoding and splicing video frames from the video frame block channel bitstream set to obtain the decoded video, may include the following steps:

[0095] The first step involves controlling the aforementioned decoding and splicing thread set to perform entropy decoding processing on the aforementioned video frame block channel bitstream set, resulting in an entropy-decoded video frame block channel bitstream set, which serves as the entropy-decoded bitstream set. Specifically, the entropy-decoded bitstream in the aforementioned entropy-decoded bitstream set can be a set of syntax elements obtained by entropy decoding the sequence parameter set, image parameter set, slice header information, and data information including residual data and motion vectors within the aforementioned video frame block channel bitstream.

[0096] The second step involves extracting decoding features from the aforementioned entropy-decoded bitstream set and the aforementioned video frame block channel bitstream set to obtain a video frame block decoding feature information set. The video frame block decoding feature information in this set can be used to characterize the complexity of the decoding process of the video frame block channel bitstream. This set can include: transform coefficient feature information, motion feature information, prediction mode feature information, video content complexity feature information, and coding parameter information. The transform coefficient feature information can include: the number of non-zero coefficients and the transform coefficient energy. The number of non-zero coefficients can be the number of non-zero coefficients in the transform coefficient matrix during the decoding process; a higher number of non-zero coefficients results in greater computational complexity for inverse quantization and inverse transform during decoding. The transform coefficient energy can be the arithmetic square root of the sum of the squares of all transform coefficients in the transform coefficient matrix, characterizing the frequency domain energy distribution of the video frame block; higher transform coefficient energy results in greater computational complexity for inverse transform. The motion feature information can include: the number of motion vectors and the amplitude of the motion vectors. The prediction mode feature information can include: prediction mode information and prediction mode number information. The aforementioned prediction mode information can characterize whether the current video frame block uses intra-frame prediction mode or inter-frame prediction mode. The aforementioned prediction mode number information can be a specific type identifier for the prediction mode. The aforementioned video content complexity feature information can include: texture complexity information, temporal activity information, and spatial activity information. The aforementioned texture complexity information can characterize the complexity of the texture within the video frame block, represented by the variance of the transform coefficients; the greater the texture complexity, the greater the computational cost of the inverse transform. The aforementioned temporal activity information can characterize the degree of difference between corresponding regions of the current video frame block and the reference video frame block. The aforementioned spatial activity information can characterize the intensity of spatial changes in pixels within the current video frame block. The aforementioned encoding parameter information can include: quantization parameters, video frame block size information, and skip label information. The aforementioned quantization parameters can characterize parameters that control quantization accuracy. The aforementioned skip label information can indicate whether the current video frame block uses a skip mode.

[0097] The third step involves inputting the aforementioned video frame block decoding feature information set into the decoding complexity ensemble prediction model to obtain a set of video frame block decoding prediction complexities. This ensemble prediction model can be a machine learning model that predicts and outputs the decoding complexity of a set of encoded video frame blocks based on the input video frame block decoding feature information set. This ensemble prediction model can be an ensemble model that extracts features using a multilayer perceptron model (including input, two hidden layers, and an output layer), and calculates the decoding complexity using a weighted summation ensemble of linear regression, decision trees, and gradient boosting trees. The video frame block decoding prediction complexities in the aforementioned set of video frame block decoding prediction complexities can characterize the complexity of decoding the entropy-decoded video frame block channel bitstream corresponding to each video frame block to be encoded.

[0098] The fourth step is to determine the video frame block dependency graph of the aforementioned set of video frame block decoding prediction complexities. This video frame block dependency graph can be a topological graph that records the dependencies between each video frame block to be encoded during the decoding process. Nodes in the video frame block dependency graph can be the video frame blocks corresponding to the bitstream obtained after the video frame blocks to be encoded have been encoded. Edges can include: intra-frame prediction dependency edges where the video frame block depends on other decoded video frame blocks that have been encoded using intra-frame coding within the same frame, and inter-frame prediction dependency edges where the video frame block depends on already decoded video frame blocks.

[0099] As an example, the aforementioned execution entity can first perform bitstream parsing on the entropy-decoded bitstream set to obtain a parsed entropy-decoded bitstream set. Secondly, it can perform motion compensation dependency analysis on the parsed entropy-decoded bitstream set to obtain a set of motion video frame block dependencies. This motion compensation dependency analysis can be performed by calculating the coding dependency of reference blocks using motion vectors. Thirdly, it can perform intra-frame prediction dependency analysis on the parsed entropy-decoded bitstream set to obtain a set of intra-frame video frame block dependencies. This intra-frame prediction dependency analysis can be performed by analyzing the spatial dependencies of video frame blocks determined by intra-frame prediction. These spatial dependencies can be the spatial location dependencies between specific video frame blocks in the intra-frame prediction mode and the current video frame block. Subsequently, the set of intra-frame video frame block dependencies, the set of motion video frame block dependencies, and the set of video frame blocks to be encoded are input into a graph database to obtain an initial decoding dependency graph. Next, based on the dependency path set corresponding to the initial decoding dependency graph and the set of video frame block decoding prediction complexities, the set of edge weight values ​​for each edge in the initial decoding dependency graph is determined. This determination can be made by a weighted sum of the path lengths of the dependency path set and the video frame block decoding prediction complexities. Longer path lengths result in lower weights, used to distinguish between strong and weak dependencies. Finally, the set of edge weight values ​​is added to the initial decoding dependency graph to obtain the video frame block dependency graph.

[0100] Fifth, based on the above video frame block dependency graph, the above video frame block decoding prediction complexity set, and the above entropy decoding bitstream set, the above decoding splicing thread set is dynamically load-adjusted to obtain the adjusted decoding splicing thread set.

[0101] As an example, the aforementioned execution entity can first use simulated annealing to perform load balancing grouping processing on the aforementioned entropy decoding bitstream set based on the aforementioned video frame block dependency graph and the aforementioned video frame block decoding prediction complexity set, thereby obtaining a grouped entropy decoding bitstream set. Each grouped entropy decoding bitstream in the aforementioned grouped entropy decoding bitstream set can be a bitstream with a basically equal and balanced load. Then, the aforementioned grouped entropy decoding bitstream set is determined as the bitstream corresponding to the aforementioned decoding splicing thread set, resulting in the adjusted decoding splicing thread set.

[0102] The sixth step involves controlling the adjusted decoding and splicing thread set to decode the corresponding entropy-decoded bitstream, resulting in a set of thread-decoded video frame blocks. This decoding process may include: inverse quantization, inverse transform, prediction compensation, loop filtering, and frame reconstruction.

[0103] Step 7: Determine the overlapping gradient map, overlapping frequency band sub-map, and overlapping motion vector information set of the overlapping video frame block set included in the above-mentioned thread-decoded video frame block set. The overlapping gradient map in the overlapping gradient map set represents the gradient intensity of each pixel, i.e., the speed and direction of image grayscale value change. The overlapping frequency band sub-map in the overlapping frequency band sub-map set represents the edge information of overlapping video frame blocks at different frequencies. Low-frequency edge information can include the contours and smooth regions of overlapping video frame blocks, while high-frequency edge information includes image details and edge information of overlapping video frame blocks. The overlapping motion vector information in the overlapping motion vector information set represents the intensity of motion of foreground objects in overlapping video frame blocks. The overlapping gradient map can be obtained using the Sobel operator. The overlapping frequency band sub-map can be obtained using the Laplacian pyramid. The overlapping motion vector information can be obtained using optical flow matching.

[0104] Step 8: Based on the aforementioned overlapping gradient map set, overlapping frequency band sub-map set, and overlapping motion vector information set, generate an overlapping stitching weighted weight set. The overlapping stitching weights in this set can be obtained by weighting the corresponding overlapping gradient map, overlapping frequency band sub-map, and overlapping motion vector information using a Gaussian function. As an example, the execution entity can use a Gaussian function to generate the overlapping stitching weighted weight set based on the aforementioned overlapping gradient map set, overlapping frequency band sub-map set, and overlapping motion vector information set.

[0105] Step 9: Based on the aforementioned overlapping and weighted set, the aforementioned set of video frame blocks to be decoded by the thread is spliced ​​together to obtain the decoded video.

[0106] As an example, the aforementioned execution entity can first transform and superimpose the overlapping video frame block set using a weighted Alpha blending algorithm based on the overlapping and stitching weighted weight set, to obtain an overlapping and superimposed video frame block set. The weighted Alpha blending algorithm can be an algorithm that replaces the weights in the Alpha blending algorithm with the aforementioned overlapping and stitching weighted weight set. Then, using multi-threaded DMA, the overlapping and non-overlapping video frame block sets included in the thread-decoded video frame block set are directly copied to the display buffer. Finally, within the display buffer, the non-overlapping region set and the overlapping and superimposed video frame block set are stitched together to obtain the decoded video.

[0107] The above-described technical solution and its related content, as an inventive point of this disclosure, solve the technical problem mentioned in the background art: "low efficiency in decoding and video frame stitching, thread idleness between threads leading to resource waste, prolonged video decoding and stitching time, and reduced decoded video quality." The factors leading to low efficiency in decoding and video frame stitching, resource waste, prolonged video decoding and stitching time, and reduced decoded video quality are often as follows: During the block decoding process after encoding the video frame block set, there are many video frame block decoding dependencies, and there is a serial delay between decoding and video frame block stitching, resulting in low efficiency in decoding and video frame stitching, thread idleness between threads leading to resource waste, prolonged video decoding and stitching time, and reduced decoded video quality. If these factors are resolved, the efficiency of decoding and video frame stitching can be improved, the waste of thread resources reduced, the video decoding and stitching time shortened, and the decoded video quality improved. To achieve this effect, this disclosure first extracts the video frame block decoding feature information set and inputs it into the decoding complexity ensemble prediction model. Predicting using the decoding complexity ensemble prediction model improves the decoding complexity of the identified video frame blocks, enhances prediction accuracy, and provides foundational data for subsequent dynamic load adjustment. Second, a video frame block dependency graph is constructed and combined with the video frame block decoding prediction complexity set for dynamic load adjustment before decoding. The video frame block dependency graph can display and quantify the dependencies between video frame blocks, improving the efficiency and accuracy of subsequent decoding and dynamic load adjustment. Dynamic load adjustment can balance the thread load of the adjusted decoding splicing thread set, reducing thread resource waste, improving decoding efficiency and accuracy, and enhancing the video frame quality of the thread-decoded video frame block set. Then, overlapping gradient maps, overlapping frequency band sub-maps, and overlapping motion vector information sets are determined and superimposed on the overlapping regions. By considering the weighted summation of multiple factors, the accuracy of overlapping region superposition can be improved, effectively reducing the block effect caused by considering a single factor and improving the image quality of video frame blocks in the overlapping regions. Finally, by using multi-threaded DMA and an efficient frame buffer for video frame stitching, redundant copying of video frame blocks can be avoided, memory access and resource waste during the stitching stage can be reduced, stitching efficiency can be improved, latency can be reduced, and the quality of decoded video can be improved.

[0108] Step 107: In response to determining that the video display terminal has received the decoded video, perform video frame display processing on the decoded video.

[0109] In some embodiments, the execution entity may perform video frame display processing on the decoded video in response to determining that the video display terminal has received the decoded video. The video display terminal may be a display terminal for displaying decoded video composed of sets of decoded video frame blocks. For example, the video display terminal may be an HDMI (High Definition Multimedia Interface) display. Figure 5 As shown, Figure 5 An internal schematic diagram of the entire process implemented jointly from step 105 to step 107 is shown.

[0110] The various embodiments of this disclosure have the following beneficial effects: the video encoding and decoding display method of some embodiments of this disclosure can improve the balance of the encoded video stream, improve the stability and real-time performance of the transmitted video, and reduce the latency of video transmission. Specifically, the reason for the low stability of the related video stream, the high latency of video output, and the extended transmission and display time of video frames is that: the entire video frame is encoded without adjustment of key video frames and ordinary frames. When there are bandwidth fluctuations during video transmission, the stability of the video stream is low. At the same time, the decoding process requires waiting for the key video frames to be decoded before decoding, resulting in a large decoding delay, which leads to a high latency of video output and an extended transmission and display time of video frames. Based on this, the video encoding and decoding display method of some embodiments of this disclosure can first, in response to determining that the image splitting and encoding sending end has received the video to be encoded, split each video frame to be encoded in the video to be encoded according to the encoding device information of the image splitting and encoding sending end, to generate a group of video frame blocks to be encoded, and obtain a set of video frame blocks to be encoded. Here, splitting the video frames to be encoded reduces the amount of encoded data during video transmission and provides a data foundation for subsequent key frame block staggering. Secondly, in response to the detection that the set of video frame blocks to be encoded meets the interruption trigger condition, a set of video frame block encoding channels that meet the interruption trigger condition is determined. Here, meeting the terminal trigger condition does not require waiting for the entire video frame to be transmitted before subsequent encoding, reducing transmission encoding latency. Thirdly, key video frame block staggering is performed on each video frame block to be encoded within the aforementioned video frame block encoding channel set, resulting in a key video frame block encoding set and a channel delay encoding time difference set. Here, key video frame block staggering achieves a uniform distribution of key video frame blocks in each video frame block encoding channel, reducing the impact of transmission bandwidth fluctuations on the stability of the encoded bitstream during video transmission, reducing bandwidth occupation during peak transmission periods, and improving the stability of bitstream transmission. Next, based on the aforementioned key video frame block encoding set and the aforementioned channel delay encoding time difference set, the aforementioned video frame block encoding channel set is controlled to encode each corresponding video frame block to be encoded, resulting in a video frame block channel bitstream set. Here, the encoding efficiency of video encoding and the transmission stability of the video frame block channel bitstream set to the decoder-merge receiver can be improved. Subsequently, in response to determining that the decoder-merge receiver has received the aforementioned video frame block channel bitstream set, the decoding and splicing threads for each video frame block channel bitstream in the set are determined, resulting in a decoding and splicing thread set. Here, multi-threaded decoding processing using the decoding and splicing thread set can improve decoding efficiency and shorten decoding time. Then, based on the aforementioned decoding and splicing thread set, video frame block decoding and splicing processing is performed on the aforementioned video frame block channel bitstream set to obtain the decoded video.Here, by using an encrypted splicing thread set to splice multi-threaded video frame blocks into video frames, the efficiency and effectiveness of splicing video frame blocks into video frames can be improved, the splicing time can be shortened, and the splicing resource consumption can be reduced. Finally, in response to the determination that the video display end has received the above-mentioned decoded video, the above-mentioned decoded video is processed for video frame display. Here, the display time of the decoded video can be reduced and the display quality of the decoded video can be improved. Thus, this video encoding and decoding display method can directly perform subsequent encoding processing when the video frame block meets the interrupt triggering condition through video frame splitting processing, without waiting for the entire video frame to be transmitted, which can reduce latency. In addition, the peak-shifting processing of key video frame blocks can evenly distribute the layout of key video frame blocks, improve the balance of the bitstream, reduce the impact of transmission bandwidth fluctuations, improve the efficiency of encoding and transmission, and reduce latency. By using a decoding splicing thread set for video frame block decoding and splicing processing, decoding efficiency and image synthesis speed can be improved, and the video frame transmission and display time can be shortened.

[0111] Further reference Figure 6 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a video encoding / decoding display device, which are similar to... Figure 1 Corresponding to the method embodiments shown, this video encoding / decoding display device can be specifically applied to various electronic devices.

[0112] like Figure 6As shown, a video encoding / decoding display device 600 includes: a splitting unit 601, a first determining unit 602, a key video frame block staggering unit 603, a control unit 604, a second determining unit 605, a video frame block decoding and splicing unit 606, and a video frame display unit 607. The splitting unit 601 is configured to: in response to a determined image splitting encoding transmitter receiving a video to be encoded, split each video frame to be encoded in the video to be encoded according to the encoding device information of the image splitting encoding transmitter to generate a set of video frame blocks to be encoded. The first determining unit 602 is configured to: in response to detecting that the set of video frame blocks to be encoded meets an interrupt triggering condition, determine a set of video frame block encoding channels that meets the interrupt triggering condition. The key video frame block staggering unit 603 is configured to: perform key video frame block staggering processing on each video frame block to be encoded included in the set of video frame block encoding channels to obtain a set of key video frame encoding blocks and a set of channel delay encoding time differences. Control unit 604 is configured to: control the video frame block encoding channel set to encode each corresponding video frame block to be encoded according to the key video frame encoding block set and the channel delay encoding time difference set, to obtain a video frame block channel bitstream set. Second determining unit 605 is configured to: determine the decoding and splicing thread of each video frame block channel bitstream in the video frame block channel bitstream set in response to the determination that the decoding and merging receiving end has received the video frame block channel bitstream set, to obtain a decoding and splicing thread set. Video frame block decoding and splicing unit 606 is configured to: perform video frame block decoding and splicing processing on the video frame block channel bitstream set according to the decoding and splicing thread set, to obtain a decoded video. Video frame display unit 607 is configured to: perform video frame display processing on the decoded video in response to the determination that the video display end has received the decoded video.

[0113] It is understandable that the units described in the video encoding / decoding display device 600 are related to the reference. Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the video encoding / decoding display device 600 and the units contained therein, and will not be repeated here.

[0114] The following is for reference. Figure 7 It shows a schematic diagram of the structure of an electronic device (e.g., an electronic device) 700 suitable for implementing some embodiments of the present disclosure. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0115] like Figure 7As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0116] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 7 Each box shown can represent a device or multiple devices as needed.

[0117] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0118] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0119] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0120] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: in response to determining that the image splitting and encoding transmitter has received a video to be encoded, split each video frame to be encoded in the video to be encoded according to the encoding device information of the image splitting and encoding transmitter to generate a set of video frame blocks to be encoded; in response to detecting that the set of video frame blocks to be encoded satisfies an interrupt triggering condition, determine a set of video frame block encoding channels that satisfies the interrupt triggering condition; and perform key video frame block staggering processing on each video frame block to be encoded included in the set of video frame block encoding channels to obtain key video frame encoding channels. The code block set and the channel delay coding time difference set are used. Based on the above key video frame coding block set and the above channel delay coding time difference set, the above video frame block coding channel set is controlled to perform video coding on each corresponding video frame block to be encoded, to obtain a video frame block channel code stream set. In response to determining that the decoding and merging receiver has received the above video frame block channel code stream set, the decoding and splicing thread of each video frame block channel code stream in the above video frame block channel code stream set is determined, to obtain a decoding and splicing thread set. Based on the above decoding and splicing thread set, the above video frame block channel code stream set is subjected to video frame block decoding and splicing processing to obtain a decoded video. In response to determining that the video display end has received the above decoded video, the above decoded video is subjected to video frame display processing.

[0121] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0123] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a splitting unit, a first determining unit, a key video frame block staggering unit, a control unit, a second determining unit, a video frame block decoding and splicing unit, and a video frame display unit. The names of these units do not necessarily limit the specific unit itself. For example, the splitting unit may also be described as "a unit that, in response to a determined image splitting encoding transmitter receiving a video to be encoded, splits each video frame included in the video to be encoded according to the encoding device information of the image splitting encoding transmitter to generate a set of video frame block groups to be encoded."

[0124] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0125] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A video encoding / decoding and display method, comprising: In response to determining that the image splitting and encoding sending end has received the video to be encoded, the encoding device information of the image splitting and encoding sending end is used to split each video frame to be encoded in the video to be encoded to generate a video frame block group, thereby obtaining a video frame block group set to be encoded. In response to detecting that the set of video frame blocks to be encoded meets the interrupt triggering condition, a set of video frame block encoding channels that meets the interrupt triggering condition is determined; The key video frame block staggering processing is performed on each video frame block to be encoded included in the video frame block encoding channel set to obtain the key video frame encoding block set and the channel delay encoding time difference set. Based on the key video frame coding block set and the channel delay coding time difference set, the video frame block coding channel set is controlled to perform video coding on each corresponding video frame block to be encoded, thereby obtaining a video frame block channel bitstream set. In response to the determination that the decoding and merging receiver has received the video frame block channel bitstream set, the decoding and splicing thread of each video frame block channel bitstream in the video frame block channel bitstream set is determined, and a decoding and splicing thread set is obtained; Based on the decoding and splicing thread set, the video frame block channel bitstream set is subjected to video frame block decoding and splicing processing to obtain the decoded video; In response to determining that the video display terminal has received the decoded video, video frame display processing is performed on the decoded video.

2. The method according to claim 1, wherein, The step of determining the video frame block encoding channel set that satisfies the interrupt triggering condition in response to detecting that the set of video frame blocks to be encoded meets the interrupt triggering condition includes: Create a set of video frame block buffers and a set of buffer fill status monitors, wherein there is a one-to-one correspondence between the set of video frame block buffers and the set of buffer fill status monitors; In response to determining that at least one buffer filling status monitor in the buffer filling status monitor set represents the completion of video frame block buffer filling, at least one set of video frame tiles to be encoded is stored in at least one video frame block buffer corresponding to the at least one buffer filling status monitor, as the target set of video frame tiles to be encoded. In response to determining that the target set of video frame tiles to be encoded satisfies the interrupt triggering condition, a set of video frame tile filling interrupt signals is generated. Based on the video frame block filling interrupt signal set, determine the video frame block encoding channel set that meets the interrupt triggering conditions.

3. The method according to claim 1, wherein, The step of performing key video frame block staggering processing on each video frame block to be encoded included in the video frame block coding channel set to obtain a key video frame coding block set and a channel delay coding time difference set includes: The video frame block coding channel set is filled and sorted in a timing order to obtain the video frame block coding channel sequence. Based on the preset key video frame block staggering conditions, the position information of the key video frame blocks of each video frame block encoding channel included in the video frame block encoding channel sequence is determined, and a key video frame block position information set is obtained. Based on the channel key video frame block location information set, determine the channel delay coding time difference set and the key video frame coding block set of the video frame block coding channel sequence.

4. The method according to claim 1, wherein, The step of performing video frame block decoding and splicing processing on the video frame block channel bitstream set according to the decoding splicing thread set to obtain decoded video includes: The decoding and splicing thread set is controlled to decode the video frame block channel bitstream set to obtain a decoded video frame block set. The display buffer of the video display terminal is cleared to obtain the cleared display buffer; The non-overlapping decoded video frame block set, comprising the decoded video frame block set, is stored in the zeroed display buffer via direct memory access. For each overlapping decoded video frame block region in the set of overlapping decoded video frame block regions included in the decoded video frame block set, the following pixel blending steps are performed: An incremental transparency transformation is performed on the overlapping region of the first decoded video frame block corresponding to the overlapping decoded video frame block region to obtain the first overlapping region transparency set. A decreasing transparency transformation is performed on the overlapping region of the second decoded video frame block corresponding to the overlapping decoded video frame block region to obtain the transparency set of the second overlapping region; Based on the first overlapping region transparency set and the second overlapping region transparency set, pixel mixing is performed on the overlapping regions of the first decoded video frame block and the second decoded video frame block to obtain the decoded overlapping video frame block region. The obtained decoded overlapping video frame block regions are transferred to the cleared display buffer via direct memory access. The decoded video is obtained by splicing together the sets of overlapping and non-overlapping decoded video frame blocks stored in the zeroed display buffer.

5. A video encoding / decoding display device, comprising: The splitting unit is configured to, in response to determining that the image splitting and encoding transmitter has received the video to be encoded, split each video frame to be encoded in the video to be encoded according to the encoding device information of the image splitting and encoding transmitter, so as to generate a video frame block group to be encoded, and obtain a set of video frame block groups to be encoded. The first determining unit is configured to determine a set of video frame block encoding channels that satisfy the interrupt triggering condition in response to detecting that the set of video frame block groups to be encoded satisfies the interrupt triggering condition. The key video frame block staggering unit is configured to perform key video frame block staggering processing on each video frame block to be encoded included in the video frame block encoding channel set to obtain the key video frame encoding block set and the channel delay encoding time difference set. The control unit is configured to control the video frame block encoding channel set to perform video encoding on each corresponding video frame block to be encoded according to the key video frame encoding block set and the channel delay encoding time difference set, so as to obtain a video frame block channel bitstream set. The second determining unit is configured to, in response to the determining decoding and merging receiver receiving the video frame block channel bitstream set, determine the decoding and splicing thread of each video frame block channel bitstream in the video frame block channel bitstream set, and obtain a decoding and splicing thread set; The video frame block decoding and splicing unit is configured to perform video frame block decoding and splicing processing on the video frame block channel bitstream set according to the decoding and splicing thread set to obtain decoded video; The video frame display unit is configured to perform video frame display processing on the decoded video in response to determining that the video display terminal has received the decoded video.

6. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-4.

7. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-4.

Images