Video processing method and device and related equipment

Abstract

The invention provides a video processing method and device and related equipment, and relates to the technical field of video processing. The method comprises the following steps: acquiring a coding data stream of a video, the video comprising a plurality of video frames, each video frame comprising a plurality of fragments; for each fragment, analyzing the coded data stream of the video to obtain the fragment data stream of the fragment and the header information of the fragment data stream, triggering to generate an entropy decoding task of the fragment, and putting the entropy decoding task of the fragment into a task queue, the entropy decoding task being an entropy decoding task performed by adopting CABAC; selecting a to-be-processed task from the task queue by adopting a multi-thread parallel mode; and performing parallel processing on the to-be-processed task according to the fragmented data stream corresponding to the to-be-processed task and the header information of the fragmented data stream corresponding to the to-be-processed task. According to the method, parallel processing of CABAC entropy decoding is realized, and the decoding speed of a decoder is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a video processing method, a video processing device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In the video coding and decoding framework, the role of the entropy coding module is to encode syntax elements such as video residuals and coding information into a binary code stream that can be stored and transmitted. At the decoding end, the entropy decoding module performs the opposite operation of the entropy coding module, that is, to restore the stored and transmitted binary code stream to syntax elements such as video residuals and coding information.

[0003] CABAC (Context-Adaptive Binary Arithmetic Coding) is an entropy coding technology in video coding standards. If CABAC is used for entropy coding at the encoding end, the corresponding entropy decoding process is used at the decoding end. However, in the related art, the decoder cannot implement parallel processing of CABAC, which affects the decoder speed in a multi-threaded environment.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a video processing method, a video processing device, an electronic device, a computer-readable storage medium, and a computer program product to overcome the above problems or at least partially solve the above problems.

[0006] The embodiment of the present disclosure provides a video processing method, the method comprising: obtaining a coded data stream of a video; the video comprises a plurality of video frames, and the video frames comprise a plurality of slices; for each of the slices, parsing the coded data stream of the video to obtain a slice data stream of the slice and header information of the slice data stream; triggering the generation of an entropy decoding task of the slice, and placing the entropy decoding task of the slice into a task queue; the entropy decoding task is an entropy decoding task performed using CABAC; selecting a task to be processed from the task queue in a multi-threaded parallel manner; performing parallel processing on the task to be processed according to the slice data stream corresponding to the task to be processed and the header information of the slice data stream corresponding to the task to be processed; the slice data stream corresponding to the task to be processed is the slice data stream corresponding to the slice to which the task to be processed belongs.

[0007] In some embodiments of the present disclosure, the slice includes at least one coding tree unit; wherein, the triggering of generating an entropy decoding task for the slice and placing the entropy decoding task for the slice into a task queue includes: based on a preset triggering generation rule for the entropy decoding task, triggering the generation of an entropy decoding task for the at least one coding tree unit and placing the generated entropy decoding task into the task queue.

[0008] In some embodiments of the present disclosure, the triggering generation rule of the preset entropy decoding task, triggering the generation of the entropy decoding task of the at least one coding tree unit, and placing the generated entropy decoding task into the task queue includes: triggering the generation of the entropy decoding task of the first coding tree unit in the at least one coding tree unit, and placing the entropy decoding task of the first coding tree unit into the task queue; the first coding tree unit is the first coding tree unit to be decoded in the slice; taking the first coding tree unit as the current coding tree unit, triggering the generation of one or more entropy decoding tasks of the second coding tree units based on the preset triggering generation rule of the entropy decoding task, and placing the entropy decoding tasks of the one or more second coding tree units into the task queue; taking the second coding tree unit as the new current coding tree unit, continuing to trigger the generation of the entropy decoding tasks of the coding tree units in the at least one coding tree unit based on the preset triggering generation rule of the entropy decoding task, and placing the generated entropy decoding task into the task queue, until the entropy decoding tasks of all the coding tree units in the at least one coding tree unit are triggered and placed in the task queue.

[0009] In some embodiments of the present disclosure, the trigger generation rule of the preset entropy decoding task includes at least one of the following options: in response to completing the entropy decoding task of the current coding tree unit, triggering the generation of the entropy decoding task of the right coding tree unit of the current coding tree unit; in response to completing the entropy decoding task of the current coding tree unit, triggering the generation of the entropy decoding task of the lower coding tree unit of the current coding tree unit; in response to completing the entropy decoding task of the current coding tree unit, triggering the generation of the entropy decoding task of the next coding tree unit of the current coding tree unit; the next coding tree unit of the current coding tree unit is determined according to a specific scanning or processing order.

[0010] In some embodiments of the present disclosure, the method further includes: storing the fragmented data stream of the fragment in a code stream memory.

[0011] In some embodiments of the present disclosure, the method further includes: dividing the sliced ​​data stream of the slice into at least one sub-sliced ​​data stream; for each of the sub-sliced ​​data streams, allocating a memory pointer to the sub-sliced ​​data stream; the memory pointer is used to locate the position of the sub-sliced ​​data stream stored in the code stream memory.

[0012] In some embodiments of the present disclosure, the method further includes: in the process of decoding the sub-segment data stream, updating the current decoding position index of the sub-segment data stream; the current decoding position index of the sub-segment data stream is used to indicate the position to which the sub-segment data stream is currently parsed.

[0013] In some embodiments of the present disclosure, the method further includes: allocating multiple entropy decoders according to the number of multi-threads so that each entropy decoder corresponds to a thread; selecting a sub-slice data stream to be processed from different sub-slice data streams in a multi-threaded parallel manner, and using the allocated entropy decoder to perform entropy decoding on the sub-slice data stream to be processed.

[0014] The embodiment of the present disclosure provides a video processing device, the device comprising: a data stream acquisition module, configured to acquire a coded data stream of a video; the video comprises a plurality of video frames, and the video frames comprise a plurality of slices; a data stream parsing module, configured to parse the coded data stream of the video for each of the slices, and obtain a slice data stream of the slice and header information of the slice data stream; trigger the generation of an entropy decoding task of the slice, and put the entropy decoding task of the slice into a task queue; the entropy decoding task is an entropy decoding task performed using CABAC; a processing module, configured to select a task to be processed from the task queue in a multi-threaded parallel manner; perform parallel processing on the task to be processed according to the slice data stream corresponding to the task to be processed and the header information of the slice data stream corresponding to the task to be processed; the slice data stream corresponding to the task to be processed is the slice data stream corresponding to the slice to which the task to be processed belongs.

[0015] An embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the above-mentioned video processing method.

[0016] An embodiment of the present disclosure provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the above-mentioned video processing method.

[0017] The embodiment of the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above-mentioned video processing method is implemented.

[0018] The video processing method provided by the embodiment of the present disclosure parses the encoded data stream of the video to obtain the sliced ​​data stream of each slice and the header information of the sliced ​​data stream, which provides a basis for subsequent decoding processing; after parsing the header information of the sliced ​​data stream of each slice, it triggers the generation of the CABAC entropy decoding task of the slice and puts the CABAC entropy decoding task of the slice into a task queue, and then selects multiple CABAC entropy decoding tasks from the task queue in a multi-threaded manner for parallel processing, thereby realizing the parallel processing of CABAC entropy decoding and improving the decoding speed of the decoder.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0021] Figure 1 A schematic diagram showing the memory and data structure of a decoder in the related art;

[0022] Figure 2 A schematic diagram showing an exemplary system architecture to which the video processing method according to an embodiment of the present disclosure can be applied;

[0023] Figure 3 A flow chart of a video processing method according to an embodiment of the present disclosure is shown;

[0024] Figure 4 A flowchart of a CABAC entropy decoding task for triggering generation of each CTU in an embodiment of the present disclosure is shown;

[0025] Figure 5 A decoder architecture diagram is shown that can implement the video processing method of the embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram showing the memory and data structure of a decoder in an embodiment of the present disclosure is shown;

[0027] Figure 7 A block diagram of a video processing device according to an embodiment of the present disclosure is shown;

[0028] Figure 8 A schematic structural diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0030] The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0031] The collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of the user personal information involved in this disclosure are in compliance with the relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0032] The accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in at least one hardware module or integrated circuit, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0033] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0034] In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of at least one element / component / etc.; the term "plurality" refers to two or more; the terms "comprising", "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first", "second" and "third" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0035] In the video encoding and decoding framework, if CABAC is used for entropy encoding at the encoding end, the corresponding entropy decoding process is used at the decoding end. The current mainstream video encoding standards provide algorithm specifications such as WPP (Wavefront Parallel Processing) that can perform parallel processing of CABAC, but there is no regulation on how to implement parallel processing of CABAC. In related technologies, due to design limitations such as data structure and bitstream memory structure, the decoder cannot implement parallel processing of CABAC, which affects the decoder speed in a multi-threaded environment.

[0036] Figure 1 A schematic diagram showing the memory and data structure of a decoder in the related art. Figure 1 In the video code stream file input, the decoder will read the code stream of each NALU (Network Abstract Layer Unit) in the AU (Access Unit) in turn and save the code stream in a single code stream memory, and decode each NALU serially. Among them, in the video coding standard, AU is the basic data unit that can be randomly accessed and decoded. An AU usually corresponds to a complete frame of an image or a group of independently decodable images. An AU can contain one or more NALUs (Network Abstract Layer Units), and NALU is the actual container containing the encoded video data.

[0037] For the entropy decoder, it will start decoding from the first bit of each NALU stream stored in the memory until all bits are decoded. From the perspective of the video, the decoder will start decoding from the first CTU (Coding Tree Unit) of each frame of video, and decode each CTU serially in the order of raster scanning until all CTUs of this frame are decoded.

[0038] In the related art, the decoder uses a single code stream memory and a single entropy decoder state management, which has the following problems: the decoder can only decode each slice of the video frame serially, and cannot perform parallel decoding between NALUs; the decoder can only perform serial entropy decoding, and cannot perform parallel entropy decoding, which makes the speed of the multi-threaded decoder slowed down by the serial entropy decoding process in actual application. Therefore, the decoder of the related art has the problem of low decoder parallel processing capability and slow decoding speed.

[0039] In order to solve the above problems, the embodiments of the present disclosure provide a video processing method, apparatus and related equipment, which parse the encoded data stream of a video to obtain the sliced ​​data stream of each slice and the header information of the sliced ​​data stream, thereby providing a basis for subsequent decoding processing; after parsing the header information of the sliced ​​data stream of each slice, trigger the generation of a CABAC entropy decoding task of the slice and put the CABAC entropy decoding task of the slice into a task queue, and then select multiple CABAC entropy decoding tasks from the task queue in a multi-threaded manner for parallel processing, thereby realizing parallel processing of CABAC entropy decoding and improving the decoding speed of the decoder.

[0040] Figure 2 The diagram shows an exemplary system architecture to which the video processing method according to the embodiment of the present disclosure can be applied.

[0041] like Figure 2 As shown, the system architecture may include a server 201, a network 202, and a client 203. The network 202 is used to provide a medium for a communication link between the client 203 and the server 201. The network 202 may include various connection types, such as wired, wireless communication links, or optical fiber cables.

[0042] In an exemplary embodiment, the client 203 for data transmission with the server 201 may include but is not limited to a smart phone, a tablet computer, a laptop computer, a smart speaker, a digital assistant, an AR (Augmented Reality) device, a VR (Virtual Reality) device, a smart wearable device, etc., or the client 203 may also be a personal computer, such as a laptop computer and a desktop computer, etc. Optionally, the operating system running on the electronic device may include but is not limited to Android system, IOS system, Linux, Windows, etc.

[0043] Server 201 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In some practical applications, server 201 can also be a server of a network platform, which can be, for example, a trading platform, a live broadcast platform, a social platform, or an audio platform, etc., which is not limited in the embodiments of the present disclosure. Among them, the server can be a single server or a cluster formed by multiple servers, and the present disclosure does not limit the specific architecture of the server.

[0044] In some embodiments of the present disclosure, the server 201 may be used for a video processing process that includes: obtaining a coded data stream of a video, wherein the video includes multiple video frames, and the video frames include multiple slices; for each slice, parsing the coded data stream of the video to obtain a slice data stream of the slice and header information of the slice data stream; triggering the generation of an entropy decoding task for the slice, and placing the entropy decoding task for the slice into a task queue; selecting a task to be processed from the task queue in a multi-threaded parallel manner; and performing parallel processing on the tasks to be processed according to the slice data stream corresponding to the task to be processed and the header information of the slice data stream corresponding to the task to be processed, wherein the slice data stream corresponding to the task to be processed is the slice data stream corresponding to the slice to which the task to be processed belongs.

[0045] In addition, it should be noted that Figure 2 What is shown is only one application environment of the method for processing video encoding parameters provided by the present disclosure. Figure 2 The numbers of the clients 203, networks 202 and servers 201 are merely illustrative, and any number of clients, networks and servers may be provided according to actual needs.

[0046] Figure 3 A flowchart of a video processing method according to an embodiment of the present disclosure is shown. Figure 3 The execution subject of the method provided in the embodiment may be any electronic device, such as Figure 2 The server 201 in the embodiment is also as follows Figure 2 Client 203 in, and Figure 2 The server 201 and the client 203 in the embodiment jointly implement the video processing method, but the present disclosure is not limited thereto. Figure 3 The video processing method provided by the embodiment of the present disclosure includes the following steps.

[0047] Step S310, obtaining a coded data stream of a video; wherein the video includes a plurality of video frames, and the video frame includes a plurality of slices.

[0048] In the embodiments of the present disclosure, the encoded data stream of the video refers to a code stream obtained by encoding the video. The video includes multiple video frames, and each video frame is divided into multiple slices.

[0049] Step S320, for each slice, parse the encoded data stream of the video to obtain the slice data stream and header information of the slice data stream; trigger the generation of the entropy decoding task of the slice, and put the entropy decoding task of the slice into the task queue; wherein the entropy decoding task is an entropy decoding task performed using CABAC.

[0050] In the disclosed embodiment, the encoded data stream of the video is parsed to obtain the sliced ​​data stream of each slice and the header information of the sliced ​​data stream. In a possible implementation, one slice corresponds to one NALU, that is, one NALU contains the sliced ​​data stream of one slice.

[0051] In the disclosed embodiment, for a slice, after parsing and obtaining the header information of the slice data stream, the entropy decoding task of the slice is triggered, and the entropy decoding task of the slice is placed in a task queue. The task queue is used to store tasks waiting to be processed, which can be, but is not limited to, a first-in-first-out data structure.

[0052] In the disclosed embodiment, the entropy decoding task is an entropy decoding task performed using CABAC. CABAC is an efficient entropy coding method commonly used in video coding standards. CABAC optimizes coding efficiency by utilizing the statistical characteristics of coded data, and corresponding CABAC entropy decoding is also required at the decoding end to restore the original data.

[0053] In a possible implementation, the decoder includes a header information parser and a triggered parallel decoder. The encoded data stream of the video is input into the header information parser, and the header information parser sequentially reads the slice data stream of each slice of each video frame in the encoded data stream of the input video, and parses the header information of each slice data stream. After parsing the header information of a slice data stream, the CABAC entropy decoding task of the slice is triggered, and the CABAC entropy decoding task of the slice is placed in the task queue of the triggered parallel decoder. Among them, the CABAC entropy decoding task refers to the entropy decoding task performed using CABAC.

[0054] Step S330, adopting a multi-threaded parallel approach, selects tasks to be processed from the task queue; performs parallel processing on the tasks to be processed according to the sharded data stream corresponding to the tasks to be processed and the header information of the sharded data stream corresponding to the tasks to be processed; wherein the sharded data stream corresponding to the tasks to be processed is the sharded data stream corresponding to the shard to which the tasks to be processed belong.

[0055] In the disclosed embodiment, the CABAC entropy decoding tasks of the slices are stored in the task queue. In a multi-threaded environment, a corresponding number of CABAC entropy decoding tasks are selected from the task queue according to the number of idle threads. Then, according to the slice data stream corresponding to the slice to which the selected CABAC entropy decoding task belongs and the header information of the slice data stream, the CABAC entropy decoding tasks are processed in parallel, thereby realizing parallel processing of CABAC.

[0056] In the video processing method provided by the embodiment of the present disclosure, the encoded data stream of the video is parsed to obtain the sliced ​​data stream of each slice and the header information of the sliced ​​data stream, which provides a basis for subsequent decoding processing; after parsing the header information of the sliced ​​data stream of each slice, the CABAC entropy decoding task of the slice is triggered to be generated and the CABAC entropy decoding task of the slice is placed in a task queue, and then a plurality of CABAC entropy decoding tasks are selected from the task queue in a multi-threaded manner for parallel processing, thereby realizing the parallel processing of CABAC entropy decoding and improving the decoding speed of the decoder.

[0057] In some embodiments of the present disclosure, a slice includes at least one CTU. Triggering the generation of an entropy decoding task for a slice and placing the entropy decoding task for the slice into a task queue may include: triggering the generation of an entropy decoding task for at least one CTU based on a preset trigger generation rule for the entropy decoding task, and placing the generated entropy decoding task into the task queue.

[0058] In the disclosed embodiment, the analysis of the encoded data stream of the video is further refined to the CTU level, wherein the CTU is a basic processing unit in video encoding and decoding, and each slice is divided into at least one CTU.

[0059] In the disclosed embodiment, for each slice, after parsing the slice data stream of the slice and the header information of the slice data stream, a CABAC entropy decoding task (i.e., an entropy decoding task performed using CABAC) of the slice is triggered. Specifically, based on a preset trigger generation rule for the entropy decoding task, the CABAC entropy decoding task of each CTU can be triggered in sequence, and the CABAC entropy decoding task of each CTU can be placed in a task queue.

[0060] In some embodiments of the present disclosure, the trigger generation rule of the preset entropy decoding task includes at least one of the following options:

[0061] (1) In response to completing the entropy decoding task of the current CTU, triggering the generation of the entropy decoding task of the right CTU of the current CTU.

[0062] In the disclosed embodiment, after the CABAC entropy decoding task of the current CTU is completed, the CABAC entropy decoding task of the right CTU of the current CTU may be triggered.

[0063] (2) In response to completing the entropy decoding task of the current CTU, triggering the generation of the entropy decoding task of the lower CTU of the current CTU.

[0064] In the disclosed embodiment, when WPP is turned on, after the current CTU completes the CABAC entropy decoding task, the CABAC entropy decoding task of the lower CTU of the current CTU may be triggered.

[0065] (3) In response to completing the entropy decoding task of the current CTU, triggering the entropy decoding task of generating the next CTU of the current CTU; the next CTU of the current CTU is determined according to a specific scanning or processing order.

[0066] In the disclosed embodiment, the CTU may be processed in a specific scanning or processing order, such as zigzag scanning, progressive scanning, etc. In this case, after the CABAC entropy decoding task of the current CTU is completed, the generation of a CABAC entropy decoding task of the next CTU may be triggered.

[0067] In the embodiment of the present disclosure, dependency rules between CTUs may also be preset, including at least one of the following options:

[0068] (1) The trigger generation of the entropy decoding task of the current CTU depends on the completion of the entropy decoding task of the CTU to the left of the current CTU.

[0069] In the disclosed embodiment, before entropy decoding is performed on the current CTU, it is ensured that the CTU on the left side of the current CTU has completed the CABAC entropy decoding task. That is, for the current CTU, after the CTU on the left side has completed the CABAC entropy decoding task, the CABAC entropy decoding task of the current CTU can be triggered.

[0070] (2) The triggering generation of the entropy decoding task of the current CTU depends on whether the entropy decoding task of the upper CTU of the current CTU has been completed.

[0071] In the disclosed embodiment, before entropy decoding is performed on the current CTU, it is ensured that the CTU above the current CTU has completed the CABAC entropy decoding task. That is, for the current CTU, after the CTU above it has completed the CABAC entropy decoding task, the CABAC entropy decoding task of the current CTU can be triggered.

[0072] (3) In a specific scanning or processing order, the triggering generation of the entropy decoding task of the current CTU depends on the completion of the entropy decoding task of the previous CTU of the current CTU.

[0073] In the disclosed embodiment, the CTU may be processed in a specific scanning or processing order, such as zigzag scanning, line-by-line scanning, etc. In this case, before entropy decoding is performed on the current CTU, it is ensured that a CTU before the current CTU has completed the CABAC entropy decoding task. That is, for the current CTU, after the previous CTU completes the CABAC entropy decoding task, the generation of the CABAC entropy decoding task of the current CTU may be triggered.

[0074] (4) The triggering generation of the CABAC entropy decoding task of the current CTU depends on the initialization of the slice where the current CTU is located.

[0075] In the disclosed embodiment, for each CTU in a slice, the CABAC entropy decoding task of generating the CTU in the slice can be triggered only after the slice is initialized. Specifically, the CABAC context variable of the slice is initialized, and through the slice-level initialization, the latest context information can be used when decoding the CTU in the slice, thereby improving the decoding efficiency.

[0076] In the video processing method provided by the embodiment of the present disclosure, the triggering generation rule of the entropy decoding task is pre-set, for example, after the current CTU completes the CABAC entropy decoding task, the CABAC entropy decoding task of the adjacent right side, bottom side or the next CTU in a specific scanning order can be triggered. In this way, the CABAC decoding tasks can be effectively managed and scheduled to ensure that each CTU triggers the generation of CABAC entropy decoding only after the CABAC entropy decoding task of the CTU it depends on is completed.

[0077] In some embodiments of the present disclosure, based on a preset trigger generation rule for entropy decoding tasks, an entropy decoding task of at least one CTU is triggered to be generated, and the generated entropy decoding task is placed in a task queue, including: triggering the generation of an entropy decoding task of a first CTU in at least one CTU, and placing the entropy decoding task of the first CTU in the task queue; the first CTU is the first CTU to be decoded in the slice; taking the first CTU as the current CTU, triggering the generation of one or more entropy decoding tasks of the second CTU based on the preset trigger generation rule for entropy decoding tasks, and placing the entropy decoding tasks of the one or more second CTUs in the task queue; taking the second CTU as the new current CTU, continuing to trigger the generation of entropy decoding tasks of CTUs in at least one CTU based on the preset trigger generation rule for entropy decoding tasks, and placing the generated entropy decoding tasks in the task queue, until the entropy decoding tasks of all CTUs in at least one CTU are triggered to be generated and placed in the task queue.

[0078] In the disclosed embodiment, a slice includes at least one CTU, and a first CTU to be decoded in the at least one CTU is taken as a first CTU. For example, a CTU located in the upper left corner of the slice is the first CTU of the slice.

[0079] In the embodiment of the present disclosure, the CABAC entropy decoding tasks of all CTUs in the slice may be generated as follows:

[0080] (1) Triggering generation of a CABAC entropy decoding task for the first CTU, and placing the CABAC entropy decoding task for the first CTU into a task queue.

[0081] (2) Taking the first CTU as the current CTU, triggering and generating CABAC entropy decoding tasks of one or more second CTUs based on a preset triggering and generating rule of the entropy decoding task, and placing the generated CABAC entropy decoding tasks into a task queue.

[0082] The one or more second CTUs refer to CTUs related to the first CTU. Exemplarily, the one or more second CTUs may include the following options.

[0083] (3) Determine whether to generate CABAC entropy decoding tasks for all CTUs in the slice.

[0084] In the disclosed embodiment, judging whether to generate CABAC entropy decoding tasks for all CTUs in a slice refers to whether there are other CABAC entropy decoding tasks for other CTUs in the slice that have not been generated.

[0085] (4) If the CABAC entropy decoding tasks of all CTUs in the slice are not generated, for each second CTU, the second CTU is used as a new current CTU, and based on the preset entropy decoding task trigger generation rule, the entropy decoding task of the CTU related to the second CTU is triggered, and the generated entropy decoding task is placed in the task queue.

[0086] For each second CTU, the number of CTUs related to the second CTU is one or more. Exemplarily, the CTUs related to the second CTU may include one or more of the following options: a CTU to the right of the second CTU, a CTU below the second CTU, and a CTU after the second CTU determined based on a scanning order.

[0087] (5) Execute the above step (3) to determine whether CABAC entropy decoding tasks for all CTUs in the slice are generated. If CABAC entropy decoding tasks for all CTUs in the slice are not generated, determine the CTU related to the second CTU as the new current CTU, and continue to generate CABAC entropy decoding tasks for other CTUs in the slice (i.e., CTUs for which CABAC entropy decoding tasks have not been generated) based on the preset entropy decoding task trigger generation rule and put them into the task queue, until CABAC entropy decoding tasks for all CTUs in the slice have been generated and put into the task queue.

[0088] Figure 4 FIG. 4 is a flowchart of a CABAC entropy decoding task for triggering generation of each CTU in an embodiment of the present disclosure. Figure 4 , you can follow the steps below to trigger the CABAC entropy decoding task for each CTU.

[0089] Step S401: after parsing the sliced ​​data stream and the header information of the sliced ​​data stream, confirm that the first CTU to be decoded in the slice is the current CTU.

[0090] Step S402: triggering the generation of a CABAC entropy decoding task for the current CTU, and placing the CABAC entropy decoding task for the current CTU into a task queue.

[0091] Step S403, determining whether CABAC entropy decoding tasks for all CTUs in the slice have been generated.

[0092] Step S404: If not, that is, the CABAC entropy decoding tasks of all CTUs in the slice have not been generated, after completing the CABAC entropy decoding task of the current CTU, based on the preset trigger generation rule of the entropy decoding task, determine that the CTU related to the current CTU is the new current CTU, and return to step S402.

[0093] Among them, the CTU related to the current CTU can be determined based on the trigger generation rule of the preset entropy decoding task, and can include one or more of the following options: the CTU to the right of the current CTU, the CTU below the current CTU, and a CTU after the current CTU determined based on the scanning order.

[0094] Step S405: If yes, confirm that the CABAC entropy decoding tasks of all CTUs in the slice have been generated.

[0095] In the video processing method provided by the embodiment of the present disclosure, based on the preset entropy decoding task trigger generation rule, the CABAC entropy decoding task of each CTU in the slice is triggered and generated in sequence and put into the task queue, which can ensure that the CABAC entropy decoding tasks are performed in sequence, and ensure that the CABAC entropy decoding task of each CTU is not processed until its predecessor task (i.e., the CABAC entropy decoding task of the CTU on which the CTU depends) is completed, thereby avoiding the delay caused by dependence on undecoded data, improving the parallel processing capability of the decoder, and accelerating the decoding speed.

[0096] In some embodiments of the present disclosure, the video processing method further includes: after completing the entropy decoding task of each CTU in the slice, based on the task triggering rule, triggering the generation of other decoding operation tasks of the CTU and placing them in the task queue.

[0097] Among them, the decoding process includes different operations such as entropy decoding, predictive decoding, inverse transform and inverse quantization, deblocking filtering, sample adaptive compensation and adaptive loop filtering. Other decoding operations refer to other operations in the decoding process except entropy decoding. In the disclosed embodiment, the operations such as entropy decoding, predictive decoding, inverse transform and inverse quantization, deblocking filtering, sample adaptive compensation, and adaptive loop filtering in the decoding process are abstracted into tasks one by one based on CTU. For each CTU in the slice, after completing the CABAC entropy decoding task of the CTU, other decoding operation tasks of the CTU can be generated based on the task triggering rules, and the generated tasks are placed in the task queue. Among them, the task triggering rules are generated based on information such as the dependency between CTUs, triggering logic rules, and the execution order of different operations in the decoding process.

[0098] It should be noted that the decoding process includes not only entropy decoding, predictive decoding, inverse transformation and inverse quantization, deblocking filtering, sample adaptive compensation and adaptive loop filtering, but also other operations and the execution order of different operations can be set according to actual needs. The embodiments of the present disclosure do not limit this.

[0099] In the video processing method provided by the embodiment of the present disclosure, for each CTU in the slice, after completing the CABAC entropy decoding task of the CTU, other decoding operation tasks of the CTU are generated based on the task triggering rules and placed in the task queue, wherein the task triggering rules are set based on information such as the dependency relationship between CTUs, triggering logic rules, and the execution order of different operations in the decoding process, thereby ensuring the orderliness of task triggering generation and avoiding conflicts and dependency problems between tasks. In addition, the introduction of the task queue makes the decoding process clear and controllable, and each task can be triggered and generated based on the task triggering rules, thereby realizing efficient management and scheduling of the decoding process, thereby improving the decoding speed in a multi-threaded environment.

[0100] Figure 5 FIG. 2 shows a decoder architecture diagram that can implement the video processing method of the embodiment of the present disclosure. Figure 5 As shown, the decoder of the embodiment of the present disclosure includes a header information parser 510 and a triggered parallel decoder 520. The header information parser 510 is single-threaded, and the triggered parallel decoder 520 is multi-threaded.

[0101] like Figure 5As shown, the header information parser 510 sequentially reads the slice data stream of each Slice (i.e., slice) of each video frame in the coded data stream of the input video, and saves them in the code stream memory respectively. For each slice data stream, the header information of the slice data stream is parsed, and the Slice (i.e., slice) information is initialized. For each Slice (i.e., slice), after parsing the slice data stream and the header information of the slice data stream, the CABAC entropy decoding task of the first CTU of the Slice (i.e., slice) is triggered, and the CABAC entropy decoding task is placed in the task queue.

[0102] Among them, for each Slice (i.e., slice), the CABAC entropy decoding task of the first CTU of the Slice (i.e., slice) is triggered and generated by the header information parser 510 and placed in the task queue. The CABAC entropy decoding tasks of other CTUs of the Slice (i.e., slice) are triggered and generated by the triggered parallel decoder 520 based on the preset trigger generation rule of the entropy decoding task and placed in the task queue. In the above embodiment, the preset trigger generation rule of the entropy decoding task and the process of triggering the generation of the CABAC entropy decoding task of the CTU in the slice based on the preset trigger generation rule of the entropy decoding task have been explained, and will not be repeated here.

[0103] like Figure 5 As shown, each operation in the decoding process, such as CABAC entropy decoding, predictive decoding, inverse transform and inverse quantization, deblocking filtering, sample adaptive compensation, and adaptive loop filtering, is abstracted into different operation tasks. The triggered parallel decoder 520 generates different operation tasks of CTU based on the task triggering rules and puts them into the task queue.

[0104] from Figure 5 It can be seen that the triggered parallel decoder 520 adopts a multi-threaded parallel method to select pending tasks from the task queue and process the selected pending tasks in parallel. Specifically, tasks corresponding to the number of idle threads are selected from the task queue and the selected tasks are processed in parallel, thereby realizing triggered parallel decoding.

[0105] In the disclosed embodiment, in order to enable the entropy decoding module to implement triggered parallel processing, the memory and data structure of the decoder are redesigned. Figure 6 FIG. 2 is a schematic diagram showing the memory and data structure of the decoder in the embodiment of the present disclosure. Figure 6 As shown, the hierarchical management design concept is adopted to uniformly manage the bitstream memory and entropy decoder status through different layers.

[0106] In some embodiments of the present disclosure, the video processing method further includes: storing the fragmented data streams in a code stream memory.

[0107] In the disclosed embodiment, one slice corresponds to one NALU, that is, one NALU contains a slice data stream of one slice. For each slice, after obtaining the slice data stream of the slice using the header information parser, the slice data stream is stored in the code stream memory, and the code stream memory is allocated for the NALU corresponding to the slice data stream.

[0108] In the disclosed embodiment, each NALU read by the header information parser will be allocated with a code stream memory for storing the fragmented data stream corresponding to the NALU, so that parallelism between NALUs can be achieved. Figure 6 As shown, at the NALU layer, NALU1 corresponds to the allocation of Slice1 code stream memory, NALU2 corresponds to the allocation of Slice2 code stream memory, and NALU3 corresponds to the allocation of Slice3 code stream memory.

[0109] In some embodiments of the present disclosure, the video processing method also includes: dividing the fragmented data stream into at least one sub-fragmented data stream; for each sub-fragmented data stream, allocating a memory pointer to the sub-fragmented data stream; the memory pointer is used to locate the position where the sub-fragmented data stream is stored in the code stream memory.

[0110] In the disclosed embodiment, for each sharded data stream, it is divided into at least one sub-sharded data stream, wherein the specific division method and the number of divisions are determined by the parallelism, so that flexible parallel processing can be achieved. Also, for each sub-sharded data stream, a memory pointer is allocated to the sub-sharded data stream, so that the position of the sub-sharded data stream in the code stream is located through the allocated memory pointer, so that the divided sub-sharded data stream does not occupy new memory and does not increase additional memory overhead.

[0111] Continue to refer to Figure 6 For the sliced ​​data stream in the Slice1 code stream memory (ie NALU1), it is divided into n sub-sliced ​​data streams, and the corresponding pointers substream0 to substream(n-1) are maintained respectively; for the sliced ​​data stream in the Slice2 code stream memory (ie NALU2), it is divided into n sub-sliced ​​data streams, and the corresponding pointers substream0 to substream(n-1) are maintained respectively; for the sliced ​​data stream in the Slice3 code stream memory (ie NALU3), it is divided into n sub-sliced ​​data streams, and the corresponding pointers substream0 to substream(n-1) are maintained respectively.

[0112] In the video processing method provided by the embodiment of the present disclosure, by storing the segmented data stream in the code stream memory and allocating a memory pointer to each sub-segmented data stream, multiple sub-segmented data streams can be processed in parallel, further improving the decoding speed; and the division of the sub-segmented data streams does not occupy new memory space, avoiding additional memory overhead and optimizing memory usage efficiency.

[0113] In some embodiments of the present disclosure, the video processing method further includes: updating a current decoding position index of the sub-segment data stream during decoding of the sub-segment data stream; the current decoding position index of the sub-segment data stream is used to indicate the position to which the sub-segment data stream is currently parsed.

[0114] In the embodiment of the present disclosure, for each sub-segment data stream, its current decoding position index is maintained, and the current decoding position index is used to record the position to which the sub-segment data stream has been parsed.

[0115] Continue to refer to Figure 6 ,At the Slice layer, the memory pointer and the current decoding position index of the sub-slice data stream are managed separately in units of the sub-slice data stream.

[0116] In the video processing method provided by the embodiment of the present disclosure, for each sub-segment data stream, the current decoding position index of the sub-segment data stream is maintained to facilitate tracking of the decoding progress.

[0117] In some embodiments of the present disclosure, the video processing method also includes: allocating multiple entropy decoders according to the number of multi-threads so that each entropy decoder corresponds to one thread; using a multi-threaded parallel method, selecting a sub-slice data stream to be processed from different sub-slice data streams, and using the allocated entropy decoder to perform entropy decoding on the sub-slice data stream to be processed.

[0118] In the disclosed embodiment, entropy decoders equal to the number of threads are allocated, each entropy decoder corresponds to one thread, and selection and entropy decoding are dynamically performed between different sub-slice data streams in a pointer manner, thereby realizing triggered parallel processing of CABAC entropy decoding tasks.

[0119] Continue to refer to Figure 6 In the CabacReader layer, CabacReader0 to CabacReader3 are assigned to threads Thread0 to Thread3 respectively. Through multiple threads, the sub-slice data stream to be processed is selected between different sub-slice data streams in a pointer manner, and the entropy decoder assigned to the thread is used to perform entropy decoding on the sub-slice data stream to be processed, which can realize the triggered parallel processing of CABAC entropy decoding tasks.

[0120] In the video processing method provided by the embodiment of the present disclosure, by allocating entropy decoders equal to the number of threads, each entropy decoder corresponds to one thread, which can realize multi-threaded parallel processing and further improve the decoding speed.

[0121] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0122] Figure 7 FIG. 1 is a block diagram of a video processing device according to an embodiment of the present disclosure. Figure 7 As shown, the device 700 includes a data stream acquisition module 710 , a data stream parsing module 720 and a processing module 730 .

[0123] The data stream acquisition module 710 is configured to: acquire the encoded data stream of the video; the video includes multiple video frames, and the video frame includes multiple slices. The data stream parsing module 720 is configured to: for each slice, parse the encoded data stream of the video to obtain the sliced ​​data stream and the header information of the sliced ​​data stream; trigger the generation of the entropy decoding task of the slice, and put the entropy decoding task of the slice into the task queue; wherein the entropy decoding task is an entropy decoding task performed using CABAC. The processing module 730 is configured to: select the task to be processed from the task queue in a multi-threaded parallel manner; perform parallel processing on the task to be processed according to the sliced ​​data stream corresponding to the task to be processed and the header information of the sliced ​​data stream corresponding to the task to be processed; the sliced ​​data stream corresponding to the task to be processed is the sliced ​​data stream corresponding to the slice to which the task to be processed belongs.

[0124] In some embodiments of the present disclosure, a slice includes at least one CTU. The data stream parsing module 720 is further configured to: trigger the generation of at least one CTU entropy decoding task based on a preset entropy decoding task trigger generation rule, and put the generated entropy decoding task into a task queue.

[0125] In some embodiments of the present disclosure, the data stream parsing module 720 is further configured to: trigger the generation of an entropy decoding task of a first CTU in at least one CTU, and put the entropy decoding task of the first CTU into a task queue; the first CTU is the first CTU to be decoded in the slice. Figure 7As shown, the device 700 also includes a task trigger generation module 740, which is configured to: use the first CTU as the current CTU, trigger the generation of one or more entropy decoding tasks of the second CTU based on the preset entropy decoding task trigger generation rule, and put the one or more second CTU entropy decoding tasks into the task queue; use the second CTU as the new current CTU, continue to trigger the generation of entropy decoding tasks of CTUs in at least one CTU based on the preset entropy decoding task trigger generation rule, and put the generated entropy decoding tasks into the task queue until the entropy decoding tasks of all CTUs in at least one CTU are triggered and placed in the task queue.

[0126] In some embodiments of the present disclosure, the preset trigger generation rule of the entropy decoding task includes at least one of the following options: in response to completing the entropy decoding task of the current CTU, triggering the generation of the entropy decoding task of the CTU to the right of the current CTU; in response to completing the entropy decoding task of the current CTU, triggering the generation of the entropy decoding task of the CTU below the current CTU; in response to completing the entropy decoding task of the current CTU, triggering the generation of the entropy decoding task of the next CTU of the current CTU; the next CTU of the current CTU is determined according to a specific scanning or processing order.

[0127] In some embodiments of the present disclosure, the data stream parsing module 720 is further configured to: store the fragmented data streams into the code stream memory.

[0128] In some embodiments of the present disclosure, the data stream parsing module 720 is further configured to: divide the fragmented data stream into at least one sub-fragmented data stream; for each sub-fragmented data stream, allocate a memory pointer to the sub-fragmented data stream; the memory pointer is used to locate the position where the sub-fragmented data stream is stored in the code stream memory.

[0129] In some embodiments of the present disclosure, the processing module 730 is further configured to: update the current decoding position index of the sub-segment data stream during decoding of the sub-segment data stream; the current decoding position index of the sub-segment data stream is used to indicate the position to which the sub-segment data stream is currently parsed.

[0130] In some embodiments of the present disclosure, the processing module 730 is further configured to: allocate multiple entropy decoders according to the number of multi-threads so that each entropy decoder corresponds to one thread; adopt a multi-threaded parallel approach to select a sub-slice data stream to be processed from different sub-slice data streams, and use the allocated entropy decoder to perform entropy decoding on the sub-slice data stream to be processed.

[0131] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0132] Figure 8 FIG. 1 is a schematic diagram showing the structure of an electronic device according to an embodiment of the present disclosure. It should be noted that: Figure 8 The electronic device 800 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0133] like Figure 8 As shown, the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 may include but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).

[0134] The storage unit stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 810 can perform the following steps: Figure 3 Follow the steps shown in .

[0135] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0136] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0137] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0138] The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 840. As shown, the network adapter 840 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0139] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0140] The program product for implementing the above method according to the embodiment of the present invention may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0141] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0142] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0143] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0144] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0145] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0146] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0147] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0148] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

[0149] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video processing method, characterized in that: The method comprises: Acquire a coded data stream of a video; the video includes a plurality of video frames, and the video frame includes a plurality of slices; For each of the slices, the encoded data stream of the video is parsed to obtain the slice data stream of the slice and the header information of the slice data stream; triggering the generation of the entropy decoding task of the slice, and putting the entropy decoding task of the slice into the task queue; the entropy decoding task is an entropy decoding task performed by using a context-based adaptive binary arithmetic coding algorithm; A multi-threaded parallel approach is adopted to select tasks to be processed from the task queue; the tasks to be processed are processed in parallel according to the sharded data stream corresponding to the tasks to be processed and the header information of the sharded data stream corresponding to the tasks to be processed; the sharded data stream corresponding to the tasks to be processed is the sharded data stream corresponding to the shard to which the tasks to be processed belong.

2. The method according to claim 1, characterized in that The slice comprises at least one coding tree unit; The triggering of generating the entropy decoding task of the slice and placing the entropy decoding task of the slice into a task queue includes: Based on a preset trigger generation rule of the entropy decoding task, the entropy decoding task of the at least one coding tree unit is triggered to be generated, and the generated entropy decoding task is placed in the task queue.

3. The method according to claim 2, characterized in that The triggering generation rule based on the preset entropy decoding task triggers the generation of the entropy decoding task of the at least one coding tree unit, and puts the generated entropy decoding task into the task queue, including: triggering generation of an entropy decoding task of a first coding tree unit in the at least one coding tree unit, and placing the entropy decoding task of the first coding tree unit into the task queue; the first coding tree unit is the first coding tree unit to be decoded in the slice; Taking the first coding tree unit as the current coding tree unit, triggering the generation of one or more entropy decoding tasks of the second coding tree units based on the preset trigger generation rule of the entropy decoding task, and placing the one or more entropy decoding tasks of the second coding tree units into the task queue; The second coding tree unit is used as a new current coding tree unit, and the entropy decoding tasks of the coding tree units in the at least one coding tree unit are continuously triggered based on the preset trigger generation rule of the entropy decoding task, and the generated entropy decoding tasks are placed in the task queue until the entropy decoding tasks of all the coding tree units in the at least one coding tree unit are triggered to be generated and placed in the task queue.

4. The method according to claim 3, characterized in that The trigger generation rule of the preset entropy decoding task includes at least one of the following options: In response to completing the entropy decoding task of the current coding tree unit, triggering the generation of an entropy decoding task of a right coding tree unit of the current coding tree unit; In response to completing the entropy decoding task of the current coding tree unit, triggering the generation of the entropy decoding task of the lower coding tree unit of the current coding tree unit; In response to completing the entropy decoding task of the current coding tree unit, triggering the generation of the entropy decoding task of the next coding tree unit of the current coding tree unit; the next coding tree unit of the current coding tree unit is determined according to a specific scanning or processing order.

5. The method according to claim 1, characterized in that The method further comprises: The fragmented data stream of the fragment is stored in the code stream memory.

6. The method according to claim 5, characterized in that The method further comprises: Dividing the sharded data stream of the shard into at least one sub-sharded data stream; For each of the sub-slice data streams, a memory pointer is allocated for the sub-slice data stream; the memory pointer is used to locate the position in the code stream memory where the sub-slice data stream is stored.

7. The method according to claim 6, characterized in that The method further comprises: In the process of decoding the sub-slice data stream, the current decoding position index of the sub-slice data stream is updated; the current decoding position index of the sub-slice data stream is used to indicate the position to which the sub-slice data stream is currently parsed.

8. The method according to claim 6, characterized in that The method further comprises: Allocate a plurality of entropy decoders according to the number of multithreads so that each of the entropy decoders corresponds to a thread; In a multi-threaded parallel manner, a sub-slice data stream to be processed is selected from different sub-slice data streams, and the sub-slice data stream to be processed is entropy decoded by using the allocated entropy decoder.

9. A video processing device, characterized in that: The device comprises: A data stream acquisition module is configured to acquire a coded data stream of a video; the video includes a plurality of video frames, and the video frame includes a plurality of slices; The data stream parsing module is configured to parse the encoded data stream of the video for each of the slices to obtain the slice data stream of the slice and the header information of the slice data stream; trigger the generation of the entropy decoding task of the slice, and put the entropy decoding task of the slice into the task queue; the entropy decoding task is an entropy decoding task performed by using a context-based adaptive binary arithmetic coding algorithm; The processing module is configured to select tasks to be processed from the task queue in a multi-threaded parallel manner; perform parallel processing on the tasks to be processed according to the shard data stream corresponding to the tasks to be processed and the header information of the shard data stream corresponding to the tasks to be processed; the shard data stream corresponding to the tasks to be processed is the shard data stream corresponding to the shard to which the tasks to be processed belong.

10. An electronic device, characterized in that: include: processor; A memory for storing the processor executable instructions; wherein the processor is configured to execute the executable instructions to implement the video processing method according to any one of claims 1 to 8. 11 . A computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the video processing method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the video processing method according to any one of claims 1 to 8 is implemented.

Images