Entropy context processing method, system on chip and electronic device

Abstract

The present application relates to the field of data processing technology, and in particular to an entropy context processing method, a system on chip, and an electronic device, which can reduce the time consumption of the entropy context saving and recovery process and avoid occupying the system bandwidth. The method is specifically applied to a system on chip including multiple cores and at least one storage unit, including: detecting that the input bit status of multiple cores does not meet the preset conditions; the multiple cores respectively back up the cached entropy context to at least one storage unit; detecting that the input bit status of multiple cores meets the preset conditions; the multiple cores respectively obtain the corresponding entropy context from at least one storage unit, and re-cache the corresponding entropy context; the multiple cores perform entropy decoding on different macroblock rows in the input bit stream in parallel, and update the cached entropy context.

Description

Technical Field

[0001] The present application relates to the field of video decoding technology, and in particular to an entropy context processing method, a system on chip, and an electronic device. Background Art

[0002] For high-definition video, the current mainstream video compression standards include High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC) and Audio Video Coding Standard (AVS), and these standards perform video decoding through a video decoder (VDEC). Usually, a video decoder adopts a pipeline design, with macroblocks (MB in AVC, i.e. MacroBlock; CTB in HEVC, i.e. Coding Tree Blocks) as pipeline units. Specifically, taking HEVC as an example, its main pipeline levels are divided into 4 levels, and the functions of each level are described as follows: the first level is entropy decoding (Entropy Decode); the second level is inverse quantization (IQT) and inverse DCT transform (IDCT); the third level is frame prediction (IPred) and image reconstruction (ReC); the fourth level is deblocking filtering (Dblock) and adaptive offset (SAO).

[0003] The video decoder decodes the bitstream through an entropy engine, which parses the bitstream based on different entropy algorithms. Different video compression standards use different entropy algorithms, for example, HEVC uses context-based adaptive binary arithmetic coding (CABAC), AVS uses context-based access control (CBAC), and AV1 uses a multi-symbol scheme. Regardless of the entropy algorithm used by the video standard, the entropy engine inside the video decoder will always parse the bitstream in the input buffer, and the capacity of the input buffer can vary according to system requirements. In traditional video decoding systems, the bits in the bitstream are loaded block by block (i.e., macroblock). Then, only a very large input buffer can save enough bitstream for the entropy engine to decode normally, otherwise there will be a "bit underflow" problem. This means that the current entropy engine consumes all available bits in the input buffer, and no new available bits have been stored in the input buffer. This bit underflow is usually caused by AXI (Advanced eXtensible Interface) delay of the system bus, Internet delay, or some system-level problems (such as a large amount of data occupying the system bus, etc.). The bit underflow problem in entropy decoding may cause decoding anomalies and failures.

[0004] In order to solve the above problems, a common method is to save the current decoding information (e.g., neighbor information (i.e., adjacent macroblock information, such as macroblock identifiers) and entropy context) before entropy decoding begins. When the decoding process encounters "bit underflow", the video decoder will exit the decoding process, wait for subsequent bit loading to complete, and then restore the decoding information to return to the decoding process. Currently, these entropy decoding processes are usually handled by the firmware in the video decoder. The firmware performs the entropy context saving and recovery process between the internal static random access memory (SRAM) of the video decoder and the external double data rate synchronous dynamic random access memory (DDR), resulting in the most time-consuming part of the entropy decoding process being the saving and recovery of the entropy context. Moreover, when the video decoder uses multi-core decoding, the entropy context saving and recovery process is not only time-consuming but also leads to high system bandwidth. Summary of the invention

[0005] The present application provides an entropy context processing method, a system on chip and an electronic device, which can reduce the time consumption of the entropy context saving and recovery process and avoid occupying the system bandwidth.

[0006] The first aspect of the present application provides an entropy context processing method, which is applied to a system on chip including multiple cores and at least one storage unit (such as at least one milestone Ram below), the method comprising: detecting that the input bit status of the multiple cores does not meet the preset condition; the multiple cores respectively back up the cached entropy context to the at least one storage unit; detecting that the input bit status of the multiple cores meets the preset condition; the multiple cores respectively obtain the corresponding entropy context from the at least one storage unit, and re-cache the corresponding entropy context; the multiple cores perform entropy decoding on different macroblock rows in the input bit stream in parallel, and update the cached entropy context. In this way, since each core does not need to save and restore the entropy context in an external DDR, the time consumption of each core for the entropy context saving and recovery process is reduced, and in a multi-core decoding scenario, the system bandwidth generated by the core saving and restoring the entropy context in the external DDR can also be avoided.

[0007] In a possible implementation of the first aspect, the preset condition is: at least one of the multiple cores does not input a bit stream within a preset time period, or the number of bits of the bit stream input to the multiple cores is less than a preset number. It can be understood that the preset condition is used to indicate a bit underflow problem in an input buffer area connected to the multiple cores.

[0008] In a possible implementation of the first aspect above, the number of the at least one storage unit is the same as the number of the multiple cores, and one core uniquely corresponds to one storage unit; each core in the multiple cores stores or obtains the entropy context in a storage unit that uniquely corresponds to the at least one storage unit.

[0009] In a possible implementation of the first aspect above, the on-chip system also includes an arbitrator connected to the at least one storage unit, and the number of the at least one storage unit is less than the number of the multiple cores; the method also includes: the multiple cores respectively send a storage request message to the arbitrator; the arbitrator assigns a storage address to the multiple cores respectively based on the multiple storage request messages; the arbitrator returns a response message allowing storage to the multiple cores respectively; wherein the entropy context cached by each of the cores is stored in the at least one storage unit according to the storage address assigned by the arbitrator.

[0010] In a possible implementation of the first aspect above, one of the storage request messages includes at least one of the following: identification information of the kernel, an entropy context of the corresponding kernel cache, and first indication information indicating a request to store the entropy context; and one of the storage permission response messages includes at least one of the following: identification information of the corresponding kernel, an identification of the storage address allocated by the corresponding kernel, and second indication information indicating that the corresponding kernel is allowed to store the corresponding entropy context in the at least one storage unit.

[0011] In a possible implementation of the first aspect above, each of the cores includes a firmware, an entropy engine and a context cache unit (i.e., context Ram hereinafter); wherein each of the cores performs entropy decoding on a bit stream and generates an entropy context through the entropy engine; each of the cores caches the entropy context through the context cache unit; each of the cores initiates a storage request to the arbitrator through the firmware, and backs up the entropy context cached in the corresponding context cache unit to the at least one storage unit, and restores the entropy context obtained from the at least one storage unit and stores it in the corresponding context cache unit.

[0012] In a possible implementation of the first aspect above, the multiple cores entropy decode the bitstream according to macroblock rows in the bitstream, and one core entropy decodes the bitstream according to macroblocks in the macroblock row, wherein one macroblock row corresponds to one core, and an entropy context cached by one core corresponds to one macroblock.

[0013] In a possible implementation of the first aspect above, the system on chip is connected to an external target storage unit; the method also includes: each of the multiple cores sends a kernel processing information to the target storage unit, wherein one kernel processing information includes at least one of the following: information of the corresponding kernel, information of the bit stream, macroblock or macroblock row input in the corresponding kernel.

[0014] A second aspect of the present application provides a system on chip, comprising multiple cores and at least one storage unit; the multiple cores are used to detect that the input bit status of the multiple cores does not meet the preset conditions; the cached entropy context is backed up to the at least one storage unit respectively; the at least one storage unit is used to store the entropy context from the multiple cores; the multiple cores are also used to detect that the input bit status of the multiple cores meets the preset conditions; the corresponding entropy context is obtained from the at least one storage unit respectively, and the corresponding entropy context is re-cached; different macroblock rows in the input bit stream are entropy decoded in parallel, and the cached entropy context is updated.

[0015] In a possible implementation manner of the second aspect above, the preset condition is: at least one of the multiple cores has not input a bitstream within a preset time period, or the number of bits of the bitstream input into the multiple cores is less than a preset number.

[0016] In a possible implementation of the second aspect above, the number of the at least one storage unit is the same as the number of the multiple cores, and one core uniquely corresponds to one storage unit; each core in the multiple cores stores or obtains the entropy context in a storage unit that uniquely corresponds to the at least one storage unit.

[0017] In a possible implementation of the second aspect above, the on-chip system also includes an arbitrator connected to the at least one storage unit, and the number of the at least one storage unit is less than the number of the multiple cores; the multiple cores are further used to send a storage request message to the arbitrator respectively; the arbitrator is used to allocate a storage address to the multiple cores respectively based on the multiple storage request messages; and return a storage permission response message to the multiple cores respectively; wherein the entropy context cached by each of the cores is stored in the at least one storage unit according to the storage address allocated by the arbitrator.

[0018] In a possible implementation of the second aspect above, one of the storage request messages includes at least one of the following: identification information of the kernel, entropy context of the corresponding kernel cache, and first indication information indicating a request to store the entropy context; and one of the storage permission response messages includes at least one of the following: identification information of the corresponding kernel, identification of the storage address allocated by the corresponding kernel, and second indication information indicating that the corresponding kernel is allowed to store the corresponding entropy context in the at least one storage unit.

[0019] In a possible implementation of the second aspect above, each of the cores includes a firmware, an entropy engine and a context cache unit; wherein each of the cores performs entropy decoding on a bit stream and generates an entropy context through the entropy engine; each of the cores caches the entropy context through the context cache unit; each of the cores initiates a storage request to the arbitrator through the firmware, and backs up the entropy context cached in the corresponding context cache unit to the at least one storage unit, and restores the entropy context obtained from the at least one storage unit and stores it in the corresponding context cache unit.

[0020] In a possible implementation of the second aspect above, the multiple cores entropy decode the bitstream according to macroblock rows in the bitstream, and one core entropy decodes the bitstream according to macroblocks in the macroblock row, wherein one macroblock row corresponds to one core, and an entropy context cached by one core corresponds to one macroblock.

[0021] In a possible implementation of the second aspect above, the system on chip is connected to a target storage unit outside of it; each of the multiple cores is further used to send a kernel processing information to the target storage unit, wherein one kernel processing information includes at least one of the following: information of the corresponding kernel, information of the bit stream, macroblock or macroblock row input into the corresponding kernel.

[0022] The third aspect of the present application provides an electronic device, which includes the system on chip as described in the second aspect above, and a target storage unit outside the system on chip; the target storage unit is used to store kernel processing information sent by the multiple kernels, wherein one of the kernel processing information includes at least one of the following: information of the corresponding kernel, information of the bit stream, macroblock or macroblock row input into the corresponding kernel. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 According to some embodiments of the present application, a schematic diagram of a scenario of entropy decoding application is shown;

[0024] Figure 2A According to some embodiments of the present application, a structural schematic diagram of an electronic device is shown;

[0025] Figure 2B According to some embodiments of the present application, a schematic diagram of the architecture of an entropy decoding module in an electronic device is shown;

[0026] Figure 3 According to some embodiments of the present application, a schematic diagram of the architecture of an entropy decoding module in an electronic device is shown;

[0027] Figure 4 According to some embodiments of the present application, a flow chart of a method for entropy context processing is shown;

[0028] Figure 5 According to some embodiments of the present application, a schematic diagram of the architecture of an entropy decoding module in an electronic device is shown;

[0029] Figure 6 According to some embodiments of the present application, a flow chart of a method for entropy context processing is shown;

[0030] Figure 7A block diagram of an electronic device is shown according to an embodiment of the present application;

[0031] Figure 8 A block diagram of a system on chip is shown according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] Embodiments of the present application include but are not limited to an entropy context processing method, a readable medium and an electronic device thereof.

[0033] The embodiment of the present application provides a method for processing entropy context, and an on-chip storage unit is set in the video decoder to store the entropy context generated when bit underflow occurs during entropy decoding. Specifically, the video decoder performs entropy decoding on the bit stream in the input buffer through the entropy engine. When there are not enough bits in the input buffer and bit underflow occurs, the current entropy context is stored in the on-chip storage unit, and when the bit underflow problem is eliminated, the last stored entropy context is taken out from the on-chip storage unit to restore the entropy decoding process. In addition, an on-chip arbiter connected to the on-chip memory is also provided in the video decoder. In a multi-core decoding scenario, the entropy contexts of multiple cores can be stored in at least one on-chip memory respectively through the on-chip arbiter, and then the entropy contexts corresponding to each core are restored by the arbiter to continue entropy decoding. Then, since the firmware does not need to save and restore the entropy context in the external DDR, the time consumption of the firmware for the entropy context saving and recovery process is reduced, and in the multi-core decoding scenario, the system bandwidth generated by the firmware saving and restoring the entropy context in the external DDR can also be avoided.

[0034] It can be understood that the video decoder in the embodiments of the present application can be set in a system on chip (SoC) in an electronic device, and the on-chip storage unit described in each embodiment of the present application refers to a memory in the system on chip, such as a static random access memory (SRAM) inside the system on chip. Specifically, as an example, the on-chip storage unit of the system on chip where the video decoder is located is referred to as a milestone Ram, and the on-chip storage unit is mainly used to store the entropy context generated when a bit underflow occurs during the decoding process.

[0035] In some embodiments, the video decoder may be a programmable system-on-chip, in which a single chip completes the main logic functions of the entire system and may have software and hardware programmable functions in the system. As an example, the system-on-chip may include one or more embedded processor cores, and have a small-capacity on-chip high-speed RAM resource and sufficient on-chip programmable logic resources.

[0036] In addition, it can be understood that the on-chip arbiter described in the embodiment of the present application can be a functional module in the system on chip where the video decoder is located, which can be implemented by software and / or hardware. As an example, the above-mentioned on-chip arbiter can be implemented by a software algorithm, mainly when two or more modules need to occupy the same resource, to determine which module to occupy the resource. Specifically, in the following of the embodiment of the application, the above-mentioned on-chip arbiter is referred to as an arbiter, which is used to determine in which storage addresses in the external storage area the entropy contexts in multiple cores are stored.

[0037] The entropy context processing method provided in this application is applied to the entropy decoding process, and is specifically applied to scenarios where audio and / or video data need to be encoded and decoded. For example, the solution of this application can be applied to panoramic sound movies, ultra-high-definition television, Internet broadband audio and video services, digital audio and video broadcasting wireless broadband multimedia communications, virtual reality and augmented reality, and video surveillance. The following is an example of the decoding process of the video.

[0038] It can be understood that there are many scenarios where video encoding is required. In addition to the common video shooting and data compression storage through smartphones, video encoding is also required in scenarios such as video conferencing, digital TV, and storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0039] In some embodiments, the video compression standard involved in the present application may be High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC) and Audio Video Coding Standard (AVS), etc., but not limited thereto. Correspondingly, the entropy decoding provided in the present application uses different entropy algorithms in different video compression standards, for example, HEVC uses context-based adaptive binary arithmetic coding (CABAC), AVS uses context-based access control protocol (CBAC), AV1 uses a multi-symbol scheme, etc. It can be understood that the entropy algorithm applied by the entropy context processing method of the embodiment of the present application can be any one of the above examples, and the comparison is not limited. In the following embodiments, the entropy context processing method provided in the present application is mainly described by taking the entropy decoding based on CBAC in AVS as an example.

[0040] Figure 1 According to an embodiment of the present application, a video encoding and decoding scene graph is shown. Figure 1 The scenario includes connecting the electronic device 100 and the electronic device 200 via one or more networks.

[0041] In some embodiments of the present application, the electronic device 200 may encode the video based on AVS to obtain compressed video data; the electronic device 200 may send the compressed video data to the electronic device 100; the electronic device 100 may decode the compressed video data to obtain decompressed video data.

[0042] In some embodiments, the electronic device 100 or the electronic device 200 may include but is not limited to smart phones, vehicle-mounted devices, personal computers, artificial intelligence devices, tablet computers, personal digital assistants, smart wearable devices (such as smart watches or bracelets, smart glasses), smart TVs (or called smart big screens, smart screens, or large-screen TVs, etc.), virtual reality / mixed reality / augmented display devices, servers, etc.

[0043] The electronic device 200 performs video encoding on the video to obtain compressed video data. The compressed video data may be a code stream, that is, a multi-bit binary (0 or 1) code. For example, a frame of image may be encoded into a binary number of several hundred bytes.

[0044] For example, in AVS encoding, different levels of information of a frame are encoded separately, and the levels of the encoded information are: image, stripe, macroblock, block. For example, AVS encoding can be processed with macroblocks as the basic unit, and the macroblock size can be 64x64, 8x8, or 16x16. It can be understood that according to actual application needs, this application does not specifically limit the size of macroblocks in AVS encoding processing.

[0045] After dividing a frame of image into macroblocks, the electronic device 200 can extract the information of the macroblocks through intra-frame prediction, inter-frame prediction, motion estimation, motion compensation and other processes, and represent the information of the macroblocks with the value of syntax elements, where the syntax elements include: motion vector difference (MVD), motion mode (intra-frame prediction or inter-frame prediction), macroblock coding template (cbp) block prediction residual coefficient, image unit mode, intra-frame prediction mode. Then, the electronic device 200 can entropy encode the value of the syntax element to obtain the final compressed data, that is, the bitstream.

[0046] In some embodiments, entropy coding needs to go through three steps: binary grammar element binarization, context model selection and update, and binary arithmetic coding calculation. Correspondingly, entropy decoding, as the inverse process of entropy coding, mainly includes steps such as binary arithmetic decoding, context model and update, and binary grammar element to be encoded.

[0047] Specifically, the binarization of syntax elements mainly maps non-binary syntax elements into corresponding binary strings. For example, the binarization of syntax elements can convert the value of the syntax element into a binary signal string represented by only 0 and 1. For example, when the value of the motion vector is 3, the binary symbol string obtained by the binarization conversion process is "1001", where 0 or 1 in the binary symbol string "1001" is called the binary value of the syntax element.

[0048] In some embodiments, after the syntax elements are binarized, it is crucial to select a suitable context model for the syntax elements for entropy coding. For example, in the AVS standard, 36 syntax elements can be included, and a total of 269 context models are defined. In the actual coding process, the parameters of each syntax element are identified by a unique context model, and each context model is independent of each other, but the parameters of the syntax elements in the same context model are correlated.

[0049] In some embodiments, the arithmetic coding of entropy coding encodes the binary symbol string of each syntax element through the parameters of the context model to generate a code stream. The arithmetic coding of entropy coding is mainly composed of conventional coding and bypass coding. Conventional coding is encoded according to the input binarized bin (binary, indicating a binary file) and the parameters in the context model. After encoding, the binarized value is fed back to the context model to complete the update of the context model. The parameters of the context model may include: Most Probable Signal (MPS) and Least Probable Signal (LPS), etc. For example, the binary symbol string of the motion vector is "1001", and the code stream generated by arithmetic coding thereof can be "01".

[0050] It should be noted that the entropy context provided in the embodiment of the present application may be the above-mentioned context model.

[0051] It can be understood that entropy coding is the last step of video coding, which is directly related to the generation of bitstream. It is mainly used to remove statistical redundancy in video signals and is a lossless compression coding method. For example, AVS entropy coding adopts CBAC coding. CBAC provides an estimation model for the conditional probability of the current syntax element, selects different context models according to the current encoded content, and obtains the conditional probability of the current syntax element and related content through this context model. Finally, this probability is used to reduce the redundancy of video coding and achieve the purpose of compression.

[0052] In some embodiments, after the electronic device 100 receives the compressed video data (i.e., bit stream), it can perform AVS decoding on the compressed video data to obtain a video image. It can be understood that AVS decoding is the inverse process of AVS encoding. That is, in the process of AVS decoding, it is necessary to first perform entropy decoding on the received bit stream to obtain the value of the syntax element. Furthermore, if a bit underflow problem occurs during the entropy decoding process, it is necessary to back up and save the current decoding information including the entropy context and neighbors, etc., until the bit underflow problem is eliminated and then restore the entropy context information to return to the entropy decoding process.

[0053] Generally, a video compression standard uses a hardware accelerator as a video decoder (Video Decoder, VDEC) to perform video decoding. Among them, the video decoder involved in this application can adopt a single-core solution or a multi-core solution.

[0054] Single-core hardware decoders mostly adopt a pipeline design, with macroblocks (for example, MB in AVC, i.e., MacroBlock; CTB in HEVC, i.e., Coding Tree Blocks) as pipeline units, and entropy decoding is usually the first stage in the pipeline.

[0055] The multi-core design of the video decoder can achieve efficient parallel decoding. Specifically, the multi-core parallel decoding solution is generally developed around the Frame, Slice, Tile, MB / CTB line and other levels. For example, during HEVC encoding, entropy decoding is directly initialized at the head of each CTB line without waiting until the entropy decoding of the previous line is completely completed, thereby blocking the serial characteristics of the macroblock lines of entropy decoding and achieving parallel calculation of macroblock lines. In the following embodiments, the entropy context processing solution is mainly described by taking the video decoder adopting a multi-core decoding solution, that is, entropy decoding adopts multi-core decoding as an example.

[0056] This application will mainly use the multi-core solution adopted by the video decoder in the electronic device as an example for explanation.

[0057] The process of entropy decoding performed by the electronic device 100 (such as the electronic device 100a and the electronic device 100b) of the present application is described in detail below with reference to the accompanying drawings.

[0058] Figure 2A As shown, it is a schematic diagram of the structure of the electronic device 100 provided in the embodiment of the present application, such as Figure 2A , which is a possible architecture diagram of an electronic device 100 .

[0059] like Figure 2AAs shown, the electronic device 100 may include a processor 110, a power module 140, a memory 180, a camera 170, a mobile communication module 130, a wireless communication module 120, a sensor module 190, an audio module 150, an interface module 160, and a display screen 102, etc.

[0060] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0061] The processor 110 may include one or more processing units, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a microprocessor (MCU), an artificial intelligence (AI) processor or a programmable logic device (Field Programmable Gate Array, FPGA) and a video processing unit (VPU) and other processing modules or processing circuits. Among them, different processing units can be independent devices or integrated in one or more processors.

[0062] For example, in some examples of the present application, the processor 110 may perform entropy decoding and other decoding operations on the compressed video data (i.e., the bitstream) through the VPU, and specifically, the bitstream may be entropy decoded through an entropy decoding module in the VPU. The entropy decoding module may be the entropy decoding module 100a, the entropy decoding module 100b1, or the entropy decoding module 100b2 described below.

[0063] The memory 180 can be used to store data, software programs and modules, and can be a volatile memory (VolatileMemory), such as a random access memory (Random-Access Memory, RAM), a double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR); or a non-volatile memory (Non-Volatile Memory), such as a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, HDD) or a solid-state drive (SSD); or a combination of the above types of memory, or it can also be a removable storage medium, such as a secure digital (Secure Digital, SD) memory card. Specifically, the memory 180 may include a program storage area (not shown) and a data storage area (not shown). For example, the data storage area (such as the data storage area in the DDR) can be used to store the entropy decoding that is backed up when the processor 110 performs entropy decoding on the code stream and a bit underflow occurs.

[0064] The mobile communication module 130 can provide wireless communication solutions including 2G / 3G / 4G / 5G etc. applied on the electronic device 100 .

[0065] The wireless communication module 120 may include an antenna, and transmit and receive electromagnetic waves via the antenna.

[0066] The wireless communication module 120 may provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), etc., applied to the electronic device 100. For example, the electronic device 100 may receive a code stream to be decoded from the electronic device 100 through the wireless communication module 120.

[0067] In some embodiments, the mobile communication module 130 and the wireless communication module 120 of the electronic device 100 may also be located in the same module.

[0068] The display screen 102 is used to display a human-computer interaction interface, images, videos, etc., such as displaying a video after decoding operations such as entropy decoding.

[0069] The sensor module 190 may include a proximity light sensor, a pressure sensor, and the like.

[0070] The audio module 150 is used to convert digital audio information into analog audio signals for output, or convert analog audio input into digital audio signals. In some embodiments, the audio module 150 may include a speaker, an earpiece, a microphone, and an earphone interface.

[0071] The camera 170 may be used to capture images, for example, to obtain images in a video to be subjected to encoding operations such as entropy encoding.

[0072] The interface module 160 includes an external memory interface, a universal serial bus (USB) interface, a subscriber identification module (SIM) card interface, and the like.

[0073] It is understandable that the above Figure 2A The hardware structure shown does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include Figure 2A More or fewer components may be shown, or some components may be combined or separated, or the components may be arranged differently.

[0074] Figure 2B According to an embodiment of the present application, a composition structure diagram of an entropy decoding module 100a of an electronic device 100 is shown. Figure 2B As shown, the entropy decoding module 100a includes: a video decoder 10a, an input buffer area 20a and an external storage area 30a.

[0075] The video decoder 10a is used to entropy decode the macroblocks in the bitstream one by one in combination with the entropy context to obtain the value of the syntax element and update and save the entropy context information, and is also used to back up the entropy context information to the external memory 30a. In some embodiments, the video decoder 10a can be implemented by a system on chip in the electronic device 100a.

[0076] The video decoder 10a includes core 0, core 1, core 2 and core 3. Each core includes a firmware, a context Ram and an entropy engine (or entropy core). The firmware can be implemented by software and is used to perform operations on the corresponding context Ram and entropy engine, such as performing read and write operations on the context Ram and controlling the entropy engine to start or end entropy decoding. For example, the firmware can be implemented by software.

[0077] As an example, the core 0 includes firmware 01, context Ram 02 and entropy engine 03. Specifically, when a bit stream is input into the core 0, the firmware 01 saves the kernel information of the core 0 into the external storage area 30, and the entropy engine 03 performs entropy decoding on the input bit stream and updates the entropy context stored in the context Ram. When the entropy engine 03 does not have enough bits to perform entropy decoding, the problem is reported to the firmware 01, indicating that a bit underflow problem occurs in the input buffer area 20, and the firmware 01 saves the entropy context backup stored in the context Ram 02 into the external storage area 30. Until the bit underflow problem of the input buffer area 20a is eliminated, the firmware 01 reads the entropy context from the external storage area 30 and restores it to the context Ram 02. Furthermore, the entropy engine 03 can perform entropy decoding on the newly input bit stream in combination with the entropy context in the context Ram 02 to restore the entropy decoding.

[0078] It can be understood that the above context Ram 02 is an internal static random access memory (SRAM) in the system on chip where the video decoder 10a is located.

[0079] The input buffer area 20 a is used to buffer the bit stream in the compressed video data (ie, code stream) received from the electronic device 200 .

[0080] In some embodiments, the input buffer area 20a may be a storage module integrated in the entropy decoding module 100a. In some other embodiments of the present application, the input buffer area 20a may also be an on-chip memory of a system-on-chip integrated in the electronic device 100a, for example, a static random access memory (SRAM) unit; or may be an off-chip memory integrated in the electronic device 100a, for example, a double data rate synchronous dynamic random access memory (DDR), a non-volatile memory, etc., without specific limitation.

[0081] The external storage area 30a is used to store the kernel information (such as the kernel serial number or name, etc.) in the video decoder 10a and save the entropy context that needs to be backed up. The external storage area 30 can be an off-chip memory on the electronic device 100a, such as a double data rate synchronous dynamic random access memory (DDR), a non-volatile memory, etc., which is not specifically limited. In addition, in some embodiments, the above-mentioned input buffer area 20a and the external storage area 30a can be set in the same DDR in the electronic device 100a.

[0082] Similarly, for the process of each unit in other cores 1-3 in the video decoder 10a interacting with the external storage area 30a to perform entropy context saving and recovery, reference may be made to the description related to core 0, which will not be repeated here.

[0083] It should be noted that based on Figure 2B In the entropy context processing scenario shown, the firmware performs the entropy context saving and recovery process between the internal context Ram of the video decoder and the external DDR where the external buffer is located, resulting in the most time-consuming part of the entropy decoding process being the saving and recovery of the entropy context. Moreover, when the video decoder uses multi-core decoding, the entropy context saving and recovery process is not only time-consuming but also causes high system bandwidth. Figure 2B Figure 2 shows the slow entropy context save and restore process.

[0084] In order to optimize the process of entropy context processing, the embodiment of the present application Figure 2B The hardware of the video decoder 10a is shown to be improved.

[0085] Reference Figure 3 As shown, Figure 3 FIG. 1 is a structural diagram showing a composition of an entropy decoding module 100b1 of an electronic device 100b according to an embodiment of the present application. Figure 3 As shown, the entropy decoding module 100b1 includes: a video decoder 10b, an input buffer area 20b and an external storage area 30b.

[0086] It should be noted that the difference between the structure of the entropy decoding module 100b1 provided in the embodiment of the present application and the above entropy decoding module 100b1 is mainly in the structure of the video decoder. Figure 3 The entropy context saving and restoration process shown is similar to Figure 2B The entropy context saving and restoring processes shown are different.

[0087] More specifically, if Figure 3 As shown, compared with the video decoder 10a, the video decoder 10b is newly added with one or more milestone Rams 11 and an arbitrator 12. Among them, the one or more milestone Rams 11 are used to store the entropy context in the context Ram in each core, and the arbitrator 12 is used to arbitrate the address or position of the entropy context information in different cores stored in the one or more milestone Rams 11, for example, arbitrate in which milestone Ram 11 the entropy context in the context Ram of a core is stored.

[0088] Accordingly, taking core 0 as an example, when bit underflow occurs, the entropy context in context Ram 02 is directly backed up and saved in a milestone Ram 11 in the chip, and then when the bit underflow problem is eliminated, the entropy context can be restored from the corresponding milestone Ram 11 to context Ram 02 through the arbiter 12. The firmware does not need to save and restore between the context Ram in the chip and the DDR where the external storage area 30b is located, thereby reducing the time consumption of entropy context saving and restoring, and avoiding the bandwidth between the on-chip Ram and the off-chip DDR in this process.

[0089] In addition, it should be noted that Figure 3 The input buffer area 20b shown is Figure 2B The input buffer 20a shown is the same. Figure 3 The external storage area 30b is shown with Figure 2B The difference of the external storage area 30a shown is that the entropy context in the entropy decoding process does not need to be stored in the external storage area 30b. In this way, as shown in Figure 3, high-speed entropy context preservation and restoration between the milestone Ram 11 and the context Ram in the kernel is provided.

[0090] In some embodiments, the video decoder 10b is a video decoder that supports one video standard, that is, the core in the video decoder 10b can only decode the bitstream of another video according to another video standard after decoding the bitstream of one video according to one video standard. For example, the core in the video decoder 10b first uses the AV1 standard to perform entropy decoding and other decoding processes on the bitstream of a video, and then uses the AV1 standard to perform entropy decoding and other decoding processes on the bitstream of another video. As an example, in this embodiment, the one or more milestone Rams 11 can be newly added Rams in the video decoder 10b, which are mainly used to store the entropy context generated when bit underflow occurs during entropy decoding.

[0091] In other embodiments, the video decoder 10b is a video decoder that supports multiple video standards, that is, the core in the video decoder 10b can use multiple video standards to decode different video streams in time-division multiplexing. For example, in the process of decoding the code stream of a video using the AV1 standard, the core in the video decoder 10b uses another video standard (such as the EP9 standard) to decode the code stream of another video in time-division multiplexing. It can be understood that the storage resources in the video decoder 10b that supports multiple video standards are relatively sufficient. For example, the video decoder 10b of the video standard contains RAM corresponding to the decoding of other video standards (such as the EP9 standard) (recorded as time-division multiplexing RAM). In the multi-standard decoding process, when the core uses AV1 to decode the code stream, the time-division multiplexing RAM corresponding to the EP9 standard decoding is in an idle state, and the time-division multiplexing RAM can be used as the milestone Ram 11 used in the AV1 standard decoding process. Then, as an example, in this embodiment, the one or more milestone Rams 11 are all Rams in the video decoder 10b corresponding to the decoding of other video standards except AV1. Alternatively, part of the one or more milestone Rams 11 is the Ram corresponding to the decoding of other video standards except AV1 in the video decoder 10b, and the other part is the Ram newly added in the video decoder 10b. In other words, the one or more milestone Rams 11 can be implemented by the existing Ram in the video decoder 10b, or by the newly added Ram.

[0092] Embodiment 1

[0093] In some embodiments of the present application, Figure 3 The number of the one or more milestone Rams 11 in the video decoder 10b is less than the number of the cores in the video decoder 10b. At this time, the arbiter 12 is used to arbitrate in which storage addresses of the one or more milestone Rams 11 the entropy contexts in different context Rams should be stored.

[0094] Combination Figure 3 The structure diagram of the entropy decoding module 100b1 of the electronic device 100b is described in detail. The process of an entropy context processing method of the present application is described.

[0095] Figure 4 According to an embodiment of the present application, a flowchart of entropy context processing is shown, and the specific process includes the following steps:

[0096] S401: The input buffer 20b inputs bit streams to cores 0-3 in the video decoder 10b respectively.

[0097] As an example, the bit stream input to one core is a macroblock row, and cores 0-3 can respectively input consecutive macroblock rows.

[0098] S402: The firmware in the cores 0-3 of the video decoder 10b respectively saves the core information into the external storage area 30b.

[0099] Among them, the kernel information stored in the above-mentioned external storage area 30b is related to the macroblock row corresponding to the bitstream. For example, the kernel information stored in kernel 0 is related to the macroblock row corresponding to the bitstream to be processed by kernel 0, which is used to distinguish which kernel processes the bitstream corresponding to which macroblock row.

[0100] S403: The entropy engines in the cores 0-3 of the video decoder 10b respectively perform entropy decoding on the input bit streams and update the entropy contexts in their respective contexts Ram.

[0101] For example, the entropy context in the context Ram 02 in the current kernel 0 may include parameters such as an entropy context model updated after entropy decoding of the previous macroblock.

[0102] S404: When the bits input to the entropy engine 03 in the core 0 of the cores 0-3 of the video decoder 10b are insufficient, the entropy engine 03 of the video decoder 10b reports first information indicating that a bit underflow occurs to the firmware 01.

[0103] It is understandable that the above S403 takes the insufficient input bits in the entropy engine 03 in the core 0 as an example to illustrate that the available bits in the input buffer 20b have been consumed and there are no new available bits, that is, bit underflow has occurred. In some other embodiments, when bit underflow occurs in the input buffer 20b, one or more cores in the cores 0-3 may detect that the input bits are insufficient and trigger the above first information.

[0104] S405 : the firmware 01 in the cores 0 - 3 of the video decoder 10 b respectively initiates a write request to the arbitrator 12 , where a write request is used to request to write the entropy context in the corresponding context Ram into one or more milestone Rams 11 .

[0105] In some embodiments, cores 0-3 in S405 may simultaneously initiate a write request to arbiter 12.

[0106] In some other embodiments, core 0 may initiate a write request first, and cores 1-3 may initiate write requests later. It is understandable that when core 0 exports the backup entropy context, the entropy engines in cores 1-3 may continue to entropy decode the macroblocks in the currently input bitstream until the current entropy decoded macroblock processing is completed, and then stop entropy decoding and update the entropy context in their respective context Rams, and then request to back up the updated entropy context to milestone Ram 11.

[0107] In some embodiments, the write request corresponding to each core carries the core information of the core. For example, the core information may be the identification, name or number of the core.

[0108] S406: The arbiter 12 in the video decoder 10b returns a write permission response to the firmware in each core according to each write request.

[0109] It can be understood that the above-mentioned write enable response is used to indicate that the entropy context in the context Ram of the corresponding core is allowed to write one or more milestone Ram 11 .

[0110] In some embodiments of the present application, after receiving a read request, the arbitrator 12 may query whether there is a free storage area in one or more milestone Rams 11, as well as the address of the free storage area, etc. If a free area is found, a write permission response is returned, and if there is no free area, a write rejection response is returned, and the write rejection response is used to indicate that the entropy context in the corresponding context Ram is not allowed to be written to one or more milestone Rams 11.

[0111] In some embodiments, a write enable response may also include a storage address of the milestone Ram 11 allocated by the arbitrator 12 to the entropy context in the corresponding context Ram.

[0112] S407 : The firmware in the cores 0 - 3 in the video decoder 10 b sends the entropy context in the corresponding context Ram to the arbitrator 12 .

[0113] It is understandable that the firmware in each core sends the entropy context in the corresponding context Ram to the arbitrator 12 at the same time or in a sequential order, which is not described in detail herein.

[0114] S408: The arbiter 12 in the video decoder 10b stores the entropy context in the corresponding context Ram to the corresponding milestone Ram 11 (ie, the milestone Ram11 where the allocated storage address is located) according to the storage address corresponding to each core.

[0115] In the embodiment of the present application, the video decoder 10b can save a backup of the entropy context on-chip, without the need to store the entropy context in an off-chip DDR where the external storage area 30b is located, which reduces time consumption and avoids high bandwidth generated between the on-chip Ram and the off-chip DDR.

[0116] It can be understood that when core 0 exports the backup entropy context, the entropy engines in cores 1-3 can continue to perform entropy decoding on the macroblocks in the currently input bitstream until the processing of the macroblock currently entropy decoded is completed, and then stop entropy decoding and update the entropy context in their respective context Rams, and then request to back up the updated entropy context to milestone Ram 11.

[0117] In addition, the arbiter 12 may establish an association relationship between a storage address of an entropy context in a core and the core, for example, an storage address of an entropy context in the milestone Ram 11 is associated with the core information of the corresponding core.

[0118] S409 : When the video stream in the input buffer 20 b is re-input into the cores 0 - 3 of the video decoder 10 b , the firmware in the cores 0 - 3 respectively initiates a read request to the arbitrator 12 .

[0119] It is understandable that the video stream in the input buffer 20b is re-input into the cores 0-3 of the video decoder 10b, indicating that enough bits are newly stored in the input buffer 20b for parsing and consumption by the entropy engines in each core.

[0120] Among them, a read request is used to request to read the entropy context in the corresponding milestone Ram 11 and write it into the corresponding context Ram.

[0121] In some embodiments, cores 0 - 3 in S405 may simultaneously initiate a read request to arbiter 12 .

[0122] In some embodiments, the read request corresponding to each core carries the core information of the core.

[0123] S410: The arbiter 12 in the video decoder 10b reads the entropy context corresponding to each core according to each read request.

[0124] In some embodiments of the present application, after receiving a read request from a core, the arbitrator 12 may query a storage address corresponding to the core, read the stored entropy context from the storage address, and clear the storage area.

[0125] S411: The arbitrator 12 in the video decoder 10b saves the read entropy context into the context Ram of the corresponding core.

[0126] It is understandable that the arbitrator 12 can save (ie, restore) the entropy context in the milestone Ram 11 in the context Ram of each core at the same time, or in a sequential order (such as the order of the core numbers from small to large, etc.), which is not elaborated in this article.

[0127] S412: The entropy engine in the core 0-3 in the video decoder 10b re-entropy decodes the newly input bit stream in combination with the entropy context restored in the corresponding context Ram.

[0128] It should be noted that the entropy engine can re-entropy decode the macroblock row where the entropy decoding was paused or failed last time.

[0129] In the embodiment of the present application, the video decoder 10b can save a backup of the entropy context on-chip, without the need to store the entropy context in an off-chip DDR where the external storage area 30b is located, which reduces time consumption and avoids high bandwidth generated between the on-chip Ram and the off-chip DDR.

[0130] Embodiment 2

[0131] In other embodiments of the present application, the number of the one or more milestones Ram 11 is equal to the number of cores in the video decoder 10b, for example, a context generated by a core is stored in a unique corresponding milestone Ram 11. Figure 3 The arbiter 12 can be removed from the video decoder 10 b shown, that is, the arbiter 12 is not needed to arbitrate in which of the multiple milestone Rams 11 the entropy context in a core is stored.

[0132] based on Figure 3 The entropy decoding module 100b1 shown, referring to Figure 5 FIG. 1 is a diagram showing another architecture of an entropy decoding module provided in an embodiment of the present application. Figure 5 The entropy decoding module 100b2 shown is compared to Figure 3 The entropy decoding module 100b1 shown is different in that four milestones Ram 11 are provided in the entropy decoding module 100b2, that is, each core is uniquely connected to a corresponding milestone Ram 11; and the arbiter 12 is not provided.

[0133] Next, combine Figure 6 The structure diagram of the entropy decoding module 100b2 of the electronic device 100b is described in detail. The process of an entropy context processing method of the present application is described.

[0134] Figure 6 According to an embodiment of the present application, a flowchart of entropy context processing is shown, and the specific process includes the following steps:

[0135] S601: The input buffer 20b inputs bit streams to cores 0-3 in the video decoder 10b respectively.

[0136] S602: The firmware in the cores 0-3 of the video decoder 10b respectively saves the core information into the external storage area 30b.

[0137] S603: The entropy engines in the cores 0-3 of the video decoder 10b respectively perform entropy decoding on the input bit streams and update the entropy contexts in their respective contexts Ram.

[0138] S604: When the bits input to the entropy engine 03 in the core 0 of the cores 0-3 of the video decoder 10b are insufficient, the entropy engine 03 reports first information indicating that a bit underflow occurs to the firmware 01.

[0139] It should be noted that Figure 6 The description of S601-S604 shown is the same as above Figure 4 S401-S404 are shown to be the same and will not be described in detail here. Figure 6 The process shown is Figure 4 The difference in the process is that S405 to S412 are replaced by the following S605 to S607. Figure 6 The illustrated flow does not require the arbiter 12 to determine how to store or read the entropy context in the milestone Ram 11 .

[0140] S605: The firmware in the core 0-3 of the video decoder 10b writes the entropy context in the corresponding context Ram into the milestone Ram 11 corresponding to the core.

[0141] In some embodiments, the firmware in the core 0-3 in S605 can directly initiate a write request to the corresponding milestone Ram11 to request to write the entropy context information, and the write request can also carry the corresponding core information. It is understandable that there is no specific limitation on the timing of initiating a write request from the firmware in the core 0-3 to the corresponding milestone Ram 11, which can be simultaneous or sequential. For example, when the core 0 exports the backup entropy context, the entropy engine in the core 1-3 can continue to entropy decode the macroblocks in the currently input bitstream until the current entropy decoded macroblock processing is completed, and then stop entropy decoding and update the entropy context in the respective context Ram, and then initiate a write request to the corresponding milestone Ram 11 to request that the updated entropy context be backed up and saved.

[0142] In some embodiments, the write request corresponding to each core carries the core information of the core. For example, the core information may be the identification, name or number of the core.

[0143] In some embodiments, the preset storage area in the milestone Ram 11 corresponding to each core is used to store the corresponding entropy context. Furthermore, when the bit underflow occurs again in the input buffer 20b, the entropy context in the preset storage area in each milestone Ram 11 is updated along with the entropy context in the corresponding context Ram.

[0144] In addition, each core may be associated with a preset storage address in the corresponding milestone Ram 11 for storing the entropy context.

[0145] S606: When the video stream in the input buffer 20b is re-input into the core 0-3 of the video decoder 10b, the firmware in the core 0-3 restores the entropy context in the corresponding milestone Ram 11 to the corresponding context Ram.

[0146] It can be understood that the bit stream in the input buffer 20b is re-input into the cores 0-3 of the video decoder 10b, indicating that enough bits are newly stored in the input buffer 20b for parsing and consumption by the entropy engines in each core.

[0147] In some embodiments, the firmware in cores 0-3 may respectively initiate a read request to the corresponding milestone Ram 11 to request to read the entropy context in the corresponding milestone Ram 11 and write it into the corresponding context Ram.

[0148] In some embodiments, cores 0-3 in S605 may simultaneously initiate a read request to the corresponding milestone Ram 11, or may initiate the read request sequentially.

[0149] In some embodiments, the read request corresponding to each core carries the core information of the core.

[0150] S407: The entropy engine in the core 0-3 in the video decoder 10b re-entropy decodes the newly input bit stream in combination with the entropy context restored in the corresponding context Ram.

[0151] It should be noted that the entropy engine can re-entropy decode the macroblock row where the entropy decoding was paused or failed last time.

[0152] In the embodiment of the present application, the video decoder 10b can save the backup of the entropy context in the chip, without the need to store the entropy context in the off-chip DDR where the external storage area 30b is located, which reduces the time consumption and avoids the high bandwidth generated between the on-chip Ram and the off-chip DDR. In addition, since the number of milestone Rams 11 is the same as the number of context Rams, there is no need to set up an additional arbitrator to arbitrate the writing or reading of entropy context information in multiple milestone Rams 11 by different context Rams, which simplifies the process of entropy context processing.

[0153] Figure 7 FIG. 1 is a block diagram of an electronic device 100 according to an embodiment of the present application. Figure 7An example electronic device 100 is schematically shown according to various embodiments. In one embodiment, the electronic device 100 may include one or more processors 1604, a system control logic 1608 connected to at least one of the processors 1604, a system memory 1612 connected to the system control logic 1608, a non-volatile memory (NVM) 1616 connected to the system control logic 1608, and a network interface 1620 connected to the system control logic 1608.

[0154] In some embodiments, the processor 1604 may include one or more single-core or multi-core processors. In some embodiments, the processor 1604 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments where the electronic device 100 uses an eNB (Evolved Node B) or RAN (Radio Access Network) controller, the processor 1604 may be configured to execute various embodiments, such as Figure 3 and Figure 5 One or more of the multiple embodiments shown.

[0155] In some embodiments, system control logic 1608 may include any suitable interface controller to provide any suitable interface to at least one of processors 1604 and / or any suitable device or component in communication with system control logic 1608 .

[0156] In some embodiments, the system control logic 1608 may include one or more memory controllers to provide an interface to the system memory 1612. The system memory 1612 may be used to load and store data and / or instructions. In some embodiments, the memory 1612 of the system 1600 may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).

[0157] NVM 1616 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, NVM 1616 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a HDD (Hard Disk Drive), a CD (CompactDisc) drive, and a DVD (Digital Versatile Disc) drive.

[0158] NVM 1616 may include a portion of storage resources on a device on which electronic device 100 is installed, or it may be accessible by the device but is not necessarily a portion of the device. For example, NVM 1616 may be accessed over a network via network interface 1620.

[0159] In particular, system memory 1612 and NVM 1616 may include, respectively, a temporary copy and a permanent copy of instructions 1624. Instructions 1624 may include instructions that, when executed by at least one of processors 1604, cause electronic device 100 to implement the following: Figure 4 or Figure 6 In some embodiments, instructions 1624, hardware, firmware, and / or software components thereof may additionally / alternatively be placed in system control logic 1608, network interface 1620, and / or processor 1604.

[0160] The network interface 1620 may include a transceiver for providing a radio interface for the electronic device 100, thereby communicating with any other suitable device (such as a front-end module, an antenna, etc.) through one or more networks. In some embodiments, the network interface 1620 may be integrated with other components of the electronic device 100. For example, the network interface 1620 may be integrated with at least one of the processor 1604, the system memory 1612, the NVM 1616, and a firmware device (not shown) having instructions. When at least one of the processors 1604 executes the instructions, the electronic device 100 implements the following. Figure 5 The method shown.

[0161] The network interface 1620 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 1620 may be a network adapter, a wireless network adapter, a telephone modem and / or a wireless modem.

[0162] In one embodiment, at least one of the processors 1604 may be packaged together with logic for one or more controllers of the system control logic 1608 to form a system in package (SiP). In one embodiment, at least one of the processors 1604 may be integrated on the same die with logic for one or more controllers of the system control logic 1608 to form a system on a chip (SoC).

[0163] The electronic device 100 may further include an input / output (I / O) device 1632 .

[0164] According to an embodiment of the present application, Figure 8 1 shows a block diagram of a SoC (System on Chip) 1700. The SOC 1700 is provided in the electronic device 100. Figure 8 In the drawings, similar parts have the same reference numerals. Figure 8 In the embodiment, SoC 1700 includes: an interconnect unit 1750, which is coupled to an application processor 1710; a system agent unit 1770; a bus controller unit 1780; an integrated memory controller unit 1740; a video processor 1720, which also includes an entropy decoding module 100b1 or 100b2; a static random access memory (SRAM) unit 1730; and a direct memory access (DMA) unit 1760. In one embodiment, SoC 1700 may also include, for example, a network or communication processor, a compression engine, a GPU, a high throughput MIC processor, or an embedded processor.

[0165] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0166] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0167] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0168] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Therefore, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.

[0169] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required, but rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0170] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0171] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0172] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims

1. An entropy context processing method, characterized in that: Applied to a system on chip including a plurality of cores and at least one storage unit, the method comprises: detecting that input bit conditions of the plurality of cores do not satisfy a preset condition, wherein the preset condition is used to indicate that bit underflow occurs in the input bits of the plurality of cores; The plurality of cores respectively back up the cached entropy context to the at least one storage unit; It is detected that the input bit conditions of the plurality of cores meet the preset conditions; The plurality of cores respectively obtain corresponding entropy contexts from the at least one storage unit, and re-cache the corresponding entropy contexts; The multiple cores perform entropy decoding on different macroblock rows in the input bitstream in parallel and update the cached entropy context.

2. The method according to claim 1, characterized in that The preset condition is: at least one of the multiple cores has not input a bitstream within a preset time period, or the number of bits of the bitstream input into the multiple cores is less than a preset number.

3. The method according to claim 2, characterized in that The number of the at least one storage unit is the same as the number of the plurality of cores, and one core uniquely corresponds to one storage unit; Each of the plurality of cores is configured to store or obtain an entropy context in a uniquely corresponding storage unit in the at least one storage unit.

4. The method according to claim 1, characterized in that: The system on chip further includes an arbiter connected to the at least one storage unit, the number of the at least one storage unit being less than the number of the plurality of cores; The method further comprises: The multiple cores respectively send a storage request message to the arbitrator; The arbiter allocates a storage address to each of the plurality of cores based on the plurality of storage request messages; The arbitrator returns a storage permission response message to each of the plurality of cores; The entropy context cached by each core is stored in the at least one storage unit according to the storage address assigned by the arbitrator.

5. The method according to claim 4, characterized in that One of the storage request messages includes at least one of the following: identification information of the kernel, an entropy context of a corresponding kernel cache, and first indication information indicating a request to store the entropy context; A response message allowing storage includes at least one of the following items: identification information of the corresponding kernel, identification of the storage address allocated by the corresponding kernel, and second indication information indicating that the corresponding kernel is allowed to store the corresponding entropy context in the at least one storage unit.

6. The method according to claim 5, characterized in that Each of the cores includes a firmware, an entropy engine and a context cache unit; Each of the cores entropy decodes the bit stream through an entropy engine and generates an entropy context; each of the cores caches the entropy context through a context cache unit; each of the cores initiates a storage request to the arbitrator through firmware, and backs up the entropy context cached in the corresponding context cache unit to the at least one storage unit, and restores the entropy context obtained from the at least one storage unit to the corresponding context cache unit.

7. The method according to claim 6, characterized in that The multiple cores entropy decode the bitstream according to the macroblock rows in the bitstream, and one core entropy decodes the bitstream according to the macroblocks in the macroblock row, wherein one macroblock row corresponds to one core, and an entropy context cached by one core corresponds to one macroblock.

8. The method according to claim 7, characterized in that The system on chip is connected to a target storage unit outside the system on chip; The method further comprises: Each of the multiple cores sends a core processing information to the target storage unit, wherein the core processing information includes at least one of the following: information of the corresponding core, information of the bit stream, macroblock or macroblock row input in the corresponding core.

9. A system on chip, characterized in that: The system on chip includes a plurality of cores and at least one storage unit; The plurality of cores are used to detect that the input bit status of the plurality of cores does not satisfy a preset condition; backing up the cached entropy contexts to the at least one storage unit respectively, the preset condition being used to indicate that a bit underflow occurs in input bits of the plurality of kernels; the at least one storage unit for storing entropy contexts from the plurality of kernels; The multiple cores are further used to detect that the input bit conditions of the multiple cores meet the preset conditions; The corresponding entropy context is respectively obtained from the at least one storage unit, and the corresponding entropy context is re-cached; entropy decoding is performed on different macroblock rows in the input bit stream in parallel, and the cached entropy context is updated.

10. The system according to claim 9, characterized in that The preset condition is: at least one of the multiple cores has not input a bitstream within a preset time period, or the number of bits of the bitstream input into the multiple cores is less than a preset number.

11. The system according to claim 10, characterized in that The number of the at least one storage unit is the same as the number of the plurality of cores, and one core uniquely corresponds to one storage unit; Each of the plurality of cores is configured to store or obtain an entropy context in a uniquely corresponding storage unit in the at least one storage unit.

12. The system according to claim 9, characterized in that The system on chip further includes an arbiter connected to the at least one storage unit, the number of the at least one storage unit being less than the number of the plurality of cores; The multiple cores are further configured to respectively send a storage request message to the arbitrator; The arbiter is used to allocate a storage address to each of the multiple cores based on multiple storage request messages; Returning a storage permission response message to each of the plurality of cores; The entropy context cached by each core is stored in the at least one storage unit according to the storage address assigned by the arbitrator.

13. The system according to claim 12, characterized in that One of the storage request messages includes at least one of the following: identification information of the kernel, an entropy context of a corresponding kernel cache, and first indication information indicating a request to store the entropy context; A response message allowing storage includes at least one of the following items: identification information of the corresponding kernel, identification of the storage address allocated by the corresponding kernel, and second indication information indicating that the corresponding kernel is allowed to store the corresponding entropy context in the at least one storage unit.

14. The system according to claim 12, characterized in that Each of the cores includes a firmware, an entropy engine and a context cache unit; Each of the cores entropy decodes the bit stream through an entropy engine and generates an entropy context; each of the cores caches the entropy context through a context cache unit; each of the cores initiates a storage request to the arbitrator through firmware, and backs up the entropy context cached in the corresponding context cache unit to the at least one storage unit, and restores the entropy context obtained from the at least one storage unit to the corresponding context cache unit.

15. The system according to claim 14, characterized in that The multiple cores entropy decode the bitstream according to the macroblock rows in the bitstream, and one core entropy decodes the bitstream according to the macroblocks in the macroblock row, wherein one macroblock row corresponds to one core, and an entropy context cached by one core corresponds to one macroblock.

16. The system according to claim 15, characterized in that The system on chip is connected to a target storage unit outside the system on chip; Each of the multiple cores is also used to send a core processing information to the target storage unit, wherein the core processing information includes at least one of the following: information of the corresponding core, information of the bit stream, macroblock or macroblock row input in the corresponding core.

17. An electronic device, characterized in that: The electronic device includes a system on chip as described in any one of claims 9 to 16, and a target storage unit outside the system on chip; the target storage unit is used to store kernel processing information sent by the multiple kernels, wherein one kernel processing information includes at least one of the following: information of the corresponding kernel, information of a bit stream, macroblock or macroblock row input into the corresponding kernel.

Images