Low-power-consumption audio and video decoding system and method based on plug-in product

Abstract

The invention relates to the technical field of audio and video decoding, and relates to a low-power-consumption audio and video decoding system and method based on a plug-in product. The system comprises a main control unit which integrates a multi-core processor and a task scheduler and is used for distributing decoding tasks in real time and supporting dynamic frequency adjustment; the plug-in decoding module is connected with the main control unit through an expansion interface; the intelligent power consumption management module is connected with the main control unit and the plug-in decoding module, comprises a load prediction algorithm and a self-adaptive voltage regulation unit, and is used for dynamically regulating the working voltage and the clock frequency of the decoding module according to the task load; and the power supply management module supplies power to the main control unit, the plug-in decoding module and the intelligent power consumption management module. Independent upgrading of the decoder is realized through plug-in hardware modular design, and the problems of high energy consumption, low expansibility, large synchronization delay and the like in a traditional scheme are solved by combining an intelligent task allocation strategy and a self-adaptive voltage regulation technology.

Description

Technical Field

[0001] This application relates to the technical field of audio and video decoding, and relates to a low-power audio and video decoding system and method based on plug-in products. Background Art

[0002] Currently, audio and video coding and decoding technology is the core of digital media transmission and processing. Its core goal is to reduce the data volume while ensuring the quality of audio and video through compression, encoding, and decoding technologies to adapt to storage, transmission, and real-time processing requirements. With the popularization of 5G, ultra-high definition (4K / 8K), Internet of Things (IoT), and artificial intelligence technologies, audio and video coding and decoding technology needs to continuously break through multi-dimensional performance bottlenecks such as efficiency, latency, and power consumption. With the rapid popularization of ultra-high definition video, low-latency streaming media, and heterogeneous coding formats, existing decoding solutions can no longer meet market demands in terms of energy efficiency ratio, dynamic scalability, and multi-standard compatibility. The audio and video decoding technology has the following key bottlenecks:

[0003] Mainstream coding and decoding standards and technical frameworks include:

[0004] Hybrid coding technology: Represented by H.264 / AVC and H.265 / HEVC, it achieves efficient compression through modules such as motion compensation, intra / inter-frame prediction, and transform coding. The compression efficiency of H.265 is more than 50% higher than that of H.264, but the algorithm complexity increases significantly.

[0005] Scalable Video Coding (SVC): Supports temporal, spatial, and quality scalability, adapts to network bandwidth fluctuations, but is complex to implement and has limited compatibility, and its actual application promotion is slow.

[0006] Domestic standard AVS series: AVS3 is optimized for 8K ultra-high definition and 5G scenarios, and its compression efficiency is close to H.266 / VVC, but its international market share still needs to be expanded.

[0007] Scenarios such as 4K / 8K video, cloud gaming, and remote medical care require coding and decoding technology to balance high image quality and real-time performance. Existing technologies are prone to a sharp increase in power consumption and latency fluctuations at high resolutions. AI-based content-aware coding (such as Huawei JPEGAI) has gradually emerged, but it needs to solve the problems of algorithm complexity and hardware adaptation.

[0008] Currently, the following technical problems are also faced:

[0009] Rigid static power consumption management mechanism

[0010] Traditional hardware decoding units operate in a fixed-frequency mode and cannot dynamically adjust energy consumption according to real-time task loads, resulting in the device being in a high-power state for a long time. This problem is particularly significant when decoding high-resolution (such as 4K / 8K) or high-dynamic range (HDR) content. The device overheats severely and the battery life drops significantly.

[0011] Lack of multi - format compatibility and insufficient heat dissipation efficiency

[0012] The adaptation of existing solutions to new coding standards (such as H.266 / VVC, AV1) relies on software post - processing, requiring frequent upgrades of the main control chip firmware, and lacking replaceable modular interfaces at the hardware level. In addition, the low decoding efficiency in complex scenarios easily causes the chip to overheat, triggering the system frequency - down protection mechanism and resulting in the degradation of audio - video quality.

[0013] Limited system scalability and resource allocation

[0014] The highly integrated design of the decoding module and the main control chip results in fixed hardware functions and inability to flexibly expand computing power through external modules. The existing dynamic resource allocation mechanism lags in responding to sudden high - load scenarios, often causing waste of memory resources or decoding delays due to task scheduling conflicts.

[0015] Inefficient audio - video synchronization mechanism

[0016] Traditional synchronization technologies achieve audio - video synchronization by expanding the buffer. Although it can alleviate the impact of data stream fluctuations, it significantly increases memory occupancy and end - to - end latency, making it difficult to meet the requirements of real - time interaction scenarios.

[0017] There is an urgent need for a decoding architecture that supports hot - plugging of hardware modules and has load self - sensing capabilities, achieving triple breakthroughs in power consumption, performance, and compatibility through software - hardware collaborative optimization.

[0018] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0019] To have a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. The summary is not a general review, nor is it intended to identify key / important constituent elements or delineate the protection scope of these embodiments, but rather serves as a preface to the subsequent detailed description.

[0020] The embodiments of the present disclosure provide a low - power audio - video decoding system and method based on plug - in products. Through plug - in hardware modular design, the decoder can be independently upgraded. Combining intelligent task allocation strategies and adaptive voltage regulation technologies, problems such as high energy consumption, low scalability, and large synchronization delays in traditional solutions are solved.

[0021] In some embodiments, the system includes:

[0022] A main control unit, integrating a multi - core processor and a task scheduler, for real - time allocation of decoding tasks and supporting dynamic frequency adjustment;

[0023] The plug-in decoding module is connected to the main control unit through an expansion interface, includes a dedicated ASIC chip, supports hardware decoding of H.264, H.265, and AV1 formats, and can be upgraded and replaced independently of the main control unit.

[0024] The intelligent power management module is connected to the main control unit and the plug-in decoding module, and includes a load prediction algorithm and an adaptive voltage regulation unit for dynamically adjusting the working voltage and clock frequency of the decoding module according to the task load.

[0025] The power management module supplies power to the main control unit, the plug-in decoding module, and the intelligent power management module.

[0026] Preferably, the main control unit includes a multi-core processor, which integrates a CPU, a GPU, and an NPU, and distributes high-complexity tasks to the plug-in decoding module through a task scheduler, and low-complexity tasks to the decoding unit built into the main control unit.

[0027] Preferably, the load prediction algorithm of the intelligent power management module uses an LSTM recurrent neural network model, the prediction period is the next decoding cycle, the dynamic voltage adjustment range is 0.8V to 1.2V, and the clock frequency range is 200MHz to 1GHz.

[0028] Preferably, the plug-in decoding module is built with a heat sink, aligned with the motherboard air duct, and realizes an initial low-power mode through hierarchical power supply.

[0029] Preferably, the system adopts a zero-buffer synchronization technology to achieve audio-visual synchronous rendering based on the timestamp matching of audio-visual output triggered by hardware interrupts.

[0030] It also adopts an intelligent sleep mechanism to control the plug-in decoding module to enter a deep sleep mode when there is no valid data input for a continuous preset number of frames.

[0031] Preferably, the task scheduler switches to a low-bitrate decoding mode during network fluctuations to reduce buffer delay.

[0032] In some embodiments, the method includes the following steps:

[0033] The audio-visual bitstream is input into the system, the device is initialized, the main control unit loads the dynamic power management driver and the task scheduler, and detects and activates the plug-in decoding module.

[0034] The input audio-visual stream is parsed for metadata by the main control unit, and decoding tasks are allocated for decoding according to the encoding format and resolution.

[0035] Predict the load through the LSTM model and dynamically adjust the voltage and frequency of the plug-in decoding module.

[0036] Synchronize audio and video output using hardware timestamps and adjust the bitrate in real time to cope with network fluctuations;

[0037] Trigger a sleep instruction when there is no data input to turn off the power supply of the plug-in decoding module.

[0038] Preferably, the method of allocating decoding tasks according to the encoding format and resolution for decoding is as follows: for low-complexity tasks, the decoding tasks are preferentially allocated to the plug-in decoding module; for high-complexity tasks, they are processed by the built-in decoding unit of the main control unit. The low-complexity tasks include audio decoding tasks, video decoding tasks of non-new encoding standards, or video decoding tasks with a resolution less than 1080P, and the high-complexity tasks include video decoding tasks of new encoding standards and with a resolution greater than or equal to 1080P.

[0039] In some embodiments, the device includes a processor and a memory storing program instructions. The processor is configured to execute the low-power audio and video decoding method based on the plug-in product when running the program instructions.

[0040] In some embodiments, the storage medium stores a computer program, and when the program is executed by a processor, it implements the low-power audio and video decoding method based on the plug-in product.

[0041] A low-power audio and video decoding system and method based on a plug-in product provided by an embodiment of the present disclosure can achieve the following technical effects:

[0042] The present invention realizes independent upgrade of the decoder through plug-in hardware modular design, combines intelligent task allocation strategies and adaptive voltage regulation technologies, and solves problems such as high energy consumption, low scalability, and large synchronization delay in traditional solutions. Through the improved maintainability brought by hardware modularity, the full-life cycle maintenance cost of the device is reduced, which is particularly suitable for scenarios requiring high-density video processing. Measured data shows excellent environmental adaptability in typical load fluctuation scenarios.

[0043] The above general description and the following description are only exemplary and explanatory, and are not used to limit this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] One or more embodiments are exemplarily illustrated by corresponding drawings. These exemplary illustrations and the drawings do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements. The drawings do not constitute a proportional limitation, and among them:

[0045] Figure 1 is a schematic diagram of the system architecture;

[0046] Figure 2It is a schematic diagram of a method flow;

[0047] Figure 3 It is a schematic diagram of a dynamic task allocation process;

[0048] Figure 4 It is a schematic diagram of the device structure of the present invention. Detailed implementation manners

[0049] In order to be able to understand the features and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are only for reference and explanation purposes and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of explanation, multiple details are provided to provide a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices can be shown in a simplified manner to simplify the drawings.

[0050] In the embodiments of the present disclosure, terms such as "first" and "second" in the description and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so as to implement the embodiments of the present disclosure described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion.

[0051] Unless otherwise specified, the term "plurality" means two or more.

[0052] In the embodiments of the present disclosure, the character " / " indicates that the objects before and after are in an "or" relationship. For example, A / B means: A or B.

[0053] The term "and / or" is a description of the associated relationship of an object and indicates that three relationships can exist. For example, A and / or B means: A or B, or, the three relationships of A and B.

[0054] The term "corresponding" may refer to an associated relationship or a binding relationship. A corresponding to B means that there is an associated relationship or a binding relationship between A and B.

[0055] Embodiment 1

[0056] As Figure 1 shown, a low-power audio and video decoding system based on plug-in products realizes independent upgrade of the decoder through plug-in hardware modular design, combines an intelligent task allocation strategy (coordination between the main control CPU and the dedicated decoding module) and an adaptive voltage regulation technology to solve problems such as high energy consumption, low scalability, and large synchronization delay in traditional solutions.

[0057] The system includes:

[0058] The main control unit integrates a multi-core processor and a task scheduler, which is used for real-time allocation of decoding tasks and supports dynamic frequency adjustment.

[0059] Specifically, the main control unit includes a multi-core processor, which integrates a CPU, a GPU, and an NPU, and distributes high-complexity tasks to the plug-in decoding module through the task scheduler, and low-complexity tasks to the decoding unit built in the main control unit.

[0060] The plug-in decoding module is connected to the main control unit through an expansion interface, includes a dedicated ASIC chip, supports hardware decoding of H.264, H.265, and AV1 formats, and can be upgraded and replaced independently of the main control unit.

[0061] Specifically, the plug-in decoding module is built with a heat sink, aligned with the motherboard air duct, and realizes the initial low-power mode through hierarchical power supply.

[0062] The intelligent power management module is connected to the main control unit and the plug-in decoding module, and includes a load prediction algorithm and an adaptive voltage regulation unit, which are used to dynamically adjust the working voltage and clock frequency of the decoding module according to the task load.

[0063] Specifically, the load prediction algorithm of the intelligent power management module adopts an LSTM recurrent neural network model, the prediction period is the next decoding cycle, the dynamic voltage adjustment range is 0.8V to 1.2V, and the clock frequency range is 200MHz to 1GHz.

[0064] The power management module supplies power to the main control unit, the plug-in decoding module, and the intelligent power management module.

[0065] As a refinement of the above embodiment, the main control unit includes a multi-core processor, which integrates a CPU, a GPU, and an NPU, and distributes decoding tasks to the plug-in decoding module or the decoding unit built in the main control chip in real time. Specifically, high-complexity tasks are distributed to the plug-in decoding module through the task scheduler, and low-complexity tasks are distributed to the decoding unit built in the main control unit. The main control unit preferentially distributes high-complexity decoding tasks (such as 4K HDR, 8K) to the plug-in decoding module, and low-complexity tasks (such as audio streams) are processed by the software decoding unit of the main control chip.

[0066] Through the cooperation of the multi-core heterogeneous processor (CPU + GPU + NPU) and the intelligent task scheduler, three-dimensional scheduling of computing resources is realized; the task complexity grading processing mechanism (the cooperation of the plug-in decoding module and the built-in unit) improves the computing efficiency and reduces the response delay, and the introduction of the NPU improves the energy efficiency ratio of AI inference tasks.

[0067] As a refinement of the above embodiment, the load prediction algorithm of the intelligent power consumption management module adopts an LSTM recurrent neural network model, the prediction period is the next decoding period, the dynamically adjustable voltage range is 0.8V to 1.2V, and the clock frequency range is 200MHz to 1GHz.

[0068] Long Short-Term Memory (LSTM) is suitable for processing and predicting important events with very long intervals and delays in time series. The performance of LSTM is usually better than that of time recurrent neural networks and Hidden Markov Models (HMMs), such as in continuous unsegmented handwritten recognition. As a non-linear model, LSTM can be used as a complex non-linear unit to construct larger deep neural networks. The LSTM prediction model combines dynamic voltage domain regulation to achieve real-time non-linear control of power consumption and improve the overall energy efficiency ratio.

[0069] As a refinement of the above embodiment, the plug-in decoding module uses a dedicated ASIC chip, supports hardware decoding of mainstream formats such as H.264 / H.265 and AV1, has a built-in heat sink, is aligned with the motherboard air duct, and realizes the initial low-power mode through hierarchical power supply.

[0070] As a refinement of the above embodiment, the system adopts zero-buffer synchronization technology to achieve audio-visual synchronous rendering based on the timestamp matching of audio-visual output triggered by hardware interrupts;

[0071] It also adopts an intelligent sleep mechanism to control the plug-in decoding module to enter the deep sleep mode when there is no valid data input for a continuous preset number of frames.

[0072] As a refinement of the above embodiment, the task scheduler switches to the low-bitrate decoding mode during network fluctuations to reduce buffer delay.

[0073] It should be noted that:

[0074] The present invention predicts the load through an LSTM neural network, combines dynamic voltage regulation and frequency conversion technology to reduce decoding power consumption; the hierarchical power supply design reduces the startup power consumption of the module, and cooperates with the motherboard air duct heat dissipation solution to improve the energy efficiency ratio.

[0075] The modular design supports hot-swap replacement, and realizes dynamic bandwidth allocation through interfaces; the task scheduler supports heterogeneous computing resource pooling, can be extended to connect multiple decoding modules, and improves the parallel processing ability.

[0076] The zero-buffer synchronization technology calibrates through hardware timestamps to eliminate audio-visual synchronization delay.

[0077] This solution realizes the improvement of energy efficiency ratio and the reduction of system upgrade cost through the deep integration of hardware reconstruction and intelligent algorithms, providing a breakthrough decoding solution for the next-generation intelligent terminals, XR devices, and streaming media systems.

[0078] Embodiment 2

[0079] As Figure 2 and Figure 3 shown, a low-power audio and video decoding method based on plug-in products includes the following steps:

[0080] S1: The audio and video bitstream is input into the system, the device is initialized, the main control unit loads the dynamic power management driver and the task scheduler, and detects and activates the plug-in decoding module.

[0081] The audio and video bitstream input ports include USB, Tuner RF input port, Ethernet, and other audio and video input ports.

[0082] S2: The input audio and video stream is parsed by the main control unit for metadata (encoding format, resolution, etc.), and decoding tasks are assigned according to the encoding format and resolution for decoding.

[0083] S3: Predict the load through the LSTM model and dynamically adjust the voltage and frequency of the plug-in decoding module.

[0084] S4: Synchronize the audio and video output using the hardware timestamp and adjust the bitrate in real time to cope with network fluctuations.

[0085] Specifically, synchronize the audio and video output using the hardware timestamp and jump to the rendering engine

[0086] S5: Trigger the sleep instruction when there is no data input and turn off the power supply of the plug-in decoding module.

[0087] As a refinement of the above embodiment, the main control unit integrates the dynamic power management driver (DPMDriver) and the task scheduler. After inserting the decoding module, the main control unit detects the external decoding module (such as an AI decoding card) through the expansion interface and loads the corresponding driver. The power management module starts hierarchical power supply and initializes to the low-power mode.

[0088] In step S2, if it is a low-complexity task, the decoding task is preferentially assigned to the plug-in decoding module; if it is a high-complexity task, it is processed by the built-in decoding unit of the main control unit. The low-complexity tasks include audio decoding tasks, video decoding tasks of non-new encoding standards, or video decoding tasks with a resolution less than 1080P. The high-complexity tasks include video decoding tasks of new encoding standards and with a resolution greater than or equal to 1080P.

[0089] It should be noted that:

[0090] Through dynamic power management, modular hardware design, and intelligent task offloading technologies, the present invention significantly improves the energy efficiency ratio and compatibility of plug-in audio and video devices. Compared with traditional solutions, the power consumption is reduced by at least 30% or more, the multi-format decoding response time is shortened by at least 30%, and it supports plug-and-play expansion functions. This technology can be widely applied in fields such as smart set-top boxes and portable media terminals, promoting the industrial development of low-power audio and video processing technologies.

[0091] Combined with Figure 4 As shown, an embodiment of the present disclosure provides a low-power audio and video decoding device 300 based on a plug-in product, including a processor 304 and a memory 301. Optionally, the device may further include a communication interface 302 and a bus 303. Among them, the processor 304, the communication interface 302, and the memory 301 can communicate with each other through the bus 303. The communication interface 302 can be used for information transmission. The processor 304 can call the logical instructions in the memory 301 to execute the low-power audio and video decoding method based on the plug-in product in the above embodiment, specifically including:

[0092] An audio and video code stream input system, device initialization, the main control unit loads a dynamic power management driver and a task scheduler, and detects and activates a plug-in decoding module.

[0093] The input audio and video stream is parsed for metadata (encoding format, resolution, etc.) by the main control unit, and decoding tasks are allocated according to the encoding format and resolution for decoding.

[0094] Predict the load through an LSTM model and dynamically adjust the voltage and frequency of the plug-in decoding module.

[0095] Synchronize the audio and video output using a hardware timestamp and adjust the bit rate in real time to cope with network fluctuations.

[0096] For low-complexity tasks, the decoding tasks are preferentially allocated to the plug-in decoding module; for high-complexity tasks, they are processed by the built-in decoding unit of the main control unit. The low-complexity tasks include audio decoding tasks, video decoding tasks for non-new encoding standards, or video decoding tasks with a resolution less than 1080P, and the high-complexity tasks include video decoding tasks for new encoding standards and with a resolution greater than or equal to 1080P.

[0097] In addition, when the logical instructions in the above-mentioned memory 301 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium.

[0098] The memory 301, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of the present disclosure. The processor 304 executes functional applications and data processing by running the program instructions / modules stored in the memory 301, that is, implements the low-power audio-video decoding method based on the plug-in product in the above embodiments.

[0099] The memory 301 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 301 may include high-speed random access memory and may also include non-volatile memory.

[0100] The embodiments of the present disclosure provide a computer-readable storage medium storing computer-executable instructions, and the computer-executable instructions are configured to execute the above-mentioned low-power audio-video decoding method based on the plug-in product.

[0101] The above-mentioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transient computer-readable storage medium.

[0102] The technical solution of the embodiments of the present disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present disclosure. The foregoing storage medium may be a non-transient storage medium, including: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, or may also be a transient storage medium.

[0103] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure, enabling those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process, and other changes. The embodiments merely represent possible variations. Unless explicitly required, the individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terms used in this application are only for describing the embodiments and do not limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to also include the plural forms. Similarly, as used in this application, the term "and / or" refers to any and all possible combinations of one or more of the associated listed items. Additionally, when used in this application, the term "comprise" and its variants "comprises" and / or "comprising" etc. mean the presence of the stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, or apparatus comprising the element. Herein, each embodiment may focus on the differences from other embodiments, and the same or similar parts among the embodiments may be referred to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method parts disclosed in the embodiments, the relevant parts may refer to the description of the method parts.

[0104] Those skilled in the art will realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner may depend on the specific application and design constraints of the technical solution. The skilled person may use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the embodiments of the present disclosure. The skilled person can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0105] In the embodiments disclosed in this article, the disclosed methods, products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units can be merely a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Additionally, the couplings or direct couplings or communication connections shown or discussed between each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to implement this embodiment. Additionally, in the embodiments of this disclosure, each functional unit can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit.

[0106] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to the embodiments of this disclosure. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the blocks can occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks can also occur in a different order than that disclosed in the description. Sometimes, there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, which can depend on the functions involved. Each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A low-power audio and video decoding system based on plug-in products, characterized in that: include: The main control unit integrates a multi-core processor and a task scheduler to allocate decoding tasks in real time and support dynamic frequency adjustment; The plug-in decoding module is connected to the main control unit through an expansion interface. It contains a dedicated ASIC chip, supports hard decoding of H.264, H.265, and AV1 formats, and can be upgraded and replaced independently of the main control unit; An intelligent power consumption management module, connected to the main control unit and the plug-in decoding module, including a load prediction algorithm and an adaptive voltage regulation unit, for dynamically adjusting the operating voltage and clock frequency of the decoding module according to the task load; The power management module provides power to the main control unit, the plug-in decoding module and the intelligent power consumption management module.

2. The low-power audio and video decoding system based on plug-in products according to claim 1, characterized in that: The main control unit includes a multi-core processor, which integrates a CPU, a GPU and an NPU, and allocates high-complexity tasks to a plug-in decoding module and low-complexity tasks to a decoding unit built into the main control unit through a task scheduler.

3. The low-power audio and video decoding system based on plug-in products according to claim 1, characterized in that: The load prediction algorithm of the intelligent power consumption management module adopts an LSTM recursive neural network model, the prediction period is the next decoding period, the dynamic adjustment voltage range is 0.8V to 1.2V, and the clock frequency range is 200MHz to 1GHz.

4. The low-power audio and video decoding system based on plug-in products according to claim 1, characterized in that: The plug-in decoding module has a built-in heat sink that is aligned with the mainboard air duct and achieves an initial low power consumption mode through graded power supply.

5. The low-power audio and video decoding system based on plug-in products according to claim 1, characterized in that: The system uses zero-buffer synchronization technology to achieve audio and video synchronous rendering based on timestamp matching of audio and video output triggered by hardware interrupts; It also adopts an intelligent sleep mechanism, which controls the plug-in decoding module to enter a deep sleep mode when there is no valid data input for a preset number of consecutive frames.

6. The low-power audio and video decoding system based on plug-in products according to claim 1, characterized in that: The task scheduler switches to a low bit rate decoding mode when the network fluctuates to reduce buffering delay.

7. A low-power audio and video decoding method based on a plug-in product based on the system of any one of claims 1-6, characterized in that: The following steps are involved: Audio and video code streams are input into the system, the device is initialized, the main control unit loads the dynamic power management driver and task scheduler, and the plug-in decoding module is detected and activated; The input audio and video streams are parsed through the main control unit to parse metadata and decode them according to the encoding format and resolution. The LSTM model is used to predict the load and dynamically adjust the voltage and frequency of the plug-in decoding module. Use hardware timestamps to synchronize audio and video output, and adjust the bit rate in real time to cope with network fluctuations; When there is no data input, the sleep command is triggered and the power supply of the plug-in decoding module is turned off.

8. The method according to claim 7, characterized in that The decoding method for allocating decoding tasks according to the encoding format and resolution is as follows: if it is a low-complexity task, the decoding task is preferentially allocated to the plug-in decoding module; if it is a high-complexity task, it is processed by the built-in decoding unit of the main control unit. The low-complexity task includes audio decoding tasks, video decoding tasks that are not new encoding standards, or video decoding tasks with a resolution less than 1080P, and the high-complexity task includes video decoding tasks that are new encoding standards and have a resolution greater than or equal to 1080P.

9. A low-power audio and video decoding device based on a plug-in product, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the low-power audio and video decoding method based on a plug-in product as described in any one of claims 6-7 when running the program instructions.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the low-power audio and video decoding method based on a plug-in product as described in any one of claims 6-7 above is implemented.

Images