Multimedia task processing method and device, computer equipment, chip and chip module

Abstract

The invention relates to a multimedia task processing method and device, computer equipment, a chip and a chip module, relates to the technical field of systems on chip, and can give consideration to resource utilization efficiency and function expansion flexibility when processing multimedia tasks. The method comprises the following steps: determining a first function module related to a multimedia task from a plurality of function modules of a chip according to a task intention of the multimedia task to be processed; in response to the fact that the first function module comprises a hardware function module and a programmable module, configuring the programmable module according to a pre-acquired software program to obtain a plurality of second function modules; the plurality of second function modules comprise a hardware function module and a configured programmable module; and reconstructing the interconnection paths of the plurality of second function modules, and transmitting data streams of the second function modules among the plurality of second function modules according to the reconstructed first interconnection paths in the process of executing the multimedia task by using the plurality of second function modules.

Description

Technical Field

[0001] This application relates to the field of system-on-a-chip technology, and in particular to a multimedia task processing method, apparatus, computer equipment, chip, chip module, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of computer technology, a System on a Chip (SoC) can include multiple basic units that make up the SoC. Each basic unit can be called an IP core and has corresponding fixed functions. An SoC can use multiple IP cores to implement various multimedia tasks, such as camera, video, and display processing. In related technologies, each IP core integrates corresponding modules internally to achieve its specific functions and handle related tasks, such as scaling, color conversion, and buffer handling modules.

[0003] However, different IP cores often integrate the same modules repeatedly, leading to wasted chip area and power consumption. Furthermore, when new types of multimedia tasks need to be processed, existing IP cores may become unusable, requiring significant resources to be spent on redesigning and updating them. Therefore, it is evident that related technologies for multimedia task-oriented systems struggle to balance resource utilization efficiency with functional expansion flexibility. Summary of the Invention

[0004] Therefore, it is necessary to provide a multimedia task processing method, apparatus, computer equipment, chip, chip module, computer-readable storage medium, and computer program product to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides a multimedia task processing method, including:

[0006] Based on the task intent of the multimedia task to be processed, a first functional module related to the multimedia task is determined from multiple functional modules of the chip; the multiple functional modules include functional modules reused in different multimedia task processing processes;

[0007] In response to the first functional module including a hardware functional module and a programmable module, the programmable module is configured according to a pre-acquired software program to obtain a plurality of second functional modules; the plurality of second functional modules include the hardware functional module and the configured programmable module;

[0008] The interconnection paths of the plurality of second functional modules are reconstructed, and during the execution of the multimedia task using the plurality of second functional modules, the data streams of the second functional modules are transmitted between the plurality of second functional modules according to the first interconnection path obtained after reconstruction.

[0009] In one embodiment, reconstructing the interconnection paths of the plurality of second functional modules includes: determining multiple candidate paths for the plurality of second functional modules on the chip; for each candidate path, determining the number of path hops based on the number of nodes included in the candidate path; and filtering the plurality of candidate paths based on the number of path hops of each candidate path to determine a first interconnection path.

[0010] In one embodiment, the step of filtering the plurality of candidate paths based on the path hop count of each candidate path to determine a first interconnection path includes: determining the congestion level of each candidate path; determining the first interconnection path of the plurality of second functional modules from the plurality of candidate paths based on the path hop count and the congestion level; wherein the path hop count and the congestion level of the first interconnection path satisfy preset filtering conditions.

[0011] In one embodiment, after reconstructing the interconnect paths of the plurality of second functional modules, the method further includes: in response to a triggering event for updating the interconnect paths, reconstructing the interconnect paths of the plurality of second functional modules again to obtain a second interconnect path of the chip; writing the second interconnect path into a second region in a register of the chip; the register further includes a first region, the second region being used to store the first interconnect path used by the plurality of second functional modules when processing the current multimedia task image frame; after transmitting the data stream of the second functional modules among the plurality of second functional modules according to the first interconnect path obtained after reconstruction, the method further includes: after the current multimedia task image frame processing is completed, transmitting the subsequent data stream of the second functional modules among the plurality of second functional modules according to the second interconnect path written in the second region.

[0012] In one embodiment, the registers include registers associated with each of the plurality of second functional modules; the step of transmitting subsequent data streams of the second functional modules among the plurality of second functional modules according to the second interconnection path written in the second region after the processing of the current multimedia task image frame includes: in response to the arrival of the frame boundary between the current multimedia task image frame and the next multimedia task image frame, switching the interconnection path reading region from the first region of each of the registers to each of the second regions, so as to transmit subsequent data streams of the second functional modules among the plurality of second functional modules according to the second interconnection path stored in each of the second regions.

[0013] In one embodiment, configuring the programmable module according to a pre-acquired software program includes: obtaining a task descriptor according to the task intent; the task descriptor representing a module task to be executed by the programmable module; adding the task descriptor to a task queue corresponding to the programmable module, so that after the programmable module reads the task descriptor from the task queue, a target software program associated with the module task is determined from a plurality of candidate software programs based on the task descriptor; and configuring the programmable module according to the target software program.

[0014] Secondly, this application also provides a multimedia task processing apparatus, comprising:

[0015] A filtering module is used to determine a first functional module related to the multimedia task from multiple functional modules of the chip according to the task intent of the multimedia task to be processed; the multiple functional modules include functional modules reused in different multimedia task processing processes.

[0016] A configuration module is configured to, in response to the first functional module including a hardware functional module and a programmable module, configure the programmable module according to a pre-acquired software program to obtain a plurality of second functional modules; the plurality of second functional modules include the hardware functional module and the configured programmable module.

[0017] The path reconstruction module is used to reconstruct the interconnection path of the plurality of second functional modules, and during the execution of the multimedia task using the plurality of second functional modules, transmit the data stream of the second functional modules between the plurality of second functional modules according to the first interconnection path obtained after reconstruction.

[0018] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method provided in the first aspect above.

[0019] Fourthly, this application also provides a chip including a processor and a communication interface, the processor being configured to cause the chip to perform the steps of the method provided in the first aspect above.

[0020] Fifthly, this application also provides a chip module, including a communication module, a power module, a storage module, and a chip, wherein:

[0021] The power module is used to provide power to the chip module;

[0022] The storage module is used to store data and instructions;

[0023] The communication module is used for internal communication within the chip module, or for communication between the chip module and external devices.

[0024] The chip is used to perform the steps of the method provided in the first aspect above.

[0025] In a sixth aspect, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect above.

[0026] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method provided in the first aspect above.

[0027] The aforementioned multimedia task processing method, apparatus, computer equipment, chip, chip module, computer-readable storage medium, and computer program product can determine a first functional module related to the multimedia task from multiple functional modules of the chip according to the task intent of the multimedia task to be processed. These multiple functional modules include functional modules reused in different multimedia task processing processes. Then, in response to the first functional module including a hardware functional module and a programmable module, the programmable module can be configured according to a pre-acquired software program to obtain multiple second functional modules, each including a hardware functional module and a configured programmable module. Furthermore, the interconnection paths of the multiple second functional modules can be reconstructed, and during the execution of the multimedia task using the multiple second functional modules, data streams of the second functional modules are transmitted between the multiple second functional modules according to the reconstructed first interconnection paths. In this embodiment, on the one hand, a first functional module related to multimedia task processing is determined from multiple reusable functional modules in the chip. Multimedia task processing is then performed subsequently based on the reconfigured interconnect path reconstruction results of the second functional module. Through a reconfigurable interconnect network, a multimedia processing pipeline that can be combined according to task needs is formed. This helps reduce redundant hardware in the chip through reusable modular functions and reconfigurable interconnects, avoiding waste of area and power consumption. On the other hand, this embodiment can provide a second functional module that simultaneously includes hardware functional modules and programmable modules. This allows for improved multimedia task processing efficiency through hardware functional modules, and, given a limited number of hardware functional modules, adaptability to constantly changing multimedia processing requirements through programmable modules. Therefore, this embodiment balances chip resource utilization efficiency and functional expansion flexibility. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating a multimedia task processing method in one embodiment;

[0030] Figure 2 This is a schematic diagram of a process for forwarding data streams between second functional modules in one embodiment;

[0031] Figure 3 This is a schematic diagram of a process for updating an interconnect path in one embodiment;

[0032] Figure 4 This is a schematic diagram of a process for switching interconnect paths at frame boundaries in one embodiment;

[0033] Figure 5 This is a comparison diagram of a second functional module and its interconnection path before and after adjustment in one embodiment;

[0034] Figure 6 This is a flowchart illustrating the configuration of a programmable module in one embodiment;

[0035] Figure 7 This is a flowchart illustrating another multimedia task processing method in one embodiment;

[0036] Figure 8 This is a schematic diagram of the framework of a multimedia task processing method in one embodiment;

[0037] Figure 9 This is a structural block diagram of a multimedia task processing device in one embodiment;

[0038] Figure 10 This is an internal structural diagram of a computer device in one embodiment;

[0039] Figure 11 This is an internal structure diagram of a chip module in one embodiment. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various objects, but these elements are not limited by these terms. These terms are only used to distinguish the first object from the second object. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0042] In one embodiment, such as Figure 1 As shown, a multimedia task processing method is provided. This method can be applied to a chip, and it is understood that it can also be applied to a terminal or server, and further applied to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0043] S101, based on the task intent of the multimedia task to be processed, determine the first functional module related to the multimedia task from multiple functional modules of the chip; the multiple functional modules include functional modules reused in different multimedia task processing processes.

[0044] The task intent, also known as intent description, refers to the processing goal of a multimedia task. It can be understood as an abstract description of the goals and constraints of a multimedia task. For example, the intent of a video decoding task could be to convert a compressed video data stream into displayable image frames. In an exemplary embodiment, the task intent may include one or more of the following information constraints: input / output format, resolution, frame rate, algorithm intent, power consumption, latency, and bandwidth.

[0045] In practice, the multimedia task to be processed can be obtained and the task intent of the multimedia task can be determined. For example, by analyzing the multimedia task, it can be determined whether the multimedia task is to perform multimedia data encoding, decoding and playback, or to perform image rendering.

[0046] On the other hand, multiple functional modules can be pre-provided on the chip. These functional modules can be designed to meet the processing needs of different types of multimedia tasks. Among these multiple functional modules are reusable functional modules that can be used in different multimedia task processing. Reusable functional modules can be understood as modules that play similar processing roles in different multimedia tasks. For example, multimedia processing capabilities can be decomposed into multiple reusable modules, thereby obtaining multiple pre-configured functional modules on the chip, such as scalers, splitters, color space converters (CSC), memory access / format transfer modules (DMA / MAU), and general multimedia processing units (UMPU), etc.

[0047] Furthermore, after clarifying the task intent of the multimedia task to be processed, a first functional module related to the multimedia task can be determined from the multiple functional modules contained in the chip according to the task intent. The first functional module can be a set of chip functional modules required to complete the task intent corresponding to the multimedia task.

[0048] In a specific embodiment, when the task intent of the multimedia task to be processed is determined to be image enhancement processing, the functions required to complete the image enhancement processing are analyzed based on the task intent, and then the first functional module related to image enhancement processing is selected from the multiple functional modules of the chip, such as the image data reading module, the pixel value adjustment module, and the enhanced image output module.

[0049] S102, in response to the first functional module including a hardware functional module and a programmable module, the programmable module is configured according to the pre-acquired software program to obtain a plurality of second functional modules; the plurality of second functional modules include the hardware functional module and the configured programmable module.

[0050] Among them, hardware functional modules can refer to modules whose functions are fixed through hardware circuits. In some examples, hardware functional modules can be used directly without software configuration. Programmable modules can refer to modules whose functions can be changed by loading software programs, such as field-programmable gate arrays (FPGAs) and digital signal processors (DSPs).

[0051] The pre-acquired software program may refer to the program code stored in the chip firmware library that corresponds to a specific function. For example, it may include one or more programs such as color space conversion program and audio filtering program.

[0052] The second functional module refers to the set of functional modules that can directly participate in task execution after configuration, including hardware functional modules and configured programmable modules.

[0053] To improve the processing efficiency of multimedia tasks, some related technologies have proposed FPGA-based reconfigurable multimedia processing. FPGA (Field-Programmable Gate Array) is a programmable hardware device that can be dynamically configured as needed, making it suitable for applications requiring high flexibility and adaptability to constantly changing demands, such as multimedia processing. While this approach offers high flexibility, it suffers from high power consumption and a large footprint. Other technologies have proposed software-defined multimedia processing, which uses software to implement multimedia processing functions. However, while software-defined multimedia processing improves flexibility, its processing efficiency is lower compared to hardware processing. Furthermore, some technologies have proposed using heterogeneous computing for multimedia task processing. Heterogeneous computing uses different types of processing units (such as CPUs, GPUs, and dedicated accelerators) to execute tasks. By using different processing units, optimizations can be made for specific tasks, thereby improving the efficiency and performance of multimedia processing. However, this approach involves programming complexity and requires coordinating different types of processing units.

[0054] In this embodiment, the chip provides multiple reusable functional modules, including both reusable hardware functional modules and programmable modules. In some possible embodiments, after determining the task intent, it can first be determined whether the hardware functional modules among the multiple reusable functional modules in the chip can satisfy the task intent. If so, each hardware functional module that satisfies the task intent can be identified as a first functional module, and the multimedia task can be directly processed using each first functional module; otherwise, the first functional module can be obtained based on the multiple hardware functional modules and programmable modules related to the task intent.

[0055] Given that the first functional module includes a hardware functional module and a programmable module, the programmable module can be configured according to the task intent and the pre-acquired software program. Thus, multiple second functional modules can be obtained based on the hardware functional module and the configured programmable module.

[0056] Related technologies suffer from module redundancy and poor flexibility. Specifically, different IP cores in a chip may contain duplicate modules, such as scalers, splitters, memory accessors, and color converters, leading to wasted chip area and power consumption. Furthermore, fixed-function IP cores cannot adapt to constantly changing multimedia processing requirements. To address this, this embodiment defines a second functional module that includes both hardware functional modules and programmable modules. This allows for modular design and the use of universal programmable modules, eliminating redundant modules and reducing chip area and power consumption. Moreover, by employing a reconfigurable multimedia SoC architecture, multiple second functional modules and multimedia processing pipelines can be dynamically configured as needed, flexibly adapting to evolving requirements.

[0057] S103, the interconnection path of multiple second functional modules is reconstructed, and during the execution of multimedia tasks using multiple second functional modules, the data stream of the second functional modules is transmitted between multiple second functional modules according to the first interconnection path obtained after reconstruction.

[0058] Interconnection paths refer to the data transmission or connection paths formed between various functional modules within a chip via interconnection structures such as buses and cross switches. Interconnection paths define the flow of data between modules. The first interconnection path is the result of interconnection path reconstruction. It can be obtained by analyzing and optimizing the possible connection relationships between second functional modules based on a preset algorithm. It reflects the suitable or optimal interconnection path when multiple second functional modules interact with data, improving the stability and efficiency of subsequent data stream transmission. Data streams, also known as logical data streams, can be understood as the data input, processed, and output by each module during the execution of a multimedia task, such as one or more types of H.265 bitstreams, YUV image data, and RGB image data.

[0059] In a specific implementation, after determining multiple second functional modules, the interconnection paths between these modules can be reconstructed to obtain a first interconnection path. In some embodiments, path reconstruction can be performed each time a new multimedia task is executed, or it can be performed during the execution of a multimedia task when a triggering event for updating the interconnection path is detected.

[0060] Then, data streams can be transmitted between the second functional modules according to the first interconnection path. For example, when a second functional module completes the A processing of a video frame, the data stream a generated by that module will be automatically transmitted to another second functional module responsible for audio data encoding according to the reconstructed interconnection path.

[0061] In an exemplary embodiment, a physical configuration object (PCO) can be compiled based on the task intent. The PCO may contain information such as module instance selection, module parameters, buffer planning, NoC routing and flow configuration. When generating the PCO, the resource and route manager (RRM) can convert the task intent into a PCO and complete the path planning. In a specific implementation, the RRM can run on the CPU driver, the firmware, or the UMPU management thread.

[0062] In one exemplary embodiment, such as Figure 2 As shown, the router node and network interface (NIU) can determine the FLIT (Flow Control Unit) / packet input to the second functional module and its corresponding data flow identifier flow_id. The FLIT (Flow Control Unit) is the smallest data unit used for flow control in the network-on-chip (NoC), dividing data into fixed-size blocks to improve transmission efficiency and reduce latency. Subsequently, the router corresponding to the second functional module looks up a table, specifically the FlowTable in the ready-active region of the register. Based on the lookup result, it selects the next_port and outputs the next-hop data to other second functional modules according to the selected next_port.

[0063] In the above-described multimedia task processing method, a first functional module related to the multimedia task can be determined from multiple functional modules of the chip according to the task intent of the multimedia task to be processed. The multiple functional modules include functional modules reused in different multimedia task processing processes. Then, in response to the first functional module including a hardware functional module and a programmable module, the programmable module can be configured according to a pre-acquired software program to obtain multiple second functional modules, which include the hardware functional module and the configured programmable module. Furthermore, the interconnection path of the multiple second functional modules can be reconstructed, and during the execution of the multimedia task using the multiple second functional modules, the data stream of the second functional modules is transmitted between the multiple second functional modules according to the first interconnection path obtained after reconstruction. In this embodiment, on the one hand, a first functional module related to multimedia task processing is determined from multiple reusable functional modules in the chip. Multimedia task processing is then performed subsequently based on the reconfigured interconnect path reconstruction results of the second functional module. Through a reconfigurable interconnect network, a multimedia processing pipeline that can be combined according to task needs is formed. This helps reduce redundant hardware in the chip through reusable modular functions and reconfigurable interconnects, avoiding waste of area and power consumption. On the other hand, this embodiment can provide a second functional module that simultaneously includes hardware functional modules and programmable modules. This allows for improved multimedia task processing efficiency through hardware functional modules, and, given a limited number of hardware functional modules, adaptability to constantly changing multimedia processing requirements through programmable modules. Therefore, this embodiment balances chip resource utilization efficiency and functional expansion flexibility.

[0064] In one embodiment, reconstructing the interconnection paths of multiple second functional modules in step S103 may include the following steps:

[0065] Multiple candidate paths for multiple second functional modules on the chip are identified; for each candidate path, the number of path hops is determined based on the number of nodes included in the candidate path; based on the number of path hops for each candidate path, the multiple candidate paths are filtered to determine the first interconnect path.

[0066] Candidate paths can be hardware connection paths between two or more second functional modules that enable data interaction. Nodes can refer to hardware switching units on the candidate paths, such as signal repeaters, data exchange nodes, and level conversion nodes. The path hop count refers to the number of nodes in the candidate path, which can characterize the complexity of the path and its transmission delay-related characteristics. The interconnect path reconstruction result is the final connection path selected from multiple candidate paths that meets the chip's functional requirements.

[0067] In practical applications, potential connections between multiple secondary functional modules can be identified, and multiple candidate paths between these modules can be determined, such as identifying all candidate paths between multiple secondary functional modules. Furthermore, the number of hops for each path can be quantified by the number of nodes, and suitable interconnection paths can be selected based on the number of hops, achieving reasonable reconfiguration of interconnection paths and thus optimizing the data transmission efficiency and stability of the chip.

[0068] Specifically, multiple candidate paths for multiple second functional modules on the chip can be determined. In some exemplary embodiments, all hardware links that can physically connect the second functional modules can be traversed. For example, the physical interface location and the range of connectable hardware links for each second functional module can be obtained through the chip's layout and routing file or a dedicated chip path analysis tool, thereby identifying all potential connection paths between the second functional modules and obtaining multiple candidate paths.

[0069] Then, the hop count of a candidate path can be determined based on the number of nodes it contains. In some examples, the hop count is equal to the number of nodes in the candidate path; if the path has no nodes, the hop count is 0. In practice, the number of nodes can affect the signal transmission delay and failure risk of the path. As the number of nodes increases, the number of transfers the signal undergoes during transmission also increases accordingly. Therefore, the hop count can be determined by the number of nodes, which intuitively reflects the path characteristics.

[0070] Furthermore, the first interconnect path that meets the chip's functional requirements can be selected based on the path hop count. For example, a path with fewer hops can be chosen to reduce transmission delay and failure risk; for instance, the minimum hop count candidate path can be calculated based on topology and module connection requirements. Alternatively, the path hop count can be combined with other path characteristics to assist in selecting suitable candidate paths; for example, one or more information such as the path's signal transmission rate, candidate path topology, and hardware cost can be used for selection. The selected candidate paths can then be used as the interconnect path reconstruction result.

[0071] Compared to related technologies where data transmission between different IP cores of a chip requires multiple intermediate steps, resulting in low transmission efficiency and increased latency, this embodiment determines the first interconnect path based on the path hop count and performs data stream transmission between the second functional modules based on the first interconnect path. This allows data to be transmitted directly between functional modules, which can improve transmission efficiency and reduce latency.

[0072] In one embodiment, filtering multiple candidate paths based on the path hop count of each candidate path to determine a first interconnection path may include the following steps:

[0073] Determine the congestion level of each candidate path; based on the hop count and congestion level of each path, determine the first interconnection path of multiple second functional modules from multiple candidate paths; the hop count and congestion level of the first interconnection path meet the preset screening conditions.

[0074] Here, congestion level can be understood as the blocking situation of data transmission. In some examples, a congestion counter can be provided for cost evaluation of path selection. The link utilization of candidate paths is determined by the congestion counter. The link utilization is positively correlated with the congestion level of candidate paths, that is, the higher the link utilization, the higher the congestion level.

[0075] Among them, the preset screening conditions can refer to the standards set in advance according to the chip design requirements to determine whether the candidate path meets the usage requirements. These can include the maximum threshold for the number of path hops, the maximum threshold for the degree of congestion, or a comprehensive judgment rule combining both.

[0076] In practical applications, the congestion level of each candidate path can be determined. Then, based on the hop count and congestion level of each path, the recommendation level of each candidate path can be determined. Furthermore, based on the recommendation level of each candidate path, interconnection paths that meet preset conditions can be selected. For example, the resource and routing manager can perform a shortest path search on each candidate path based on the interconnection topology map. During this search, the path hop count can be used as the base cost, and congestion weights can be added to the paths to obtain the interconnection path from the source module (i.e., the second functional module as the starting point) to the destination module (i.e., the second functional module as the ending point). This interconnection path can consist of a sequence of routing nodes and / or a sequence of ports.

[0077] In this embodiment, by determining the first interconnection path of multiple second functional modules from multiple candidate paths based on the hop count and congestion level of each path, in addition to realizing the dynamic reconstruction of the connection between modules according to the functional module call situation of different multimedia tasks, the performance of candidate paths can be evaluated more comprehensively, so that the selected interconnection path meets both the transmission delay requirements and the signal stability requirements, effectively improving the overall working reliability of the chip when processing multimedia tasks.

[0078] In one embodiment, after reconstructing the interconnect paths of the plurality of second functional modules, the following steps may be further included: in response to the occurrence of a trigger event for updating the interconnect paths, reconstructing the interconnect paths of the plurality of second functional modules again to obtain the second interconnect paths of the chip; writing the second interconnect paths into a second region in a register of the chip; the register further includes a first region, and the second region is used to store the first interconnect paths used by the plurality of second functional modules when processing the current multimedia task image frame.

[0079] Accordingly, after transmitting the data streams of the second functional modules among the multiple second functional modules according to the first interconnection path obtained after reconstruction, the following steps may also be included:

[0080] After the current multimedia task image frame processing is completed, the subsequent data stream of the second functional modules is transmitted between multiple second functional modules according to the second interconnection path written in the second region.

[0081] The triggering event for updating the interconnect path can refer to specific conditions or signals that can trigger the interconnect path update. For example, it can include one or more of the following: receiving a new multimedia task, switching the multimedia task type (e.g., switching from image processing to video processing), excessive load on the second functional module, detection of a newly added candidate path, input of external control commands, etc., for example... Figure 3 As shown, the triggering condition can be determined when any of the following conditions are detected: a new task intent is found, the congestion level reaches a threshold, or a new branch is added. The multimedia task image frame can be an image frame related to the multimedia task.

[0082] Registers are high-speed storage units in a chip used for temporary data storage. In this embodiment, the storage area of ​​a register may include at least two areas (dual banks), namely a first area and a second area. The first area and the second area are physically or logically divided independent storage spaces, which are used to store the updated interconnect path (i.e., the second interconnect path) and the interconnect path used by the current processing task (i.e., the first interconnect path), respectively. For ease of distinction, the area storing the second interconnect path is called the second area (shadow bank), and the area storing the first interconnect path is called the first area (active bank).

[0083] In a specific implementation, after obtaining the first interconnection path, the first interconnection path can be written into the first area that is in a ready and usable state, and then data streams can be transmitted between multiple second functional modules according to the first interconnection path.

[0084] When a trigger event is detected, the interconnection path can be updated to obtain a second interconnection path between the second functional modules. Then, the second interconnection path can be written into the second region to avoid interfering with the currently processed multimedia task image frame. For the currently processed multimedia task image frame, it can continue to be processed according to the previously obtained first interconnection path.

[0085] Once the processing of the current multimedia task image frame is completed, the subsequent data stream of the second functional modules can be transmitted between multiple second functional modules according to the second interconnection path written in the second region.

[0086] In this embodiment, by storing the updated second interconnect path in the second region of the register and storing the currently used first interconnect path in the first region, independent storage of the old and new interconnect paths is achieved, avoiding interference of path update operations on the current frame processing. At the same time, switching to the updated second interconnect path after the current frame processing is completed ensures the integrity of image data processing for each frame and effectively avoids problems such as image tearing and stuttering.

[0087] In one embodiment, the registers include registers associated with each of the multiple second functional modules. Specifically, each second functional module can be associated with a separate register, and each register associated with a second functional module includes two parts: a first region and a second region. The first region stores the first interconnection path used when processing the current multimedia task image frame, and the second region stores the second interconnection path to be used by the next multimedia task image frame.

[0088] Accordingly, after the current multimedia task image frame processing is completed, the subsequent data streams of the second functional modules are transmitted between multiple second functional modules according to the second interconnection path written in the second region, which may include the following steps:

[0089] In response to the arrival of the frame boundary between the current multimedia task image frame and the next multimedia task image frame, the interconnect path reading area is switched from the first area of ​​each register to the second area, so as to transmit the subsequent data stream of the second functional modules between multiple second functional modules according to the second interconnect path stored in each second area.

[0090] Among them, the frame boundary can be understood as the time node when the current multimedia task image frame processing is completed and the next multimedia task image frame processing is about to begin. It can be identified by signals such as the hardware counter reaching the single frame processing cycle threshold, the row boundary of the last row of pixels in the image frame, or the data buffer being empty. In addition, in some other possible examples, the switching of the interconnect path reading area can also be triggered when the buffer content (buffer level) meets the preset conditions.

[0091] In a specific implementation, after obtaining the second interconnect path, the second interconnect path can be written into the register associated with each second functional module. For each register associated with a second functional module, the second interconnect path can be written into the second area of ​​the register. At the same time, the first area of ​​the register stores the first interconnect path.

[0092] Then, it can be identified whether the current multimedia task image frame has been processed. In this step, when the EOF (End of Frame) marker is detected, the frame boundary between the current multimedia task image frame and the next multimedia task image frame can be determined. This allows for a safe point where the current multimedia task image frame has been processed and the next multimedia task image frame has begun processing, such as... Figure 4 As shown, an atomic switch is performed, controlling the interconnect path read area of ​​each second functional module to switch from the first area in the associated register to the second area, thereby making the new interconnect path configuration atomically effective. In some exemplary embodiments, a configuration switch unit (CSU) may be provided, which can write the PCO to the module register and the NoC flow table, supporting shadow / active dual banks and atomic switching at frame boundaries.

[0093] After the switch, the state of the first region changes from ready / active to waiting / shadow, where it can wait for the updated interconnect path (such as the third interconnect path) to be written. The state of the second region changes from waiting / shadow to ready / active. Furthermore, when processing the next multimedia task image frame, the subsequent data streams of the second functional modules can be transmitted between multiple second functional modules based on the second interconnect paths stored in each second region.

[0094] For example, when switching from preview (multimedia task A) to video encoding (multimedia task B), such as... Figure 4 As shown, the pipeline in preview mode is "Camera-CSC-Scaler-Display", while the pipeline in video encoding mode is "Camera-Splitter-(A) CSC-Scaler-Display; (B) CSC-UMPU (pre-filtering / noise reduction)-Encoder-Memory". This means that different pipelines are used when processing different multimedia tasks. When switching multimedia tasks, RRM generates a new PCO. CSU first configures the required modules and routes for the Splitter and encoding branch in the shadow bank, and then commits the switch at the next frame boundary to keep the preview flowing and add new encoding branches.

[0095] For example, there are two paths from the second functional module A to the second functional module D: Path1 is 2 hops and Path2 is 3 hops. When starting to process a multimedia task, Path1 can be selected; when it is detected that the congestion of the link traversed by Path1 exceeds the threshold, RRM can write Path2 into the shadow table in the register and make it effective in the next frame, thereby avoiding congestion and maintaining stable throughput.

[0096] In this embodiment, a secure switching mechanism is provided by switching the interconnect path reading area from the first area of ​​each register to the second area when the frame boundary between the current multimedia task image frame and the next multimedia task image frame is reached. This mechanism can achieve secure switching of the interconnect path and avoid data inconsistency and tearing.

[0097] In one embodiment, configuring the programmable module according to the pre-acquired software program in step S102 may include the following steps:

[0098] Obtain a task descriptor based on the task intent; the task descriptor represents the module task to be executed by the programmable module; add the task descriptor to the task queue corresponding to the programmable module so that after the programmable module reads the task descriptor from the task queue, it can determine the target software program associated with the module task from multiple candidate software programs based on the task descriptor; configure the programmable module according to the target software program.

[0099] The programmable module can be a general-purpose multimedia processing unit (UMPU). In some exemplary embodiments, the UMPU is a programmable processing unit oriented towards multimedia operators. It can adopt a SIMD DSP-style multimedia-specific ISA optimized for pixel / block processing. For example, it can include instructions or micro-operations such as vector SIMD arithmetic, saturation / rounding, mixed precision MAC, data permutation / packing / unpacking, and 2D stride / tile memory access, thereby reducing the complex control and large cache area required by general-purpose processors.

[0100] A Task Descriptor (TD) is a structured data representation of a task's intent, containing key information such as the module task's type, parameters, and execution priority. It provides a clear basis for task execution for the programmable module. A module task is a task to be executed by the programmable module; this task is required during multimedia task processing.

[0101] A task queue is an ordered data structure allocated to a programmable module to cache descriptors of tasks to be executed. For example, a first-in-first-out (FIFO) processing mechanism can be used to avoid conflicts caused by multiple tasks requesting programmable module resources at the same time.

[0102] In a practical implementation, multiple candidate software programs can be pre-defined. In some possible embodiments, developers can use a preset programming language (such as C, C++ (optionally intrinsic / library functions)) to write or assemble multimedia software programs (kernels), thereby obtaining multiple candidate software programs. Different candidate software programs can perform different tasks, such as convolutional denoising, sharpening, color enhancement, format conversion, or one or more other processing methods. In one example, the candidate software programs, after compilation / assembly, can form binary instruction blocks that are understandable by programmable modules.

[0103] When the second functional module includes a programmable module, a task descriptor can be obtained according to the task intent. For example, the task intent can be parsed, and the parsing result can be converted into a structured task descriptor according to preset format rules.

[0104] In some specific embodiments, the host-side / CSU (such as the host or CPU) driver constructs the task descriptor of the programmable module. When constructing the task descriptor, intent-based fields can be used to trigger the programmable module to execute its tasks. For example, the task descriptor can be constructed based on a combination of kernel_id, parameters, and buffer descriptions to trigger the programmable module's execution. In one example, the task descriptor may contain a code entry point (code_ptr), a parameter table (param_ptr) (such as image address, stride, size, coefficients), input / output buffer descriptions, synchronization events (synchronized based on a preset event synchronization mechanism), and Quality of Service (QoS) information. Then, the task descriptor can be added to the task queue corresponding to the programmable module. For example, the host-side can write the task descriptor to the task queue corresponding to the programmable module and trigger it using a doorbell register (doorbell trigger mechanism).

[0105] Subsequently, the programmable module can read the task descriptor from the task queue, determine the target software program associated with the module task from multiple candidate software programs based on the task descriptor, configure local registers and streaming memory access, and then configure the programmable module according to the target software program. For example, the target software program can be downloaded to the programmable module via SDK, and the programmable module can then execute the target software program and output the corresponding data stream. In some examples, such as... Figure 5As shown, the programmable module can determine the type of the task descriptor, locate the corresponding kernel library / code segment based on the type, configure the programmable module according to the parsing results of the kernel library / code segment, and, after waiting for input events from the programmable module, execute relevant processing through the target software program to complete the module task. After the programmable module completes its task, it can update its status and trigger events / interrupts to drive downstream modules to continue processing or submit the next task.

[0106] In this embodiment, by providing a programmable module and constructing a task descriptor according to the task intent, and triggering the configuration of the programmable module using the relevant target software program according to the task descriptor, it is possible to reduce area overhead with multimedia-oriented instructions and memory access, while retaining the software updateability. This achieves programmable execution of multimedia operator software while maintaining energy efficiency and area efficiency.

[0107] To enable those skilled in the art to better understand this embodiment, the following specific examples illustrate the embodiments of this application. However, it should be understood that the embodiments of this application are not limited thereto.

[0108] In related technologies, multimedia SoCs mainly use fixed-function IP pipelines to realize camera, video and display processing. Its structure has the following main shortcomings: (1) The repeated integration of scaling, color conversion, buffer handling and other modules between different IPs leads to waste of area and power consumption; (2) It is difficult to reuse when the algorithm is upgraded or the standard changes, resulting in poor flexibility; (3) The splicing of multiple IPs brings intermediate handling, additional delay and bandwidth waste; (4) The evolution cycle of the solid design of ASIC (Application-Specific Integrated Circuit) is too long.

[0109] In this embodiment, to achieve rapid adaptation to different multimedia tasks, a dynamic configuration mechanism combining a Configuration Description Object (PCO), a Resource and Routing Manager (RRM), and a Configuration Controller (CSU) is provided. The PCO can include pipeline structures (module instances and connection relationships), module parameters, input / output buffer descriptions (address / format / stride / tile), QoS constraints, and interconnection routing constraints. The RRM can run on the Host CPU driver, firmware, or UMPU management thread to parse task requirements, select available module instances, plan buffers, and calculate interconnection paths. The CSU can write the PCO into the routing / connection tables of each module register and the interconnection network, using shadow / active dual banks and performing atomic commit switching at frame or line boundaries.

[0110] like Figure 7 As shown, the dynamic configuration steps may include:

[0111] S1, Task Analysis and Goal Determination.

[0112] In practice, during the task parsing and target determination steps, the multimedia task to be processed can be fully analyzed to clarify the input and output format, resolution, and algorithm chain required for the task.

[0113] S2, Module Selection and Instantiation.

[0114] In this step, based on the previously determined task objectives, a suitable functional module (i.e., the first functional module) can be selected from multiple pre-provided functional modules on the chip and instantiated. During the module instantiation process, clock management and power domain management can be taken into account to ensure that the instantiated module (i.e., the second functional module) can operate stably under reasonable clock control and power supply.

[0115] In this embodiment, each functional module can have a standardized data interface and a configurable register interface module port that are interconnected in a configurable manner, supporting forwarding and QoS control based on data flow. Each data flow can have a corresponding data flow identifier (flow_id), which can be used for routing table indexing and QoS control.

[0116] S3, Buffer Planning and Format / Step Configuration.

[0117] After module selection and instantiation are completed, buffer planning and formatting, as well as stride configuration, will be performed. Specifically, the data buffer area can be rationally planned based on the data processing requirements of the task and the operating characteristics of the module, determining parameters such as the buffer size and location. At the same time, the data format and the stride during data reading or writing can also be configured to ensure that data can flow accurately and efficiently between modules during transmission and processing.

[0118] S4, Interconnection Path Planning.

[0119] Subsequently, the interconnection path planning step can be executed, and dynamic reconfiguration of inter-module connections can be achieved on the NoC mesh. In this step, various path planning strategies can be adopted. For example, the path with the fewest hops can be selected as the interconnection path to shorten the data transmission distance and improve transmission efficiency. In addition, the congestion weight of the path can also be taken into account during the planning process to avoid selecting paths that are prone to data congestion, thereby ensuring the data transmission stability of the entire interconnection system. Specifically, when there are multiple paths with similar hop counts, the path with the lower congestion weight will be selected first.

[0120] In this embodiment, the reconfigurable interconnection network can adopt NoC mesh, hierarchical interconnection or configurable cross switch structure, wherein the interconnection includes: (1) mapping configuration from module port to interconnection port; (2) routing / flow table indexed by flow_id, used to determine the next hop port of each flow at each routing node; (3) optional link utilization / congestion counter, used for cost evaluation of path selection; (4) shadow / active dual-bank routing table and security switching mechanism.

[0121] S5, write module parameters and interconnect table to the second area of ​​the register.

[0122] After the interconnect path planning is completed, the module parameters and interconnect table can be written to the second area of ​​the register, the shadowbank. The module parameters include various operating parameters configured for the modules in the previous steps, while the interconnect table records the planned interconnect path information between the modules. By writing this information to the shadowbank, temporary storage of configuration information can be achieved, preparing for the atomic application of subsequent configurations.

[0123] S6 triggers the interconnect path read region aspect at the safe switching point, making the configuration atomically effective.

[0124] After storing the above configuration information, a commit operation will be triggered at the safe switch point, ensuring that the module parameters and interconnect table configurations previously written to the shadow bank take effect atomically. Atomic effect means that the configuration information written to the registers associated with each module can be updated simultaneously and begin to take effect, avoiding system malfunctions caused by some configurations taking effect first while others do not. Triggering this operation at the safe switch point ensures the system remains in a stable operating state during configuration switching, minimizing the impact of configuration switching on the execution of current tasks.

[0125] S7, Run monitoring, triggers the reconstruction of the first functional module and / or interconnection path based on the monitoring results.

[0126] During task execution, monitoring can be performed, and the reconfiguration of the first functional module and / or interconnection path can be triggered based on the monitoring results. For example, congestion bypass and power-reducing links can be achieved by reconfiguring the first functional module and / or interconnection path.

[0127] In some examples, such as Figure 8As shown, after the application / middleware generates a multimedia task, the Resource Management Controller (RRM) can parse the target and constraints, construct a Directed Acyclic Graph (DAG) candidate, and then perform resource allocation, buffer planning, and NoC routing configuration. The NoC Mesh is a two-dimensional mesh on-chip network containing router nodes and network interfaces (NIUs), supporting flow-based table lookup forwarding. Based on the RRM's parsing and planning results, a Process Control Order (PCO) is generated, including module parameters, flow definitions, routing tables, and other information. Then, the Corresponding Service Unit (CSU) writes the relevant PCO to the shadow bank, for example, writing NoC table information to the module register and adding the task descriptor to the UMPU task queue. Subsequently, it can determine whether a frame boundary has been reached, for example, based on one or more signals such as EOF, SOF, VBLank, and frame_sync_event. After determining that a frame boundary has been reached, the CSU atomically commits, performs an Active / Shadow switch, and runs the new configuration, allowing data flow forwarding and processing to be performed according to the new pipeline / new routing. Simultaneously, during multimedia task processing, performance anomalies, congestion, or errors can be monitored; if any occur, dynamic reconfiguration is performed based on the task intent.

[0128] S8, the task ends and resources are recycled.

[0129] Once a task is completed, a resource reclamation step will be performed. In this step, various resources allocated to the current task, including instantiated modules, planned buffer areas, and occupied interconnect paths, will be reclaimed and released for use by other subsequent tasks. This improves resource utilization and ensures that the entire system can efficiently and orderly handle multiple consecutive or concurrent tasks.

[0130] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0131] Based on the same inventive concept, this application also provides a multimedia task processing apparatus for implementing the multimedia task processing method described above. This apparatus can be applied to or integrated into a chip or chip module, for example. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more multimedia task processing apparatus embodiments provided below can be found in the limitations of the multimedia task processing method described above, and will not be repeated here.

[0132] In one exemplary embodiment, such as Figure 9 As shown, a multimedia task processing device is provided, comprising:

[0133] The filtering module 901 is used to determine a first functional module related to the multimedia task from multiple functional modules of the chip according to the task intent of the multimedia task to be processed; the multiple functional modules include functional modules reused in different multimedia task processing processes.

[0134] The configuration module 902 is configured to, in response to the first functional module including a hardware functional module and a programmable module, configure the programmable module according to a pre-acquired software program to obtain a plurality of second functional modules; the plurality of second functional modules include the hardware functional module and the configured programmable module.

[0135] The path reconstruction module 903 is used to reconstruct the interconnection path of the plurality of second functional modules, and during the execution of the multimedia task using the plurality of second functional modules, transmit the data stream of the second functional modules between the plurality of second functional modules according to the first interconnection path obtained after reconstruction.

[0136] In one embodiment, the configuration module 902 is configured to: determine multiple candidate paths of the plurality of second functional modules on the chip; for each candidate path, determine the number of path hops based on the number of nodes included in the candidate path; and filter the plurality of candidate paths based on the number of path hops of each candidate path to determine a first interconnect path.

[0137] In one embodiment, the configuration module 902 is configured to: determine the congestion level of each candidate path; determine a first interconnection path of the plurality of second functional modules from the plurality of candidate paths based on the path hop count and the congestion level of each path; wherein the path hop count and the congestion level of the first interconnection path satisfy preset filtering conditions.

[0138] In one embodiment, the configuration module 902 is further configured to: reconstruct the interconnect paths of the plurality of second functional modules in response to a trigger event for updating the interconnect path, to obtain a second interconnect path for the chip; write the second interconnect path into a second region in a register of the chip; the register further includes a first region, the second region being used to store the first interconnect path used by the plurality of second functional modules when processing the current multimedia task image frame; after the current multimedia task image frame processing is completed, transmit the subsequent data stream of the second functional modules among the plurality of second functional modules according to the second interconnect path written in the second region.

[0139] In one embodiment, the registers include registers associated with each of the plurality of second functional modules; the configuration module 902 is configured to: in response to the arrival of the frame boundary between the current multimedia task image frame and the next multimedia task image frame, switch the interconnect path reading area from the first area of ​​each of the registers to each of the second areas, so as to transmit subsequent data streams of the second functional modules among the plurality of second functional modules according to the second interconnect path stored in each of the second areas.

[0140] In one embodiment, the path reconstruction module 903 is configured to: obtain a task descriptor according to the task intent; the task descriptor represents a module task to be executed by the programmable module; add the task descriptor to the task queue corresponding to the programmable module, so that after the programmable module reads the task descriptor from the task queue, it determines a target software program associated with the module task from multiple candidate software programs according to the task descriptor; and configure the programmable module according to the target software program.

[0141] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for various devices and products applied to or integrated into a chip, all of their modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits; for various devices and products applied to or integrated into a chip module, all of their modules / units can be implemented using hardware methods such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0142] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a multimedia task method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0143] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0144] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0145] Based on the same inventive concept, this application also provides a chip, including a processor coupled to a memory, for executing a computer program or instructions stored in the memory, and implementing the steps in the above method embodiments when the processor executes the computer program or instructions.

[0146] It is understood that the chip involved in the embodiments of this application may be a field-programmable gate array (FPGA), may be an application-specific integrated circuit (ASIC), may be a system on chip (SoC), may be a central processor unit (CPU), may be a network processor (NP), may be a digital signal processor (DSP), may be a microcontroller unit (MCU), may be a programmable logic device (PLD), or other integrated chips, etc.

[0147] Based on the same inventive concept, this application also provides a chip module, such as... Figure 11 As shown, the chip module includes a communication module, a power module, a storage module, and a chip. Among them:

[0148] The power module is used to provide power to the chip module; the storage module is used to store data and instructions; the communication module is used for internal communication within the chip module, or for communication between the chip module and external devices; this chip corresponds to the chip in the above chip embodiment.

[0149] The implementation method of this chip module can be found in the relevant content of the above chip embodiment, and will not be repeated here.

[0150] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0151] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0153] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0154] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0155] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A multimedia task processing method, characterized in that, The method includes: Based on the task intent of the multimedia task to be processed, a first functional module related to the multimedia task is determined from multiple functional modules of the chip; the multiple functional modules include functional modules reused in different multimedia task processing processes; In response to the first functional module including a hardware functional module and a programmable module, the programmable module is configured according to a pre-acquired software program to obtain a plurality of second functional modules; the plurality of second functional modules include the hardware functional module and the configured programmable module; The interconnection paths of the plurality of second functional modules are reconstructed, and during the execution of the multimedia task using the plurality of second functional modules, the data streams of the second functional modules are transmitted between the plurality of second functional modules according to the first interconnection path obtained after reconstruction.

2. The method according to claim 1, characterized in that, The reconstructing of the interconnection paths of the plurality of second functional modules includes: Determine multiple candidate paths for the plurality of second functional modules on the chip; For each candidate path, the number of hops is determined based on the number of nodes included in the candidate path; The candidate paths are filtered based on the path hop count of each candidate path to determine the first interconnection path.

3. The method according to claim 2, characterized in that, The step of filtering the multiple candidate paths based on the path hop count of each candidate path to determine the first interconnection path includes: Determine the congestion level of each of the candidate paths; Based on the path hop count and the congestion level of each path, a first interconnection path for the multiple second functional modules is determined from the multiple candidate paths; the path hop count and the congestion level of the first interconnection path meet preset filtering conditions.

4. The method according to claim 1, characterized in that, After reconstructing the interconnection paths of the plurality of second functional modules, the method further includes: In response to the occurrence of a trigger event for updating the interconnect path, the interconnect paths of the plurality of second functional modules are reconstructed again to obtain the second interconnect path of the chip; The second interconnect path is written into a second area of ​​the register of the chip; the register further includes a first area, and the second area is used to store the first interconnect path applied by the plurality of second functional modules when processing the current multimedia task image frame; After transmitting the data stream of the second functional modules among the plurality of second functional modules according to the first interconnection path obtained after reconstruction, the method further includes: After the current multimedia task image frame processing is completed, the subsequent data stream of the second functional modules is transmitted between multiple second functional modules according to the second interconnection path written in the second region.

5. The method according to claim 4, characterized in that, The registers include multiple registers associated with each of the second functional modules; After the current multimedia task image frame processing is completed, the subsequent data stream of the second functional modules is transmitted among multiple second functional modules according to the second interconnection path written in the second region, including: In response to the arrival of the frame boundary between the current multimedia task image frame and the next multimedia task image frame, the interconnect path reading area is switched from the first area of ​​each of the registers to each of the second areas, so as to transmit the subsequent data stream of the second functional modules between the plurality of second functional modules according to the second interconnect path stored in each of the second areas.

6. The method according to any one of claims 1 to 5, characterized in that, The step of configuring the programmable module according to the pre-acquired software program includes: A task descriptor is obtained based on the task intent; the task descriptor represents the module task to be executed by the programmable module. The task descriptor is added to the task queue corresponding to the programmable module, so that after the programmable module reads the task descriptor from the task queue, the target software program associated with the module task is determined from multiple candidate software programs based on the task descriptor. Configure the programmable module according to the target software program.

7. A multimedia task processing device, characterized in that, The device includes: A filtering module is used to determine a first functional module related to the multimedia task from multiple functional modules of the chip according to the task intent of the multimedia task to be processed; the multiple functional modules include functional modules reused in different multimedia task processing processes. A configuration module is configured to, in response to the first functional module including a hardware functional module and a programmable module, configure the programmable module according to a pre-acquired software program to obtain a plurality of second functional modules; the plurality of second functional modules include the hardware functional module and the configured programmable module. The path reconstruction module is used to reconstruct the interconnection path of the plurality of second functional modules, and during the execution of the multimedia task using the plurality of second functional modules, transmit the data stream of the second functional modules between the plurality of second functional modules according to the first interconnection path obtained after reconstruction.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A chip, characterized in that, The device includes a processor and a communication interface, wherein the processor is configured to cause the chip to perform the steps of the method described in any one of claims 1 to 6.

10. A chip module, characterized in that, This includes communication modules, power modules, storage modules, and chips, among which: The power module is used to provide power to the chip module; The storage module is used to store data and instructions; The communication module is used for internal communication within the chip module, or for communication between the chip module and external devices. The chip is used to perform the steps of the method according to any one of claims 1 to 6.

Images