Video task scheduling method and device, electronic equipment and storage medium

Abstract

This invention provides a video task scheduling method, apparatus, electronic device, and storage medium, belonging to the field of artificial intelligence chip technology. The method includes: receiving a video task instruction, the video task instruction comprising a data stream identifier and first frame indication information; when the first frame indication information indicates that the current instruction is the first frame of the bitstream, acquiring the real-time status of multiple video execution units and selecting a target execution unit from the video execution units in an idle state; establishing a mapping relationship between the data stream identifier and the target execution unit; and sending the video task instruction to the target execution unit. This invention solves the problem of task blocking and resource waste caused by the inability of existing software scheduling to perceive hardware busy / idle conditions by dynamically allocating target units and establishing mappings based on the real-time idle state of the video execution units upon receiving the first frame instruction of the bitstream. This improves the overall scheduling efficiency and resource utilization of video encoding / decoding units in general-purpose graphics processors.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence chip technology, and in particular to a video task scheduling method, apparatus, electronic device, and storage medium. Background Technology

[0002] In general-purpose graphics processors, multiple video execution units are typically integrated to process a large number of encoding and decoding tasks in parallel. A scheduling mechanism is needed to accurately distribute the video instruction stream to each execution unit.

[0003] To ensure contextual continuity during encoding and decoding of the same video stream, existing technologies typically employ a software-driven static binding strategy. This means that when issuing instructions, the software pre-specifies the target execution unit based on the stream identifier, thereby ensuring that all instruction packets belonging to the same stream are consistently distributed to the same video execution unit for processing.

[0004] However, this software-preset scheduling method cannot obtain real-time information about the actual load status of the underlying hardware. Because of the lack of real-time interaction between the software layer and the hardware execution units regarding their busy / idle status, new task instructions are frequently sent to currently busy execution units during task distribution, while other idle units remain unutilized. This mismatch in resource allocation leads to a waste of hardware computing power, thereby reducing the overall scheduling efficiency and resource utilization of the video encoding / decoding unit. Summary of the Invention

[0005] This invention provides a video task scheduling method, apparatus, electronic device, and storage medium to address the deficiencies in the prior art and improve the overall scheduling efficiency and resource utilization of video encoding and decoding units in general-purpose graphics processors.

[0006] This invention provides a video task scheduling method, comprising the following steps:

[0007] Receive a video task instruction, the video task instruction including a data stream identifier and first frame indication information;

[0008] When the first frame indication information indicates that the current instruction is the first frame of the bitstream, the real-time status of multiple video execution units is obtained, and the target execution unit is selected from the video execution units that are in an idle state;

[0009] Establish a mapping relationship between the data stream identifier and the target execution unit;

[0010] The video task instruction is sent to the target execution unit.

[0011] According to a video task scheduling method provided by the present invention, after receiving a video task instruction, the method further includes:

[0012] When the first frame indication information indicates that the current instruction is not the first frame of the code stream, the pre-established mapping relationship is retrieved according to the data stream identifier to determine the target execution unit bound to the data stream identifier;

[0013] The video task instruction is sent to the target execution unit bound to the data stream identifier.

[0014] According to a video task scheduling method provided by the present invention, the data stream identifier includes a bitstream identifier and a channel identifier;

[0015] Establishing the mapping relationship between the data stream identifier and the target execution unit includes:

[0016] The bitstream identifier and the channel identifier are concatenated or combined to obtain the address of the hardware storage space.

[0017] The number of the target execution unit is used as data content and written into the storage unit corresponding to the address.

[0018] According to a video task scheduling method provided by the present invention, the step of selecting a target execution unit from video execution units in an idle state includes:

[0019] Traverse all video execution units and detect the current real-time status flag of each video execution unit;

[0020] The video execution units whose real-time state is marked as the idle state are identified as a candidate set;

[0021] According to a preset cyclic order, a video execution unit is sequentially determined from the candidate set as the target execution unit; wherein, when the previously assigned video execution unit is the Nth in the cyclic order, the N+1th video execution unit in the cyclic order is preferentially determined to be a candidate set.

[0022] According to a video task scheduling method provided by the present invention, after sending the video task instruction to the target execution unit, the method further includes:

[0023] Update the real-time status flag of the target execution unit from the idle state to the busy state;

[0024] When a video task instruction indicating the first frame of the bitstream is received again, the video execution unit in the busy state will be excluded from the candidate set.

[0025] A video task scheduling method provided by the present invention further includes:

[0026] Receive the task completion signal fed back by the target execution unit after completing the video task;

[0027] In response to the task completion signal, the real-time status flag of the target execution unit is reset from the busy state to the idle state, so that the target execution unit can be reinstated into the candidate set.

[0028] A video task scheduling method provided by the present invention further includes:

[0029] When it is detected that the real-time status of all video execution units is marked as busy, the dispatch operation of the current video task instruction is suspended.

[0030] The real-time status is continuously monitored until the status of at least one of the video execution units changes to the idle state.

[0031] The present invention also provides a video task scheduling device, comprising the following modules:

[0032] The instruction receiving module is used to receive video task instructions, which include data stream identifiers and first frame indication information;

[0033] The allocation selection module is used to obtain the real-time status of multiple video execution units and select the target execution unit from the video execution units that are in an idle state when the first frame indication information indicates that the current instruction is the first frame of the bitstream.

[0034] The instruction issuing module is used to establish the mapping relationship between the data stream identifier and the target execution unit;

[0035] The instruction issuing module is also used to send the video task instruction to the target execution unit.

[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the video task scheduling method described above.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video task scheduling method as described above.

[0038] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the video task scheduling method as described above.

[0039] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0040] By receiving video task instructions containing data stream identifiers and first frame indication information, the start time of a new bitstream task can be directly identified at the hardware level. Then, at the critical node when the first frame arrives, by acquiring the real-time status of the video execution unit and specifically selecting the target from the idle units, the problem of blocking and resource waste caused by software assigning new tasks to busy cores due to the inability of software to perceive hardware busyness is effectively solved, thus achieving hardware load balancing. Furthermore, by establishing a mapping relationship between the data stream identifier and the target execution unit, the execution path of subsequent tasks of the bitstream is locked, thereby decoupling the software's dependence on hardware status while ensuring the consistency of the encoding and decoding context of the same bitstream, and significantly improving the overall scheduling efficiency and resource utilization of the video encoding and decoding unit in the general-purpose graphics processor. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is one of the flowcharts illustrating the video task scheduling method provided by the present invention.

[0043] Figure 2 This is the second flowchart of the video task scheduling method provided by the present invention.

[0044] Figure 3 This is the third flowchart of the video task scheduling method provided by the present invention.

[0045] Figure 4 This is the fourth flowchart of the video task scheduling method provided by the present invention.

[0046] Figure 5 This is the fifth flowchart of the video task scheduling method provided by the present invention.

[0047] Figure 6 This is the sixth flowchart of the video task scheduling method provided by the present invention.

[0048] Figure 7 This is the seventh flowchart of the video task scheduling method provided by the present invention.

[0049] Figure 8 This is a schematic diagram of the video task scheduling device provided by the present invention.

[0050] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0052] It should be noted that in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships according to the accompanying drawings, are only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0053] The terms "first," "second," etc., used in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0054] The following is combined with Figures 1-9 This invention describes the video task scheduling method, apparatus, electronic device, storage medium, and computer program product provided by the present invention.

[0055] This embodiment provides a video task scheduling method. It should be noted that the execution entity in this embodiment can be a video task scheduling device integrated into a general-purpose computing on graphics processing units (GPGPU) or a video processing chip (artificial intelligence chip). This video task scheduling device is an independent hardware circuit unit integrated into the graphics processor, capable of directly processing the scheduling logic of video tasks at the speed of hardware circuitry, thereby reducing the load on software drivers.

[0056] Reference Figure 1 , Figure 1 This is one of the flowcharts illustrating the video task scheduling method provided by the present invention, such as... Figure 1 As shown, the method specifically includes the following steps:

[0057] Step 101: Receive video task instructions, which include data stream identifiers and first frame indication information.

[0058] In step 101, the video task scheduling device receives video task instructions from upper-layer software (such as drivers) through the instruction channel module inside the chip. These video task instructions refer to data packets or command words used to instruct the hardware to perform video encoding / decoding operations; their internal payload typically includes information such as the address of the video data to be processed and encoding / decoding parameters.

[0059] To achieve precise scheduling, video task instructions carry two key scheduling criteria: one is the data stream identifier, which is an identifier used to uniquely identify the video stream to which the current video task belongs. In multi-channel concurrent scenarios, different video files or video streams are assigned different data stream identifiers so that the device can distinguish different contexts.

[0060] Secondly, there is the first frame indication information, which is a flag used to indicate whether the current video task instruction is the starting task of the video stream (e.g., the first frame of video data). For example, this first frame indication information can be a specific Flag bit in the instruction packet header (e.g., First_Frame_Flag). When this bit is set (e.g., set to 1), it indicates the start of a new decoding session; when this bit is reset (e.g., set to 0), it indicates that the current instruction is a subsequent frame of the stream.

[0061] Step 102: When the first frame indication information indicates that the current instruction is the first frame of the bitstream, obtain the real-time status of multiple video execution units, and select the target execution unit from the video execution units that are in an idle state.

[0062] In step 102, the allocation selection module inside the video task scheduling device parses the received video task instructions. When the first frame indication information is identified as valid (i.e., indicating that the current frame is the first frame of the bitstream), it means that no fixed hardware resources have been allocated to the data stream, so a dynamic allocation mechanism needs to be triggered.

[0063] Specifically, the video task scheduling device (which may be the allocation selection module within the video task scheduling device) directly reads the real-time working status of multiple video execution units connected to the backend via hardware signal lines. In this embodiment, the video execution unit can also be called a Video Core or codec core. The real-time status in step 102 refers to the physical busy / idle status of the video execution unit at the current moment, which typically includes an idle state (indicating no current task and the ability to immediately receive instructions) and a busy state (indicating that it is processing a task and cannot receive new instructions).

[0064] After obtaining the real-time status of all video execution units, the video task scheduling device will filter out the video execution units currently marked as idle as candidate objects, and select one of these idle video execution units as the target execution unit for this task according to the preset selection rules.

[0065] It should be noted that the preset selection rules here can be random selection, selection based on load balancing, or selection based on priority, as long as the selected target execution unit is currently in an idle state. This embodiment does not impose specific restrictions on this.

[0066] Step 103: Establish the mapping relationship between data stream identifiers and target execution units.

[0067] Because video encoding and decoding tasks are highly context-dependent, subsequent frames of the same bitstream must depend on the decoding environment of the preceding frames. Therefore, it is essential to ensure that all tasks within the same bitstream are processed by the same video execution unit. To achieve this, after dynamically selecting an idle target execution unit for the first frame of the bitstream in step 102, step 103 requires immediately establishing a mapping relationship between the data stream identifier and the target execution unit.

[0068] Specifically, the video task scheduling device (which may be the instruction issuing module within the video task scheduling device) associates the current data stream identifier with the selected target execution unit's number (Core ID), and stores this association as a mapping relationship in the Core ID recording module. The Core ID recording module may be a register or SRAM within a hardware circuit unit. This mapping relationship acts like a routing table, indicating the destination for all subsequent non-first frame tasks arriving from this data stream.

[0069] Step 104: Send the video task instruction to the target execution unit.

[0070] After identifying the recipient and recording the mapping relationship, the video task scheduling device (specifically, the instruction distribution module within the video task scheduling device) distributes the video task instruction to the identified target execution unit via the internal bus. Upon receiving the video task instruction, the target execution unit begins executing the specific encoding and decoding operations.

[0071] The technical solution provided in this embodiment offers at least the following advantages: By parsing the first frame indication information on the hardware side and triggering dynamic allocation based on real-time hardware state awareness only when the first frame of the bitstream is detected, this application can directly perceive the busy / idle status of the video execution unit using hardware, avoiding the blocking and resource waste caused by "sending tasks to busy cores" due to state awareness lag in software scheduling; furthermore, by establishing a mapping relationship, it ensures that subsequent tasks of the same bitstream can maintain context consistency. This method realizes the hardware-based and intelligent scheduling of video tasks, significantly improving the scheduling efficiency and resource utilization of multi-core video devices.

[0072] As a further extension of the above embodiments, this embodiment also relates to the processing flow when the received video task instruction belongs to a non-first frame task. Since the video encoding and decoding process has strong context dependencies—that is, the decoding of subsequent frames often depends on the reference data or state information of the preceding frames—ensuring that tasks of the same bitstream are distributed to the same video execution unit is crucial.

[0073] Based on this, refer to Figure 2 , Figure 2 This is the second flowchart illustrating the video task scheduling method provided by the present invention, as shown below. Figure 2 As shown, after receiving the video task instruction in step 101, the video task scheduling method further includes the following steps:

[0074] Step 201: When the first frame indication information indicates that the current instruction is not the first frame of the code stream, retrieve the pre-established mapping relationship according to the data stream identifier and determine the target execution unit bound to the data stream identifier.

[0075] In step 201, the video task scheduling device (specifically, the allocation selection module within the video task scheduling device) first parses the first frame indication information in the instruction packet. When the parsing result indicates that the current instruction is not the first frame of the bitstream, for example, when the Flag bit of the first frame indication information is 0, the device enters the "static binding" scheduling mode.

[0076] At this point, the video task scheduling device no longer triggers the search logic for the idle state, but instead switches to a query operation. Specifically, the video task scheduling device uses the data stream identifier carried in the current video task instruction as an index to retrieve the Core ID record module in the aforementioned embodiment.

[0077] During the processing of the first frame of the stream, the mapping relationship between the stream identifier and the core ID of the selected video execution unit has been established and stored. Therefore, through this retrieval, the video task scheduling device can quickly and accurately read the video execution unit number previously assigned to the stream. The retrieved video execution unit is then identified as the target execution unit bound to the current stream identifier.

[0078] It should be noted that this retrieval process is performed entirely at the hardware level, avoiding complex calculations at the software level and ensuring extremely low latency from receiving instructions to determining the target execution unit.

[0079] Step 202: Send the video task instruction to the target execution unit bound to the data stream identifier.

[0080] After locking onto the target execution unit, the video task scheduling device (specifically, the instruction issuing module within the video task scheduling device) routes the current video task instruction and sends it to the target execution unit.

[0081] This step ensures that all subsequent video task instructions belonging to the same data stream identifier are continuously delivered to the same video execution unit. This allows the context information cached by the video execution unit to be continuously reused, thereby guaranteeing the correct execution of video encoding and decoding tasks and avoiding decoding errors or screen tearing caused by out-of-order tasks or cross-core allocation.

[0082] The technical solution provided in this embodiment brings at least the following beneficial effects: by performing retrieval and distribution based on mapping relationship for non-first frame instructions, the context consistency of the video stream can be automatically maintained without software intervention, ensuring continuous processing of the same bitstream on the same hardware core; at the same time, compared with recalculating resource allocation for each task, the method of directly retrieving pre-stored mapping relationship greatly reduces scheduling overhead and improves the throughput of the video task scheduling device.

[0083] To further clarify the specific composition of the data stream identifier and the specific implementation logic of establishing the mapping relationship at the hardware level in the above embodiments, this embodiment provides a specific hardware storage solution.

[0084] In practical general-purpose graphics processing unit (GPU) training and inference scenarios, video tasks are typically delivered concurrently across multiple channels. Therefore, the data stream identifier in this embodiment is not a single variable; it includes information from at least two dimensions:

[0085] Stream ID: Used to distinguish different video stream objects within the same channel;

[0086] Channel ID: Used to distinguish different hardware input channels (for example, a device may have multiple video task distribution channels). Together, these two constitute a unique identifier for the video task, ensuring task uniqueness in a multi-channel, multi-stream concurrent environment.

[0087] Accordingly, Figure 3 This demonstrates the specific hardware implementation process for establishing the mapping relationship between data stream identifiers and target execution units. (Refer to...) Figure 3 , Figure 3 This is the third flowchart of the video task scheduling method provided by the present invention, as shown below. Figure 3 As shown, in step 103 above, establishing the mapping relationship between data stream identifiers and target execution units specifically includes the following sub-steps:

[0088] Step 301: Concatenate or combine the stream identifier and the channel identifier to obtain the address of the hardware storage space.

[0089] In step 301, the video task scheduling device uses its internal Core ID recording module to store the mapping relationship. Unlike the hash map or linked list lookup commonly used at the software level, this embodiment adopts the most efficient hardware-based direct mapping mechanism.

[0090] Specifically, the video task scheduling device extracts the channel identifier and stream identifier from the video task instruction through hardware logic circuits. Then, according to preset bit width rules, it performs binary bit-level concatenation or combination operations on these two identifiers. For example, assuming the channel identifier occupies the high bits and the stream identifier occupies the low bits, the device concatenates them into a complete binary value. This binary value is directly used as the address for the hardware storage space. This method ensures that each unique "channel + stream" combination corresponds to a unique physical storage unit in the storage space, without the need for complex hash operations or collision resolution.

[0091] Step 302: Write the target execution unit number as data content into the storage unit corresponding to the address address.

[0092] After calculating the address, the video task scheduling device uses the unique number of the target execution unit selected in step 102 (e.g., Core ID, such as 0x01, 0x02, etc.) as the data content to be stored.

[0093] Subsequently, the video task scheduling device initiates a write operation to the Core ID recording module, writing the number of the target execution unit into the specific storage unit pointed to by the addressing address calculated in step 301.

[0094] Furthermore, based on the above storage structure, when processing non-first frame instructions, the video task scheduling device only needs to extract the bitstream identifier and channel identifier from the instruction and concatenate them into an address. It can then directly obtain the bound target execution unit number from that address through a single read operation.

[0095] The technical solution provided in this embodiment offers at least the following advantages: This application employs a hardware direct addressing method of "ID concatenation to generate address" to store the mapping relationship. This design transforms the complex logical lookup problem into a physical memory access problem. Compared to software-maintained lookup tables, this fully hardware-implemented lookup mechanism achieves zero search overhead, greatly improving the instruction throughput of the video task scheduling device when processing high-concurrency video streams, while simultaneously relieving the CPU of the computational burden of maintaining the mapping table.

[0096] To further optimize load balancing in a multi-core environment and prevent some video execution units from being overloaded while others remain idle for extended periods, this embodiment details the specific implementation logic of selecting the target execution unit from idle video execution units in step 102 above.

[0097] Reference Figure 4 , Figure 4 This is the fourth flowchart of the video task scheduling method provided by the present invention, as shown below. Figure 4 As shown, selecting a target execution unit from idle video execution units includes the following sub-steps:

[0098] Step 401: Traverse all video execution units and detect the current real-time status flag of each video execution unit.

[0099] In step 401, the video task scheduling device (specifically, the allocation selection module integrated within the video task scheduling device) performs a parallel or serial status scan of all video execution units (e.g., Core 0 to Core 3) integrated within the device.

[0100] Specifically, each video execution unit is equipped with a hardware register bit or signal line to indicate its current busy / idle status. The video task scheduler reads these signals to detect the current "real-time status flag" of each video execution unit. For example, a real-time status flag of "1" represents busy, and a real-time status flag of "0" represents idle.

[0101] Step 402: Determine the video execution units whose real-time status is marked as idle as the candidate set.

[0102] After completing the status detection, the video task scheduling device filters all video execution units through logic circuits. All video execution units whose real-time status flags indicate an idle state are logically categorized into a candidate set.

[0103] It's important to note that the candidate set here doesn't necessarily mean creating a list of entities in memory. In hardware implementations, it can be a list of currently available execution units filtered through mask operations. For example, if Core 0 and Core 2 are busy, while Core 1 and Core 3 are idle, the candidate set will only contain Core 1 and Core 3.

[0104] Step 403: According to the preset cyclic order, determine a video execution unit as the target execution unit from the candidate set in sequence; wherein, when the video execution unit allocated in the previous time is the Nth in the cyclic order, this time we will prioritize judging whether the (N+1)th video execution unit in the cyclic order belongs to the candidate set.

[0105] To ensure that tasks are evenly distributed among the various video execution units, the video task scheduling device does not always select the first unit in the candidate set (e.g., it does not always prioritize Core 0), but instead selects in a cyclical order.

[0106] The specific selection logic is as follows: If the previously assigned video execution unit was the Nth in the loop sequence, then this time we first determine whether the (N+1)th video execution unit in the loop sequence belongs to the candidate set. This can be achieved using a "pointer" or a "recorder" to remember which video execution unit the task was previously assigned to.

[0107] For example, suppose the device has 4 video execution units (numbered 0, 1, 2, 3). If the previous first frame task of the bitstream was assigned to the first unit (Core 0, i.e., N=0), then when the new first frame task of the bitstream arrives, the video task scheduling device will first check whether the second unit (Core 1, i.e., N+1) is in the candidate set determined in step 402.

[0108] Scenario 1: If Core 1 is currently idle (belonging to the candidate set), then Core 1 is directly selected as the target execution unit for this operation.

[0109] Scenario 2: If Core 1 is currently busy (not in the candidate set), then continue searching downwards in the circular order for the 3rd unit (Core 2), and so on, until the first video execution unit belonging to the candidate set is found in the circular queue.

[0110] In this way, the video task scheduling device implements a dynamic polling selection mechanism that skips busy nodes.

[0111] The technical solution provided in this embodiment brings at least the following beneficial effects: This application achieves dynamic load balancing at the video execution unit level by combining idle state filtering and a round-robin mechanism. Compared with simple random allocation or fixed-order allocation, this method can ensure that tasks are always sent to idle units and that all idle units are used evenly, avoiding uneven heat dissipation or lifespan loss caused by some cores being idle for a long time or some cores being overused, thus maximizing the overall efficiency of multi-core parallel processing.

[0112] To ensure the security of multi-core parallel processing and prevent multiple different video stream tasks from being incorrectly assigned to the same busy video execution unit (i.e., to avoid resource contention), this embodiment further includes a real-time update operation of the hardware status flag after sending the video task instruction to the target execution unit for execution.

[0113] Reference Figure 5 , Figure 5 This is the fifth flowchart illustrating the video task scheduling method provided by the present invention, as shown below. Figure 5 As shown, after sending the video task instructions to the target execution unit, the following steps are also included:

[0114] Step 501: Update the real-time status flag of the target execution unit from idle to busy.

[0115] In this step, once the video task scheduling device (specifically, the allocation selection module within the video task scheduling device) determines the target execution unit and issues the current instruction, the video task scheduling device will immediately trigger a status write operation.

[0116] Specifically, the video task scheduler sends a set signal to the hardware register that records the status of the target execution unit, flipping its real-time status flag from an idle state (e.g., logic "0") to a busy state (e.g., logic "1") representing an occupied state. This update is instantaneous, typically completed within the same clock cycle as the instruction is issued or the immediately following cycle. Its purpose is to lock the video execution unit at the hardware level, indicating that the video execution unit is currently processing video encoding and decoding tasks at full capacity and cannot accept new context environments at this time.

[0117] Step 502: When a video task instruction indicating the first frame of the bitstream is received again, the video execution unit in the busy state is excluded from the candidate set.

[0118] Based on the updated status flag in step 501, when the video task scheduling device receives a new video task instruction indicating the first frame of the bitstream (i.e., a new round of dynamic core selection process needs to be started), the device will automatically execute the exclusion mechanism.

[0119] Specifically, going back to step 402, when the video task scheduling device is traversing and detecting the status of all video execution units, it will detect that the real-time status flag of the aforementioned target execution unit has changed to a busy state. Therefore, this video execution unit will not pass the idle state filtering condition and will be directly excluded from the candidate set for this scheduling.

[0120] In other words, in the new scheduling cycle, regardless of whether the preset cycle sequence pointer points to the video execution unit, the video task scheduling device will automatically skip the video execution unit and look for other video execution units that are idle.

[0121] The technical solution provided in this embodiment offers at least the following beneficial effects: By updating the hardware status flag immediately after the instruction is issued, this application ensures that the hardware scheduler can determine in real time which video execution units are busy. This fundamentally eliminates the possibility of allocating new bitstreams to busy video execution units during the next allocation of new tasks. This completely solves the "multi-task collision" problem caused by the lag in software scheduling perception in the prior art, effectively ensuring the exclusive processing of video execution units and improving the stability and decoding success rate of the device.

[0122] To ensure the recycling of video execution unit resources and prevent scheduling deadlock caused by all cores being locked for a long time, this embodiment further provides a hardware state reset mechanism based on the above embodiment.

[0123] Reference Figure 6 , Figure 6This is the sixth flowchart of the video task scheduling method provided by the present invention, as shown below. Figure 6 As shown, this process mainly describes the signal interaction between the video task scheduling device and the video execution unit, specifically including the following steps:

[0124] Step 601: Receive the task completion signal fed back by the target execution unit after completing the video task.

[0125] In step 601, the video task scheduling device (specifically, the allocation selection module within the video task scheduling device) continuously receives feedback signals from each video execution unit. When a target execution unit completes its assigned video encoding / decoding task (e.g., decoding or encoding a frame of image), the internal logic of the target execution unit triggers a hardware signal, namely a task completion signal. This task completion signal is fed back to the video task scheduling device through a dedicated hardware line to notify the video task scheduling device that the current hardware resources have been released and are ready to receive new tasks.

[0126] Step 602: In response to the task completion signal, reset the real-time status flag of the target execution unit from the busy state to the idle state so that the target execution unit can be reinstated into the candidate set.

[0127] Upon detecting the rising edge or level change of the task completion signal, the video task scheduling device immediately performs a status reset operation.

[0128] Specifically, the video task scheduling device locates the hardware status register bit corresponding to the target execution unit and clears or resets the real-time status flag of the target execution unit from the busy state indicating occupation to the idle state indicating availability.

[0129] This reset action is crucial for the continued operation of the entire scheduling device. Once the status flag is restored to the idle state, it means that the target execution unit has been logically unlocked. When the video task scheduling device traverses the detection states again, the target execution unit that has just been released will be able to pass the idle state filtering and thus be reinstated into the candidate set.

[0130] At this point, if the polling order pointer happens to point to the target execution unit or its vicinity, the target execution unit will have the opportunity to be selected again to undertake the task of processing the first frame of the new video stream.

[0131] It should be further explained that this "busy-execute-reset" mechanism constitutes a complete hardware state machine closed loop. This mechanism is entirely triggered by hardware signals, without requiring the CPU to poll the core state, ensuring a nanosecond-level response speed for state updates.

[0132] The technical solution provided in this embodiment brings at least the following beneficial effects: This application realizes the dynamic recycling and reuse of video execution unit resources by establishing an automatic reset mechanism based on task completion signals. This closed-loop design ensures that as long as a hardware unit completes its task, it can immediately return to the resource pool to wait for reallocation, eliminating the time gap of resource idleness, thereby greatly improving the overall throughput and hardware utilization of the video task scheduling device on a macroscopic level.

[0133] In real-world high-throughput training or inference scenarios using GPGPUs, the video task delivery rate may exceed the processing rate of the video execution units, causing all video execution units within the device to be operating at full capacity at any given moment. To ensure the robustness of the scheduling logic under such extreme conditions and prevent hardware conflicts caused by instruction loss or forced allocation, this embodiment, based on the above embodiments, further provides a flow control mechanism that marks all video execution units as busy in real time.

[0134] Reference Figure 7 , Figure 7 This is the seventh flowchart of the video task scheduling method provided by the present invention, as shown below. Figure 7 As shown, Figure 7 A flowchart illustrating task suspension and resumption when all video execution units are in a fully busy state is shown. The process specifically includes the following steps:

[0135] Step 701: When it is detected that the real-time status of all video execution units is marked as busy, suspend the distribution operation of the current video task instruction.

[0136] In this step, the video task scheduling device (specifically, the allocation selection module in the video task scheduling device) will detect global resources before or during the execution of step 102 above.

[0137] If the detection results show that the real-time status flags of all available video execution units are busy, it indicates that there are currently no hardware resources available to handle new video encoding / decoding tasks. In this case, the video task scheduling device triggers a suspension protection mechanism. Specifically, the instruction channel module or instruction issuing module will temporarily stop pushing the current video task instruction to the backend. The pending video task instruction will be temporarily stored in the input buffer queue (First In First Out, FIFO) of the video task scheduling device, or the upper layer will be notified to pause packet sending through the bus backpressure mechanism, thus ensuring that the current instruction is not discarded or mistakenly sent to any busy execution unit.

[0138] Step 702: Continuously monitor the real-time status until the status of at least one video execution unit changes to an idle state.

[0139] During the suspension period, the video task scheduler does not stop working, but enters a "busy waiting" mode. The video task scheduler will continuously poll the status registers of each video execution unit at a very high frequency, or silently wait for a task completion signal from any video execution unit.

[0140] Once at least one video execution unit is detected to have completed its current task and its status changes from busy to idle, the fully busy state is lifted. At this point, the video task scheduling device immediately releases the suspended state, activates the currently pending video task instructions, and re-triggers the selection logic in step 102 above, distributing the video task instructions to the newly idle target execution unit, thereby resuming normal pipeline operation.

[0141] The technical solution provided in this embodiment brings at least the following beneficial effects: By introducing a hardware flow control mechanism of "full busy suspension - idle wake-up," this application greatly enhances the stability of the video task scheduling device under extreme pressure. This mechanism acts as a "safety valve" when the device is overloaded, effectively preventing the risk of instruction overwriting, execution unit logic errors, or data loss caused by forcibly distributing tasks when resources are exhausted, and ensuring that video encoding and decoding services can be executed in an orderly manner even in high-concurrency congestion scenarios.

[0142] Reference Figure 8 , Figure 8 This is a schematic diagram of the video task scheduling device provided by the present invention. The video task scheduling device is integrated inside an artificial intelligence chip or a general-purpose graphics processor (GPGPU) for efficiently processing and distributing video encoding and decoding tasks.

[0143] like Figure 8 As shown, the video task scheduling device includes an instruction receiving module, an allocation selection module, and an instruction issuing module. Furthermore, the device interacts with upper-layer software or drivers via an instruction channel.

[0144] exist Figure 8 In this context, the command channel acts as a bridge connecting the upper-layer software and the hardware scheduling device. It is responsible for receiving video task commands from the software / driver and transmitting these commands to the scheduling device. These command packets carry crucial information such as the stream ID, channel ID, a flag indicating whether it is the first frame of the stream, and the specific task to be executed.

[0145] The instruction receiving module is the entry point of the scheduling device; it receives video task instructions from the instruction channel.

[0146] The allocation selection module is the core decision-making unit. It is responsible for parsing video task instructions and determining which allocation strategy to adopt based on the first frame identification information carried in the video task instructions.

[0147] If the first frame indication information indicates that the current instruction is the first frame of the bitstream: the allocation selection module will activate the allocation submodule (in another possible embodiment, the allocation submodule may be called the Round Robin random selection module) to randomly select one from all idle video execution units for task allocation through round-robin or other load balancing algorithms. Simultaneously, the ID of the selected video execution unit, along with its corresponding bitstream ID and channel ID, will be stored in the recording submodule (in another possible embodiment, the recording submodule may be called the Core ID recording module) for use in subsequent frames.

[0148] If the first frame indication information indicates that the current instruction is not the first frame of the bitstream: the allocation selection module will find the video execution unit ID bound to the bitstream by indexing the previously stored mapping relationship in the recording submodule based on the bitstream ID and channel ID in the instruction.

[0149] The instruction issuance module is the exit point of the scheduling device. It distributes video task instruction packets to the corresponding video execution units based on the target video execution unit ID determined by the allocation and selection module (regardless of whether the target video execution unit ID is obtained through random selection or indexing).

[0150] The Video Core is the hardware module that actually performs video encoding and decoding tasks. It receives instructions from the instruction issuing module, completes the encoding and decoding operations, and sends a task completion signal to the scheduling device upon completion, so that the video task scheduling device can update its status and perform subsequent task allocation.

[0151] This hardware framework enables real-time perception and dynamic allocation of the video execution unit status through hardware-based task scheduling, thereby effectively avoiding resource waste and improving scheduling efficiency and the utilization rate of the video execution unit.

[0152] It should be noted that the video task scheduling device provided by the present invention can execute the video task scheduling method of any of the above embodiments during specific operation, which will not be described in detail in this embodiment.

[0153] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 9As shown, the electronic device may include a processor 910, a communication interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute the video task scheduling methods provided in the above embodiments.

[0154] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0155] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is able to execute the video task scheduling method provided in the above embodiments.

[0156] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the video task scheduling method provided in the above embodiments.

[0157] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0158] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video task scheduling method, characterized in that, The execution entity of the video task scheduling method is an independent hardware circuit unit integrated in the graphics processor; the method includes: Receive video task instructions, the video task instructions including data stream identifier and first frame indication information, the data stream identifier including bitstream identifier and channel identifier; When the first frame indication information indicates that the current instruction is the first frame of the bitstream, the real-time status of multiple video execution units is directly read through the hardware signal line, and the target execution unit is selected from the video execution units in the idle state. Establishing the mapping relationship between the data stream identifier and the target execution unit includes: using hardware logic circuits, according to a preset bit width rule, performing binary bit-level concatenation or combination operations on the code stream identifier and the channel identifier to obtain the addressing address of the hardware storage space, so that each unique channel and code stream combination corresponds to a unique physical storage unit in the storage space; and writing the number of the target execution unit as data content into the storage unit corresponding to the addressing address. The video task instruction is sent to the target execution unit.

2. The video task scheduling method according to claim 1, characterized in that, After receiving the video task instructions, it also includes: When the first frame indication information indicates that the current instruction is not the first frame of the code stream, the pre-established mapping relationship is retrieved according to the data stream identifier to determine the target execution unit bound to the data stream identifier; The video task instruction is sent to the target execution unit bound to the data stream identifier.

3. The video task scheduling method according to any one of claims 1-2, characterized in that, The data stream identifier includes a stream identifier and a channel identifier; Establishing the mapping relationship between the data stream identifier and the target execution unit includes: The bitstream identifier and the channel identifier are concatenated or combined to obtain the address of the hardware storage space. The number of the target execution unit is used as data content and written into the storage unit corresponding to the address.

4. The video task scheduling method according to claim 1, characterized in that, The step of selecting a target execution unit from video execution units in an idle state includes: Traverse all video execution units and detect the current real-time status flag of each video execution unit; The video execution units whose real-time state is marked as the idle state are identified as a candidate set; According to a preset cyclic order, a video execution unit is sequentially determined from the candidate set as the target execution unit; wherein, when the previously assigned video execution unit is the Nth in the cyclic order, the N+1th video execution unit in the cyclic order is preferentially determined to be a candidate set.

5. The video task scheduling method according to claim 4, characterized in that, After sending the video task instruction to the target execution unit, the method further includes: Update the real-time status flag of the target execution unit from the idle state to the busy state; When a video task instruction indicating the first frame of the bitstream is received again, the video execution unit in the busy state will be excluded from the candidate set.

6. The video task scheduling method according to claim 4, characterized in that, Also includes: Receive the task completion signal fed back by the target execution unit after completing the video task; In response to the task completion signal, the real-time status flag of the target execution unit is reset from the busy state to the idle state, so that the target execution unit can be reinstated into the candidate set.

7. The video task scheduling method according to claim 1, characterized in that, Also includes: When it is detected that the real-time status of all video execution units is marked as busy, the dispatch operation of the current video task instruction is suspended. The real-time status is continuously monitored until the status of at least one of the video execution units changes to the idle state.

8. A video task scheduling device, characterized in that, include: The instruction receiving module is used to receive video task instructions, which include data stream identifiers and first frame indication information. The data stream identifiers include bitstream identifiers and channel identifiers. The allocation selection module is used to directly read the real-time status of multiple video execution units through hardware signal lines when the first frame indication information indicates that the current instruction is the first frame of the bitstream, and select the target execution unit from the video execution units that are in an idle state. The instruction issuing module is used to establish the mapping relationship between the data stream identifier and the target execution unit, including: using hardware logic circuits, according to a preset bit width rule, performing binary bit-level concatenation or combination operations on the code stream identifier and the channel identifier to obtain the addressing address of the hardware storage space, so that each unique channel and code stream combination corresponds to a unique physical storage unit in the storage space; and writing the number of the target execution unit as data content into the storage unit corresponding to the addressing address. The instruction issuing module is also used to send the video task instruction to the target execution unit.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the video task scheduling method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the video task scheduling method as described in any one of claims 1 to 7.

Images