Scalable hardware thread scheduler

Abstract

A device includes a hardware data processing node configured to perform corresponding tasks, and a hardware thread scheduler including a hardware task scheduler (300). The hardware task scheduler (300) is coupled to the hardware data processing node and has a producer socket (304), a consumer socket (302), and a spare socket (306, 307). The spare socket is configured to provide data control signals, also provided by the first socket of the producer and consumer sockets, in response to a first value of the memory-mapped register (224). The spare socket is configured to provide data control signals, also provided by the second socket of the producer and consumer sockets, in response to a second value of the memory-mapped register (224).

Description

BACKGROUND

[0001] Various systems use hardware accelerators, optionally in combination with software executing on programmable processors (e.g., digital signal processors and general purpose processors), to perform data processing. Hardware accelerators can perform certain operations more efficiently than software executing on such processors. A hardware scheduler dispatches execution of tasks of a thread through the hardware accelerators. Due to the nature of the hardware scheduler being fixed in hardware, it is difficult or impossible to adjust the hardware scheduler and hardware accelerators for use cases that were not known or considered at design time. SUMMARY

[0002] In one example, a device includes hardware data processing nodes configured to perform respective tasks, and a hardware thread scheduler including a hardware task scheduler. The hardware task scheduler is coupled to the hardware data processing nodes and has a producer socket, a consumer socket, and a spare socket. The spare socket is configured to provide a data control signal that is also provided by a first one of the producer and consumer sockets in response to a memory-mapped register being a first value. The spare socket is configured to provide a data control signal that is also provided by a second one of the producer and consumer sockets in response to the memory-mapped register being a second value.

[0003] In another example, a device includes hardware data processing nodes configured to perform respective tasks, and a hardware thread scheduler including a hardware task scheduler. The hardware task scheduler is coupled to the hardware data processing nodes and has a plurality of producer sockets. The device also includes a spare scheduler having a consumer socket coupled to one of the plurality of producer sockets of the hardware task scheduler, and a plurality of producer sockets configured to provide data control signals from the consumer socket.

[0004] In yet another example, a method includes configuring, by scheduling software executing on a processor, a hardware thread scheduler to perform tasks of a thread. The hardware thread scheduler includes a hardware task scheduler coupled to hardware data processing nodes configured to perform one of the tasks of the thread. The method also includes configuring, by the scheduling software, a spare socket of the hardware task scheduler to provide a data control signal that is also provided on one of a plurality of producer sockets or a plurality of consumer sockets of the hardware task scheduler. BRIEF DESCRIPTION OF DRAWINGS

[0005] For a detailed description of various examples, reference will now be made to the accompanying drawings in which:

[0006] Figure 1 is a block diagram of a multi-processor system-on-a-chip (SOC) in some examples;

[0007] Figure 2 is a block diagram of a multi-processor system-on-a-chip (SOC) in some examples;Figure 1 a block diagram of a hardware task scheduler with spare sockets in some examples;

[0008] Figure 3 a block diagram of a hardware task scheduler with spare sockets in some examples;

[0009] Figure 4 a block diagram of a hardware task scheduler coupled to a spare scheduler in some examples;

[0010] Figure 5 a block diagram of a hardware task scheduler coupled to a chain of spare schedulers in some examples; and

[0011] Figure 6 a flowchart of a method for configuring a hardware accelerator in some examples. DETAILED DESCRIPTION

[0012] As noted above, various systems use hardware accelerators, optionally in conjunction with software executing on programmable processors (e.g., digital signal processors and general purpose processors), to perform data processing. In one case, embedded safety systems, such as advanced driver assistance systems (ADAS), are used in automobiles to reduce human operational errors. These systems provide various functionality, such as rear-facing cameras, electronic stability control, and vision-based pedestrian detection systems. These systems perform computer vision processing on images captured by one or more cameras to detect objects within the field of view of the one or more cameras. The vision processing includes, for example, image processing, lens distortion correction, noise filtering, edge detection, motion detection, image scaling, etc.

[0013] Examples of the present description generally relate to data processing systems, which can include line- and / or block-based data processing systems (including ADAS that perform vision processing). In particular, examples of the present description include block-based data processing systems having hardware task schedulers with one or more configurable spare sockets. In some examples, the systems also include a spare scheduler that is not directly coupled to the hardware processing nodes and is configured to expand one consumer socket (e.g., of another hardware task scheduler) to multiple producer sockets. Thus, data provided by one producer socket of another hardware task scheduler is provided to or made available to multiple consumers via the multiple producer sockets of the spare scheduler. In some examples, the spare scheduler is also configured to replicate data (e.g., from one of its consumer sockets to its multiple producer sockets). These examples are described in detail below.

[0014] Figure 1This is a block diagram of an example multiprocessor system-on-a-chip (SOC) 100 configured to perform box-based processing, such as computer vision processing in camera-based ADAS. SOC 100 includes a general-purpose processor (GPP) 102, a digital signal processor (DSP) 104, a vision processor 106, and a vision preprocessing accelerator (VPAC) 112 coupled via a high-speed interconnect 122. See reference... Figure 2 Further described, VPAC 112 includes a hardware accelerator configured to perform various preprocessing operations on incoming camera images. In one example, vision processor 106 is a vector processor configured for computer vision processing such as gradient computation, orientation binning, histogram normalization, etc. In one example, the computer vision processing performed by vision processor 106 uses the preprocessed output of VPAC 112. GPP 102 hosts the operating system and provides overall control over the operation of SOC 100, including the scheduling of preprocessing tasks performed by VPAC 112. In this example, DSP 104 provides support for computer vision processing such as object detection and classification.

[0015] The SOC 100 also includes a direct memory access (DMA) component 108, a camera capture component 110 coupled to the camera 124, a display management component 114, on-chip random access memory (RAM) 116 (e.g., non-transitory computer-readable medium), and various input / output (I / O) peripherals 120, all of which are coupled to the processor and VPAC 112 via interconnect 122. Furthermore, in some examples, the SOC 100 includes a security component 118 that includes safety-related functions to enable compliance with automotive safety requirements. These functions may include cyclic redundancy check (CRC) for supporting data, a clock comparator for drift detection, error signaling, a windowed watchdog timer, and a damage and failure self-test for the SOC 100.

[0016] Figure 2 This is a block diagram of example VPAC 112. VPAC 112 includes a hardware thread scheduler 200 coupled to example hardware accelerators 202, 204. Hardware accelerators 202, 204 are coupled to shared memory 206. For ease of explanation, two hardware accelerators 202, 204 are shown, and other examples include more or fewer accelerators. Hardware accelerators may be, for example, lens distortion correction accelerators, image scaling accelerators, noise filtering and / or vision-specific image processing accelerators. Groups of storage regions in shared memory 206 may be designated as buffers for data blocks processed by hardware accelerators 202, 204.

[0017] The hardware thread scheduler 200 is also coupled to two channels of the DMA 250. For ease of explanation, two DMA channels are shown, and in other examples, the hardware thread scheduler is coupled to more or fewer DMA channels. In one example, the DMA channels are configured to move blocks of data between shared memory 206 and external memory (e.g., RAM 116).

[0018] The hardware thread scheduler 200 is configurable to schedule the execution of a single task thread or multiple concurrent task threads via hardware processing nodes (“nodes”) of the VPAC 112. A thread (also called a pipeline) is one or more tasks. A task is a specific function performed by a node, and a node executes a single task. In various examples, a node is a hardware accelerator configured to perform a single task, part of a hardware accelerator configured to perform a task, a channel of DMA 250, or software implementing the task on a processor (e.g., DSP 104) external to the VPAC 112. Furthermore, the hardware thread scheduler 200 includes hardware task schedulers, each of which manages the execution of tasks on a node in a dedicated manner.

[0019] Examples of hardware accelerators configured to perform a single task (e.g., hardware accelerator 202) include noise filtering accelerators and lens distortion correction accelerators. Examples of hardware accelerators configured to perform tasks (e.g., hardware accelerator 204) are image scaling accelerators configurable to perform image scaling on two or more images simultaneously. Therefore, an image scaling accelerator may include multiple scalers that can be configured to perform multiple concurrent scaling tasks. An example of such an image scaler is described in U.S. Patent Application Serial No. 15 / 143,491, filed April 29, 2016.

[0020] In this example, VPAC 112 includes five nodes: a single-task hardware accelerator 202 (one node), a dual-task hardware accelerator 204 (two nodes), and two DMA channels. Hardware thread scheduler 200 includes hardware task schedulers 208, 210, and 212 for each hardware accelerator node in VPAC 112, DMA hardware task schedulers 214 and 216 for each channel of DMA 250 used by VPAC 112, and two proxy hardware task schedulers 218 and 220 for two external nodes (e.g., two software tasks implemented on an external processor). Task schedulers 208-220 are coupled to scheduler crossbar 222, which is configured by various memory-mapped registers in memory-mapped register 224 to link task schedulers to create threads. In one example, scheduler crossbar 222 facilitates communication of control information between task schedulers assigned to threads. In some examples, scheduler crossbar 222 is a single, uninterrupted crossbar where any task scheduler can be linked to any other task scheduler. In other examples, scheduler lever 222 is a partial lever with more limited linking capabilities. In some examples, the hardware thread scheduler 200 also includes a backup scheduler 226 that is not directly coupled to the hardware processing node. The backup scheduler 226 is described in further detail below.

[0021] Typically, the thread scheduling of the hardware thread scheduler 200 follows a consumer / producer model. For example, a thread is a set of tasks with consumer / producer dependencies. Tasks / nodes managed by the task scheduler can be consumer tasks that consume data from one or more producer tasks, producer tasks that produce data for one or more consumer tasks, or a combination of consumer and producer tasks. Depending on the task type of the associated node, the task scheduler includes one or more producer and / or consumer sockets.

[0022] For example, if a task scheduler is coupled to a node that executes consumer tasks that consume data from a producer task, then the task scheduler has at least a single consumer socket. If a consumer task consumes data from more than one producer task, then the task scheduler has at least one consumer socket for each producer task.

[0023] If a task scheduler is coupled to a node that executes a producer task that produces data for a single consumer task, then the task scheduler has at least a single producer socket. If a producer task produces data for more than one consumer task, then the task scheduler has at least one producer socket for each consumer task.

[0024] If a task scheduler is coupled to a node that executes consumer / producer tasks, then the task scheduler has at least one consumer socket for each producer task that produces data for the consumer / producer tasks, and at least one producer socket for each consumer task that consumes data from the consumer / producer tasks.

[0025] In VPAC 112, for simplicity, task schedulers 208, 210, and 212 are described as having two consumer sockets and two producer sockets. In other examples, the number of producer and consumer sockets in the task scheduler varies depending on the consumption and production attributes of the tasks / nodes coupled to the task scheduler as described above. Each node corresponding to a DMA channel performs a task that consumes data (e.g., a consumer channel programmed to transfer data from shared memory 206 to external memory) or produces data (e.g., a producer channel programmed to transfer data from external memory (as relative to VPAC 112) to shared memory 206). DMA task scheduler 216 is coupled to a consumer channel and includes a single consumer socket. DMA task scheduler 214 is coupled to a producer channel and includes a single producer socket.

[0026] Each socket of task schedulers 208-220 is coupled to scheduler bar 222 via two signals: a suspend signal indicating the availability of consumable data and a decrement signal indicating that a block of produced data has been consumed. The suspend signal is referred to herein as the "pend" signal, and the decrement signal as the "dec" signal. When data is available, the task scheduler sends the suspend signal to the consumer socket coupled to the producer socket via the producer socket, and receives the decrement signal from the coupled consumer socket via the producer socket when the produced data has been consumed. When the produced data has been consumed, the task scheduler sends the decrement signal to the producer socket coupled to the consumer socket via the consumer socket, and receives the suspend signal from the coupled producer socket at the consumer socket when data is available for consumption. In some examples, a spare socket, such as a spare producer socket or a spare consumer socket, is configured to replicate data control over a buffer containing data (e.g., in shared memory 206). In one example, control over a buffer containing example data rows branches between the producer socket and a spare producer socket configured for that buffer. Therefore, the data rows in the buffer will not be overwritten until the producer socket and the backup producer socket receive decrement signals from the consumer sockets coupled to them. In another example, control of the buffer containing the example data rows branches between the consumer socket and the backup consumer socket configured for that buffer. Thus, in response to the data rows in the buffer being consumed (e.g., via an associated hardware accelerator), each of the consumer socket and the backup consumer socket sends a decrement signal to the producer socket coupled to it.

[0027] The connection of suspend and decrement signals between the producer and consumer sockets of the task scheduler that forms a thread is controlled by scheduler lever 222. In some examples, the scheduling software executing on GPP 102 configures a threaded task to be executed on VPAC 112 by setting the control signal value of the multiplexer in scheduler lever 222, thereby "connecting" the suspend and decrement signals of the task scheduler to the required task.

[0028] In some examples, the scheduler lever 222 includes a multiplexer for incoming pending signals for each consumer outlet and a multiplexer for incoming decrement signals for each producer outlet. For example, if the task schedulers 208-220 include M consumer outlets and N producer outlets, then the scheduler lever 222 includes M+N multiplexers. The multiplexer for the incoming decrement signals coupled to the producer outlets includes a single output coupled to the incoming decrement signal and M inputs, one input for each outgoing decrement signal for each consumer outlet. The multiplexer for the incoming pending signals coupled to the consumer outlets includes a single output coupled to the incoming pending signal and N inputs, one input for each outgoing pending signal for each producer outlet. Each multiplexer is coupled to a corresponding control register in MMR 224, which can be programmed to select one of the multiplexer inputs as an output.

[0029] Each of the task schedulers 208-220 is coupled to the corresponding node via various signals. These signals can be, for example, a node initialization signal, an initialization completion acknowledgment signal, a task start signal, a task completion signal, and a processing end signal. The processing end signal is useful because a task may be executed multiple times to process incoming data (e.g., a task may be executed multiple times to process different subsets of a video data block). Nodes / tasks know when all data has been processed and use the processing end signal to indicate that all data has been processed and that the task does not need to be executed again. Whenever the task scheduler receives a task completion signal, it will restart the task unless a processing end signal is received.

[0030] MMR 224 is configured to store various control and configuration parameters for the hardware thread scheduler 200. These parameters include those for configuring and controlling threads, as well as those for configuring and controlling task schedulers 208-220, one or more standby schedulers 226, and configurable standby sockets for task schedulers 208-220. In some examples, MMR 224 includes thread control registers for each of the maximum number of threads that can execute concurrently on VPAC 112. Each thread control register corresponds to a specific number of threads, for example, 0, 1…n-1, where n is the maximum number of threads. The thread control registers include enable / disable bits that can be used to activate or deactivate the corresponding thread. Furthermore, MMR 224 includes task scheduler control registers for each task scheduler 208-220. These task scheduler control registers include enable / disable bits that can be used to activate or deactivate the corresponding task scheduler, and fields identifying the number of threads to which the task scheduler is assigned.

[0031] In some examples, the MMR 224 also includes a consumer control register for each consumer outlet and a producer control register for each producer outlet. The consumer control register includes enable / disable bits for activating or deactivating the corresponding consumer outlet, and a producer selection value for the multiplexer coupled to the consumer outlet. The consumer control register for consumer outlets in the agent task scheduler also includes a "suspend" bit, which is set by an external task to indicate when data is available for consumption.

[0032] The producer control register includes enable / disable bits for activating or deactivating the corresponding producer socket, and consumer selection values ​​for the multiplexer coupled to the producer socket. The producer control register for the producer socket in the agent task scheduler also includes a "decrement" bit, which is set by an external task to indicate that a data block has been consumed.

[0033] The MMR 224 also includes a producer buffer control register and a producer count register for each producer socket. The producer buffer control register includes a buffer depth field that specifies the maximum number of data blocks a producer can have to consume, and the producer count register includes a count field that stores a count of the number of data blocks a producer currently has to consume. The value of the buffer depth field depends on the amount of shared memory allocated to store the produced data blocks and the size of the data blocks. The combination of the maximum buffer depth and the count can be used to prevent buffer overflows and underflows, as data production and consumption may be asynchronous.

[0034] In the examples described herein, the scheduling software executing on GPP 102 configures the threads to be executed on VPAC 112 by writing appropriate values ​​to the registers of task schedulers 208-220 to include them in the threads. In these examples, the scheduling software writes appropriate producer selection and consumer selection values ​​to the sockets of the task schedulers 208-220 for the tasks to include them in the threads to synchronize the tasks and enable the sockets. The scheduling software also appropriately sets enable / disable bits in the task scheduler control register of each task scheduler in the thread to enable each task scheduler. In these examples, the scheduling software also selects the number of threads for the thread and writes that number to the task scheduler register of each task scheduler in the thread. Once a thread is configured, the scheduling software is configured to activate the thread by appropriately setting the enable / disable bits in the corresponding thread control register to initiate the execution of the thread.

[0035] In addition to the above, in the examples described herein, the scheduling software configures MMR 224 to enable the hardware task scheduler's backup producer socket to provide additional data control (e.g., sending suspend signals and / or receiving decrement signals) for buffers generated by the associated hardware accelerators (e.g., in addition to the producer socket configured for the buffer to enable copying or stealing data in the buffer). In other examples, the scheduling software configures MMR 224 to enable the hardware task scheduler's backup consumer socket to provide additional data control (e.g., sending decrement signals and / or receiving suspend signals) for buffers consumed by the associated hardware scheduler (e.g., in addition to the consumer socket configured for the buffer). Configurable backup sockets utilize the additional programmable connectivity of the hardware task scheduler to handle various use cases not considered in the hardware design. In some examples, the scheduling software also configures MMR 224 to specify coupling between the hardware task scheduler's producer socket and the backup scheduler's consumer socket. Unlike the hardware task scheduler, the backup scheduler is not directly coupled to the hardware processing node. In other examples, additional backup schedulers can be configured similarly. In these examples, arbitrary dataflow graphs can be configured by the scheduling software to address use cases determined after hardware design time. Furthermore, from a hardware design perspective, implementing a backup scheduler chain to extend (e.g., extend to additional consumer tasks) the data generated at the hardware task scheduler's producer sockets may be less costly than implementing additional producer sockets within the hardware task scheduler itself. In other examples, as an alternative to configuring MMR 224 to provide additional data control for backup producer or consumer sockets, or to specify coupling with a backup scheduler, this configuration can be implemented via scheduling software or other programming (e.g., via firmware or hardware) to bind one or more configuration register values ​​to achieve the desired configuration of the backup producer socket, backup consumer socket, and / or backup scheduler.

[0036] exist Figure 2 In the examples, hardware task schedulers 208, 210, and 212 are shown and described as each having two consumer sockets and two producer sockets. However, as mentioned above, in other examples, the hardware task scheduler includes a different number of consumer and / or producer sockets. Furthermore, the hardware task scheduler is typically configured to handle various input / output scenarios, and / or possible combinations or arrangements of consumer and / or producer sockets.

[0037] In one scenario, for illustrative purposes, a hardware task scheduler is coupled to a Visual Imaging Subsystem (VISS) hardware accelerator. This scheduler is referred to as the VISS scheduler, and the accelerator as the VISS accelerator. The VISS accelerator is configured to consume or process data from up to three input sources (e.g., three exposures from a raw image sensor) to generate and provide six output planes (e.g., Luma-Y 12-bit (Y12), Chroma-UV 12-bit (UV12), Luma-Y 8-bit (Y8), Chroma-UV 8-bit (UV8), Saturation-S 8-bit (S8), and Statistics (H3A) outputs in response to the consumed data). Furthermore, in this example, the VISS scheduler is configured to process 32 scenes, or permutations of data input and output combinations. Therefore, a VISS scheduler implemented in hardware would actually require 96 consumer sockets (e.g., 3 inputs * 32 scenes) and 192 producer sockets (e.g., 6 outputs * 32 scenes).

[0038] However, in the above scenario, the VISS scheduler is limited to consuming data produced by three producer nodes and generating data to six consumer nodes. Therefore, the possible graphs that can be implemented by the thread scheduler are limited to those use cases known at the time of hardware design. Furthermore, as mentioned above, since the number of sockets is proportional to the number of possible multi-producer and multi-consumer scenarios, it is expensive to design a hardware task scheduler with an "oversized" number of consumer and / or producer sockets.

[0039] To address the aforementioned issues, examples described herein include a hardware task scheduler providing additional or backup producer outlets configured to provide additional data control for a buffer, wherein the data control is also provided by another producer outlet. Other examples described herein include a hardware task scheduler providing additional or backup consumer outlets configured to provide additional data control for a buffer, wherein the data control is also provided by another consumer outlet.

[0040] For example, the spare socket is a spare producer socket that provides data control for a first buffer in response to MMR 224 with a first value, where the data control is also provided by the first producer socket of the hardware task scheduler. Continuing this example, the spare producer socket provides data control for a second buffer, where the data control is also provided by the second producer socket of the hardware task scheduler in response to MMR 224 with a second value. Therefore, the spare producer socket enables additional programmable connectivity for the hardware task scheduler, such as to handle post-development use case diagrams or use cases not considered in the hardware design. For example, the hardware task scheduler might be configured to produce data from the buffer to additional consumer tasks, which would not be possible using the originally designed set of producer sockets.

[0041] In some examples, the hardware task scheduler is provided with multiple backup producer sockets that can be configured by MMR 224 as described above. For example, in response to a first MMR 224 having a first value, the first backup producer socket is configured to buffer and provide data control, wherein data control is also provided by the first producer socket (e.g., Y12) of the hardware task scheduler. In response to a first MMR 224 having a second value, the first backup socket is configured to buffer and provide data control, wherein data control is also provided by a second producer socket (e.g., UV12). The second backup producer socket plays a similar role. For example, in response to a second MMR 224 having a first value, the second backup producer socket is configured to buffer and provide data control, wherein data control is also provided by the first producer socket (e.g., Y12) of the hardware task scheduler. In response to a second MMR 224 having a second value, the second backup producer socket is configured to buffer and provide data control, wherein data control is also provided by the second producer socket (e.g., UV12).

[0042] In another example, the backup outlet is a backup consumer outlet that enables the hardware task scheduler to accept data from additional producer tasks (or from multi-producer tasks). Continuing with the VISS example above, the backup consumer outlet enables the VISS scheduler to consume or process data from an additional fourth input source (e.g., an additional DMA channel or an additional producer task). The backup consumer outlet also enables the VISS scheduler to consume or process data from multi-producer tasks (such as tasks that generate two such input sources, e.g., exposures from the raw image sensor). Therefore, the backup consumer outlet enables additional programmable connectivity for the hardware task scheduler, such as to handle post-development use case diagrams or use cases not considered in the hardware design.

[0043] For example, the hardware task scheduler is configured to consume data from additional producer tasks, which would be impossible using the originally designed set of producer sockets. Furthermore, in some examples, the hardware task scheduler is provided with multiple alternative consumer sockets. In these examples, the hardware accelerator (e.g., a VISS accelerator) can itself be software-programmable to utilize additional inputs provided by the alternative consumer sockets. For example, the VISS accelerator is initially configured to process three exposures from the raw image sensor, but is software-programmable to process additional inputs, such as a fourth exposure (e.g., from an additional DMA channel). In these examples, the producer of such additional inputs for the alternative consumer sockets is specified by the alternative consumer MMR 224. For example, in response to an alternative consumer MMR 224 having a first value, the alternative consumer socket consumes data from a first producer. In response to an alternative consumer MMR 224 having a second value, the alternative consumer socket consumes data from a second producer.

[0044] Figure 3 This is a block diagram of a hardware task scheduler 300 (such as the VISS scheduler described above) in some examples. As mentioned above, the associated hardware accelerator (e.g., the VISS accelerator) is configured to consume or process data from three input sources to generate and provide six outputs (e.g., Y12, UV12, Y8, UV8, S8, H3A outputs). In this example, the hardware task scheduler 300 is configured to be coupled to a node (e.g., the VISS accelerator) that performs consumer / producer tasks, and therefore the hardware task scheduler 300 includes consumer sockets 302 and producer sockets 304. For simplicity, the hardware task scheduler 300 is shown as including three consumer sockets 302 and six producer sockets 304. As mentioned above, the actual number of consumer sockets 302 and producer sockets 304 is determined by multiplying the number of inputs by the number of possible scenarios that the hardware task scheduler 300 is configured to process.

[0045] In some examples, the hardware task scheduler 300 also includes a configurable backup producer socket 306 and / or a configurable backup consumer socket 307. MMR 224 designates one of the data buffers for which backup producer socket 306 and / or backup consumer socket 307 provides data control. For example, an MMR 224 with a first value causes backup producer socket 306 to provide data control for the Y12 buffer / output (e.g., sending a suspend signal and / or receiving a decrement signal), while an MMR 224 with a second value causes backup producer socket 306 to provide data control for the UV12 buffer / output, etc. Similarly, backup consumer socket 307 provides data control for a first buffer (e.g., sending a decrement signal and / or receiving a suspend signal) in response to an MMR 224 with a first value, and provides data control for a second buffer in response to an MMR 224 with a second value.

[0046] Therefore, during hardware design, the hardware task scheduler 300 is configured to receive three inputs from external memory (e.g., via DMA 250) at its consumer socket 302 and produce six outputs to external memory (e.g., via DMA 250) at its producer socket 304. However, additional use cases are identified after hardware design time, which also includes scaling on the Y12 data outputs. Therefore, the scheduling software executing on GPP 102 configures MMR 224 to set data control settings for the Y12 buffer / output on the standby socket 306, which is provided to the multi-scaler or resizing accelerator (whose scheduler is MSC-SCH) to perform scaling on the Y12 data outputs (in addition to producing six raw data outputs to DMA 250). Thus, in an example where one consumer (e.g., DMA) consumes Y12 data in a shorter time than the other consumer (e.g., MSC-SCH), the HWA scheduler 300 may not overwrite the buffer until a decrement signal is received at both the Y12 producer socket and the standby producer socket 306. Therefore, the backup producer outlet 306 enables additional programmable connectivity for the hardware task scheduler 300 to handle various use cases not considered in the hardware design. In other examples, the backup consumer outlet 307 similarly enables additional programmable connectivity for the hardware task scheduler 300 to handle various use cases not considered in the hardware design.

[0047] As described above, in some examples, the hardware task scheduler 300 is provided with multiple backup producer sockets 306, each socket 306 which can be configured by MMR 224 to provide data control for a buffer that is also provided by another producer socket. For example, the scheduling software executing on GPP 102 configures MMR 224 to set data control for the Y12 buffer / output on the first backup socket 306, which is provided to the MSC-SCH. The scheduling software also configures MMR 224 to set data control for the Y12 buffer / output or another buffer / output on a second backup socket, which is provided to another consumer task. Thus, multiple backup producer sockets 306 enable further programmable connectivity for the hardware task scheduler 300 to handle various use cases not considered in the hardware design.

[0048] In some examples, the configuration of the backup producer socket 306 is modified by scheduling software during the operation of SOC 100. For example, the scheduling software configures MMR 224 to a first value during a first time period (e.g., to set data control for the Y12 buffer / output on the backup socket 306), and then updates MMR 224 to a second value during a second time period. Therefore, the backup socket 306 transitions from providing data control for the Y12 buffer / output during the first time period to providing data control for another buffer / output (e.g., UV12) during the second time period.

[0049] Figure 4 It is coupled to the backup scheduler 400. Figure 3 A block diagram of the hardware task scheduler 300 is provided. In some examples, the standby scheduler 400 is similar to the standby scheduler 226 described above. As mentioned above, the standby scheduler 400 is not directly coupled to hardware processing nodes, such as hardware accelerators. The standby scheduler 400 has consumer sockets 402 and multiple producer sockets 404. Although not described in... Figure 4As shown, the backup scheduler 400 may also include configurable backup outlets, as described above. The backup scheduler 400 is configured to provide data control to its multiple producer outlets 404 via its consumer outlet 402 (e.g., sending a suspend signal), and to provide data control from its producer outlets 404 to its consumer outlets 402 (e.g., receiving a decrement signal). In this example, the backup scheduler 400 includes two producer outlets 404, although in other examples the backup scheduler 400 includes more than two producer outlets 404. Furthermore, while the backup scheduler 400 is shown with a single consumer outlet 402 and thus implements one or more expansion schemes, in other examples the backup scheduler 400 includes more than one consumer outlet 402 and implements a combination of one or more expansion schemes (e.g., two consumer outlets 402 and four producer outlets 404 to implement the first and second one or more expansion schemes).

[0050] In another example, another standby scheduler 410 is also not directly coupled to hardware processing nodes, such as hardware accelerators. The standby scheduler 410 has multiple consumer sockets 412 and producer sockets 414. Although not in... Figure 4 As shown, however, the backup scheduler 410 may also include configurable backup outlets, such as those described above. The backup scheduler 410 is configured to provide data control to its producer outlet 414 via its consumer outlet 412 (e.g., to receive a suspend signal), and to provide data control from its producer outlet 414 to its consumer outlet 404 (e.g., to send a decrement signal).

[0051] As relative to Figure 3 The description states that the spare socket 306 of the hardware task scheduler 300 (e.g., via the MMR224 value) is configured to provide data control for the Y12 buffer / output via the spare producer socket 306. Figure 4 In the example, after the hardware design time, which includes providing data control from the spare socket 306 to more than one consuming node, additional use cases are identified. For example, in addition to providing Y12 data output to a multi-scaler or resizing accelerator (MSC-SCH), additional use cases include using a lens distortion correction (LDC) accelerator (or another accelerator not specifically listed) to perform lens distortion correction on the Y12 data.

[0052] Therefore, scheduling software configuration MMR 224 is performed on GPP 102 to provide data control for the Y12 buffer / output on the standby producer socket 306. Furthermore, the consumer socket 402 of the standby scheduler 400 is configured to receive data control signals (e.g., suspend signals) generated at the standby socket 306. The producer socket 404 of the standby scheduler 400 is configured to replicate the data control signals received at the consumer socket 402. One of the producer sockets 404 provides signals (and therefore Y12 data connections) to the MSC-SCH, while the other of the producer sockets 404 provides signals (and therefore Y12 data connections) to the LDC. Thus, as described above, the standby producer socket 306 enables additional programmable connectivity for the hardware task scheduler 300 to handle various use cases not considered in the hardware design. Additionally, in some examples, the standby scheduler 400 enables additional provisioning or replication of data received at its consumer socket 402 to its producer socket 404. In terms of hardware design, it may be less costly to implement a backup scheduler 400 to replicate the data generated at the producer socket of the hardware task scheduler 300 compared to implementing an additional producer socket in the hardware task scheduler 300 itself.

[0053] Figure 5 It is coupled to the first-level backup scheduler 400. Figure 3 A block diagram of a hardware task scheduler 300 is provided. In some examples, this first-level backup scheduler 400 is linked to an additional second-level backup scheduler 500. Similar to backup scheduler 400, backup scheduler 500 is not coupled to hardware processing nodes, such as hardware accelerators. Backup scheduler 500 also includes consumer sockets and multiple producer sockets, and its function is similar to that of backup scheduler 400 as described above. In this example, backup scheduler 400 includes three producer sockets 404, although in other examples backup scheduler 400 includes a different number of producer sockets 404.

[0054] exist Figure 5In the examples, backup schedulers 400 and 500 are linked together to facilitate the generation of arbitrary data flow graphs. For example, the scheduling software executing on GPP 102 configures MMR 224 to specify the desired coupling between hardware task scheduler 300 producer socket 304 or backup sockets 306, 307 and the first-level backup scheduler 400. The scheduling software also configures MMR 224 to specify the desired coupling between the first-level backup scheduler 400 producer socket 404 and the second-level backup scheduler 500. In some examples, the second-level backup scheduler 500 produces data to be consumed by hardware nodes. In another example, the second-level backup scheduler 500 provides an indication of data availability for distribution (e.g., by providing suspend / decrement signals as described above), such as to another hardware accelerator. In other examples, the second-level backup scheduler 500 produces data to an additional level of backup scheduler to further replicate data or data availability indications (e.g., by providing suspend / decrement signals).

[0055] In these examples, arbitrary data flow graphs can be configured via scheduling software to address use cases determined after the hardware design time. Furthermore, from a hardware design perspective, implementing a chain of backup schedulers 400, 500 to replicate or extend data control or indications of data availability generated at the producer socket of the hardware task scheduler 300 may be less costly than implementing additional producer sockets within the hardware task scheduler 300 itself.

[0056] In some examples, the configurations of backup schedulers 400 and 500 are modified by the scheduling software during the operation of the SOC 100. For example, the scheduling software configures MMR 224 to specify a first desired coupling between the hardware task scheduler 300 producer socket 304 or backup socket 306 and the first-level backup scheduler 400 (and / or between the first-level backup scheduler 400 producer socket 404 and the second-level backup scheduler 500) during a first time period. The scheduling software then updates MMR 224 to specify a second desired coupling between the hardware task scheduler 300 producer socket 304 or backup socket 306 and the first-level backup scheduler 400 (and / or between the first-level backup scheduler 400 producer socket 404 and the second-level backup scheduler 500) during a second time period. Thus, the coupling between the hardware task scheduler 300, the first-level backup scheduler 400, and the second-level backup scheduler 500 is updated from the first time period to the second time period.

[0057] Figure 6This is a flowchart of method 600 for configuring a backup outlet (e.g., backup outlet 306 described above) and a backup scheduler (e.g., backup schedulers 400 and 500 described above) in some examples. In some examples, method 600 includes steps performed by scheduling software (such as the one described above and executed by GPP 102).

[0058] Method 600 begins by configuring a hardware thread scheduler to execute a task in block 602. Specifically, the hardware thread scheduler includes a hardware task scheduler coupled to the hardware data processing node as described above. The hardware data processing node is configured to execute one task in that thread.

[0059] The method continues in block 604 by configuring a backup socket for the hardware task scheduler to provide data control signals (e.g., suspend and decrement signals) that are also provided on one of the multiple producer sockets or multiple consumer sockets of the hardware task scheduler. For example, the backup socket is a backup producer socket that, in response to MMR 224 with a first value, provides data control signals that are also provided by the first producer socket of the hardware task scheduler. Continuing this example, the backup producer socket, in response to MMR 224 with a second value, provides data control signals that are also provided by the second producer socket of the hardware task scheduler. Thus, the backup socket enables additional programmable connectivity for the hardware task scheduler, such as to handle post-development use case diagrams or use cases not considered in the hardware design. For example, the hardware task scheduler may be configured to produce data to additional consumer tasks, which would not be possible using the originally designed set of producer sockets.

[0060] The term "coupled" is used throughout this specification. This term can encompass the connection, communication, or signal path that makes the functional relationship consistent with this description possible. For example, in the first example, if device A generates a signal that controls device B to perform an operation, then device A is coupled to device B via intervention component C. Alternatively, in the second example, if intervention component C does not substantially alter the functional relationship between device A and device B such that device B is controlled by device A via a control signal generated by device A, then device A is coupled to device B via intervention component C.

[0061] A device “configured” to perform a task or function can be configured (e.g., programmed and / or hardwired) by the manufacturer to perform that function, and / or can be user-configurable (or reconfigurable) after manufacturing to perform that function and / or other additional or alternative functions. This configuration can be through firmware and / or software programming of the device, through the construction and / or layout of the device’s hardware components and interconnects, or a combination thereof.

[0062] This document describes how circuits or devices including certain components can be modified to be coupled to those components to form the described circuit or device. For example, a structure described as including one or more semiconductor elements (e.g., transistors), one or more passive elements (e.g., resistors, capacitors, and / or inductors), and / or one or more sources (such as voltage sources and / or current sources) may include only semiconductor elements within a single physical device (e.g., a semiconductor die and / or integrated circuit (IC) package) and may be adapted to be coupled to at least a portion of the passive elements and / or the respective sources to form the described structure at the time of manufacture or after the time of manufacture (e.g., by an end user and / or a third party).

[0063] While this document may describe certain components using a particular process technology, these components may be interchanged with components using other process technologies. The circuits described herein can be reconfigured to include the replaced components to provide at least partially similar functionality to that available before the component replacement. Unless otherwise stated, components shown as resistors generally represent any one or more elements coupled in series and / or parallel to provide the impedance represented by the resistor shown. For example, a resistor or capacitor shown and described herein as a single component may be multiple resistors or capacitors coupled in parallel between the same nodes. For example, a resistor or capacitor shown and described herein as a single component may be multiple resistors or capacitors coupled in series between the same two nodes as the single resistor or capacitor.

[0064] In the foregoing description, the term “ground voltage potential” is used to include chassis grounding, earth grounding, floating grounding, virtual grounding, digital grounding, common grounding, and / or any other form of grounding connection that may be used or adapted to the teachings of this description. Unless otherwise stated, “approximately,” “about,” or “substantially” preceding a value means + / - 10% of that value. Modifications within the scope of the appended claims are possible in the described examples, and other examples are also possible.

Claims

1. An apparatus for data processing, the apparatus comprising: hardware data processing nodes configured to perform respective tasks; and a hardware thread scheduler comprising a hardware task scheduler coupled to the hardware data processing nodes and having a producer socket, a consumer socket, and a spare socket; wherein the spare socket is configured to provide a data control signal that is also provided by a first socket of the producer socket and the consumer socket in response to a memory mapped register being a first value; and wherein the spare socket is configured to provide a data control signal that is also provided by a second socket of the producer socket and the consumer socket in response to the memory mapped register being a second value.

2. The apparatus of claim 1, wherein: the producer socket is one of a plurality of producer sockets, and the spare socket is a spare producer socket; the spare producer socket is configured to provide a data control signal that is also provided by a first producer socket of the plurality of producer sockets in response to the memory mapped register being the first value; and the spare producer socket is configured to provide a data control signal that is also provided by a second producer socket of the plurality of producer sockets in response to the memory mapped register being the second value.

3. The apparatus of claim 2, wherein: the memory mapped register is a first memory mapped register, and the hardware task scheduler is a first hardware task scheduler; the consumer socket is one of a plurality of consumer sockets, and the first hardware task scheduler comprises a spare consumer socket; the spare consumer socket is configured to provide a data control signal to a producer socket of a second hardware task scheduler in response to a second memory mapped register being a third value, and to receive a data control signal from a producer socket of a second hardware task scheduler in response to a second memory mapped register being a third value; and the spare consumer socket is configured to provide a data control signal to a producer socket of a third hardware task scheduler in response to the second memory mapped register being a fourth value, and to receive a data control signal from a producer socket of a third hardware task scheduler in response to the second memory mapped register being a fourth value.

4. The apparatus of claim 2, wherein: the spare producer socket is a first spare producer socket, and the memory mapped register is a first memory mapped register; the hardware task scheduler comprises a second spare producer socket; the second spare producer socket is configured to provide a data control signal that is also provided on the first producer socket in response to a second memory mapped register being a third value; and the second spare producer socket is configured to provide a data control signal that is also provided on the second producer socket in response to the second memory mapped register being a fourth value. ​ ​ ​ ​ 5. The device of claim 2, wherein the spare socket is configured to transition from replicated data provided on the first producer socket to replicated data provided on the second producer socket in response to updating the memory-mapped register from the first value to the second value.

6. The device of claim 2, further comprising a spare dispatcher having: a consumer socket coupled to one of the plurality of producer sockets of the hardware task dispatcher; and a plurality of producer sockets configured to provide data replicated from the consumer socket.

7. The device of claim 6, wherein the consumer socket of the spare dispatcher is coupled to the spare producer socket.

8. A device for data processing, the device comprising: hardware data processing nodes configured to perform respective tasks; a hardware thread dispatcher comprising a hardware task dispatcher coupled to the hardware data processing nodes and having a plurality of producer sockets; and a spare dispatcher having: a consumer socket coupled to one of the plurality of producer sockets of the hardware task dispatcher; and a plurality of producer sockets configured to provide data control signals from the consumer socket.

9. The device of claim 8, wherein the spare dispatcher is not directly coupled to a hardware data processing node.

10. The device of claim 8, wherein the spare dispatcher is a first level spare dispatcher, and the device further comprises: a second level spare dispatcher having: a consumer socket coupled to one of the plurality of producer sockets of the first level spare dispatcher; and a plurality of producer sockets configured to provide data control signals from the consumer socket.

11. The device of claim 10, wherein one or more memory-mapped registers are configured to specify: a first coupling between the consumer socket of the first level spare dispatcher and one of the plurality of producer sockets of the hardware task dispatcher; and a second coupling between the consumer socket of the second level spare dispatcher and one of the plurality of producer sockets of the first level spare dispatcher.

12. The device of claim 11, wherein one of the couplings is updated from a first time period to a second time period in response to an update to the one or more memory-mapped registers between the first time period and the second time period.

13. The device of claim 8, wherein: the hardware task dispatcher comprises a spare producer socket; the spare producer socket is configured to provide data control signals that are also provided on a first producer socket of the plurality of producer sockets in response to a memory-mapped register being a first value; and the spare producer socket is configured to provide data control signals that are also provided on a second producer socket of the plurality of producer sockets in response to the memory-mapped register being a second value.

14. The device of claim 13, wherein: ​ ​ ​ the memory-mapped register is a first memory-mapped register, and the hardware task scheduler is a first hardware task scheduler; the first hardware task scheduler includes a plurality of consumer sockets and a spare consumer socket; the spare consumer socket is configured to receive data control signals from a producer socket of a second hardware task scheduler in response to the second memory-mapped register being a third value; and the spare consumer socket is configured to receive data control signals from a producer socket of a third hardware task scheduler in response to the second memory-mapped register being a fourth value.

15. A method for data processing, the method comprising: configuring, by scheduling software executing on a processor, a hardware thread scheduler to execute tasks of a thread, wherein the hardware thread scheduler includes a hardware task scheduler coupled to hardware data processing nodes configured to execute one of the tasks in the thread; and configuring, by the scheduling software, a spare socket of the hardware task scheduler to provide data control signals also provided on one of a plurality of producer sockets or a plurality of consumer sockets of the hardware task scheduler by writing a value to a memory-mapped register; providing, in response to the memory-mapped register being a first value, the spare socket with data control signals also provided on a first producer socket of the plurality of producer sockets; and providing, in response to the memory-mapped register being a second value, the spare socket with data control signals also provided on a second producer socket of the plurality of producer sockets.

16. The method of claim 15, further comprising: writing, by the scheduling software, the first value to the memory-mapped register; and subsequently writing, by the scheduling software, the second value to the memory-mapped register; wherein in response to updating the memory-mapped register from the first value to the second value, the spare socket transitions from providing data control signals also provided on the first producer socket to providing data control signals also provided on the second producer socket.

17. The method of claim 15, wherein the spare socket is a first spare socket, the method further comprising configuring, by the scheduling software, a second spare socket of the hardware task scheduler to provide data control signals also provided on another of the plurality of producer sockets or the plurality of consumer sockets of the hardware task scheduler.

18. The method of claim 15, further comprising configuring, by the scheduling software, a spare scheduler consumer socket to couple to one of the plurality of producer sockets of the hardware task scheduler, wherein the spare scheduler includes a plurality of producer sockets configured to provide data control signals from the spare scheduler consumer socket, and wherein the spare scheduler does not directly couple to hardware data processing nodes.

19. The method of claim 15, wherein the hardware task scheduler is a first hardware task scheduler including a plurality of consumer sockets and a spare consumer socket, the method further comprising: By the scheduling software, the alternate consumer socket is configured by writing a value to a memory mapped register; In response to the memory mapped register being a first value, data control signals are received at the alternate consumer socket from producer sockets of a second hardware task scheduler; And In response to the memory mapped register being a second value, data control signals are received at the alternate consumer socket from producer sockets of a third hardware task scheduler.

Images