Real-time scheduling for heterogeneous multicore systems

Abstract

A heterogeneous multi-core system to execute a real-time system of an automobile includes a plurality of system-on-chips in electronic communication with one another. Each system-on-chip includes a plurality of central processing units (CPUs) arranged into a plurality of logical domains. The heterogeneous multi-core system also includes a plurality of dispatched tasks that execute based on an execution pipeline and each perform a particular set of tasks for one of the logical domains. The plurality of dispatched tasks includes at least one offset dispatched task that executes at an offset time and a reference dispatched task that is located upstream of the execution stage of the offset dispatched task on the execution pipeline. The reference dispatched task communicates data to the offset dispatched task and the offset time represents a period of time measured relative to the reference dispatched task.

Description

Technical Field

[0001] This disclosure relates to real-time scheduling of high-performance heterogeneous multicore systems in automobiles. More specifically, this disclosure relates to a heterogeneous multicore system including real-time scheduling that provides guaranteed latency and guaranteed end-to-end latency for each stage of the execution pipeline. Background Technology

[0002] Heterogeneous system-on-a-chip (SoC) can include several different types of processors and shared memory. For example, a heterogeneous SoC can include a general-purpose processor combined with hardware accelerators, memory, and input / output (I / O) devices. One type of hardware accelerator is a graphics processing unit (GPU), which can be used to rapidly render images and videos in infotainment applications or to preprocess sensor data and execute neural networks in advanced driver assistance systems (ADAS) of vehicles. High-performance heterogeneous multi-core systems with shared hardware accelerators can be used in a variety of real-time systems that perform tasks that need to be executed immediately with a certain degree of urgency. An example of a real-time safety-critical system is ADAS, which can use data from sources such as, but not limited to, cameras, radar, global positioning system (GPS), and map data to perform functions that assist the driver when operating the vehicle.

[0003] Some existing vehicle safety-critical systems are based on traditional operating systems, such as the Open Systems for Automotive Electronics (OSEK) standard and its interfaces. Traditional OSEK-based operating systems were originally written for single-core processors and do not scale well to multi-core processors. It should be understood that OSEK-based operating systems utilize a cycle-ring architecture that executes at different rates. However, these types of traditional operating systems use priority-based scheduling, where higher-rate cycle rings complete before lower-rate cycle rings and are allowed to preempt lower-rate cycle rings. Furthermore, these types of traditional operating systems do not take into account hardware accelerators such as GPUs.

[0004] Many autonomous vehicles utilize event-driven scheduling-based software stacks, such as the Robotics Operating System (ROS), which effectively supports multi-core processors, including shared hardware accelerators. However, these event-driven scheduling systems are not designed with real-time safety systems in mind, and therefore lack mechanisms to determine system latency. As a result, each step of the execution pipeline can accumulate jitter uncontrollably.

[0005] Therefore, while current scheduling systems achieve their intended purpose, there is a need in the art for a scalable and efficient scheduling system for multi-core systems, particularly for heterogeneous multi-core systems that include hardware accelerators. There is also a need in the art for a scheduling system that guarantees end-to-end latency while still preventing jitter accumulation. Summary of the Invention

[0006] According to several aspects, a heterogeneous multi-core system for executing a real-time automotive system includes multiple system-on-a-chip (SoC) devices that communicate electronically with each other. Each SoC includes multiple central processing units (CPUs) arranged in multiple logical domains. The heterogeneous multi-core system also includes multiple scheduled tasks executed based on an execution pipeline, with each scheduled task executing a specific set of tasks in one of the logical domains. The multiple scheduled tasks include at least one offset scheduled task executed at an offset time and a reference scheduled task located upstream in the execution pipeline relative to the offset scheduled task. The reference scheduled task transmits data to the offset scheduled task, and the offset time represents a time period measured relative to the reference scheduled task.

[0007] On the other hand, the offset time of the offset scheduling task is determined by selecting a predefined percentage of the total runtime of all completed tasks.

[0008] On the other hand, the execution pipeline includes guaranteed end-to-end waiting time.

[0009] On the other hand, each stage of the execution pipeline of a real-time system includes guaranteed waiting time.

[0010] On the other hand, each logical domain executes one or more scheduled tasks. Each scheduled task is executed based on a unique period.

[0011] On the other hand, each of the multiple system-on-a-chip includes a corresponding shared memory.

[0012] On the other hand, the shared memory stores a priority table that indicates the runtime priority of each of one or more scheduled tasks.

[0013] On the other hand, the runtime priority of each of one or more scheduled tasks is determined based on unique periodicity.

[0014] In one aspect, heterogeneous multicore systems also include one or more synchronizers. These synchronizers synchronize two or more inputs with different periodicities and transfer data from the two or more inputs to one of multiple scheduled tasks.

[0015] On the other hand, heterogeneous multicore systems also include one or more synchronizers. These synchronizers alter the periodicity of a single input and transfer data from that single input to one of multiple scheduled tasks.

[0016] On the other hand, each of the multiple scheduling tasks includes a unique periodicity.

[0017] On the other hand, each system-on-a-chip also includes one or more hardware accelerators shared between CPUs.

[0018] In one respect, real-time systems are Advanced Driver Assistance Systems (ADAS).

[0019] On the other hand, each CPU is assigned to one of multiple logical domains.

[0020] In one aspect, a heterogeneous multi-core system for executing a real-time automotive system includes multiple system-on-a-chip (SoC) devices that communicate electronically with each other. Each SoC includes multiple CPUs arranged in multiple logic domains. The heterogeneous multi-core system also includes one or more synchronizers. The one or more synchronizers synchronize two or more inputs with different periodicities together. The heterogeneous multi-core system also includes multiple scheduled tasks based on a specific set of tasks executed in an execution pipeline and each executing in one of the logic domains. The one or more synchronizers transmit data from two or more inputs to one of the multiple scheduled tasks. The multiple scheduled tasks include at least one offset scheduled task executed at an offset time and a reference scheduled task located upstream in the execution pipeline relative to the offset scheduled task. The reference scheduled task transmits data to the offset scheduled task, and the offset time represents a time period measured relative to the reference scheduled task.

[0021] On the other hand, the offset time of the offset scheduling task is determined by selecting a predefined percentage of the total runtime of all completed tasks.

[0022] On the other hand, the execution pipeline includes guaranteed end-to-end waiting time.

[0023] On the other hand, each stage of the execution pipeline of a real-time system includes guaranteed waiting time.

[0024] In one aspect, a heterogeneous multi-core system executing a real-time automotive system includes multiple system-on-a-chip (SoC) devices that communicate electronically with each other. Each SoC includes multiple CPUs arranged in multiple logical domains, one or more synchronizers, and multiple scheduled tasks based on an execution pipeline and each executing one of the specific task sets in one of the logical domains. The one or more synchronizers alter the periodicity of a single input and transmit data from the single input to one of the multiple scheduled tasks. The multiple scheduled tasks include at least one offset scheduled task executed at an offset time and a reference scheduled task located upstream in the execution pipeline relative to the offset scheduled task. The reference scheduled task transmits data to the offset scheduled task, and the offset time represents a time period measured relative to the reference scheduled task.

[0025] On the other hand, the offset time of the offset scheduling task is determined by selecting a predefined percentage of the total runtime of all completed tasks.

[0026] Further applications will become apparent from the description provided herein. It should be understood that the description and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0027] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure in any way.

[0028] Figure 1 This is a physical view of an exemplary computer board including multiple system-on-a-chip for a real-time system, according to an exemplary embodiment.

[0029] Figure 2 This is according to an exemplary embodiment. Figure 1 The logical view of the computer board shown;

[0030] Figure 3 An exemplary embodiment is shown. Figure 2 One of the synchronizers shown in the logical view of the computer board;

[0031] Figure 4 An execution pipeline of a real-time system according to an exemplary embodiment is shown; and

[0032] Figure 5 yes Figure 1 The schematic diagram of the computer board shown illustrates the allocated central processing unit (CPU). Detailed Implementation

[0033] The following description is merely exemplary in nature and is not intended to limit this disclosure, application, or use.

[0034] refer to Figure 1 The figure illustrates a physical view of an exemplary computer board 10. The computer board 10 includes a high-performance heterogeneous multi-core system 12 with multiple system-on-a-chip (SoC) 14. The heterogeneous multi-core system 12 executes a real-time system 16. In the non-limiting embodiment described and illustrated in the figure, the real-time system 16 is an advanced driver assistance system (ADAS) for an autonomous vehicle. Figure 1 As shown, each system-on-a-chip (SoC) 14 of the heterogeneous multi-core system 12 includes a corresponding shared memory unit 20, a shared memory device 22, multiple central processing units (CPUs) 24, and one or more hardware accelerators 26. These one or more shared hardware accelerators 26 are shared among the CPUs 24 of each SoC 14. The SoCs 14 of the computer board 10 communicate electronically with each other via communication lines 28 connected to a communication switch 30. Figure 1In the non-limiting embodiment shown, two system-on-chips 14A and 14B are illustrated; however, it should be understood that this illustration is merely exemplary in nature, and the computer board 10 may include any number of system-on-chips 14. Furthermore, although the figures depict the real-time system 16 as an ADAS for autonomous vehicles, it should be understood that the figures are merely exemplary in nature. In practice, the disclosed real-time system 16 is not limited to autonomous vehicles and can be used in a variety of other applications, such as safety-critical aerospace and medical applications.

[0035] CPU 24 is configured to execute control logic or instructions and can operate under the control of an operating system residing in shared memory unit 20. The operating system can manage computer resources, enabling computer program code implemented as one or more computer software applications (e.g., applications residing in memory) to have instructions that can be executed by CPU 24. In an alternative embodiment, CPU 24 can directly execute the application, in which case the operating system can be omitted. One or more data structures can also reside in memory and can be used by CPU 24, the operating system, or the application to store or manipulate data.

[0036] Figure 2 yes Figure 1 The logical view of computer board 10 is shown. (Reference) Figure 1 and Figure 2 In the non-limiting embodiment shown, the real-time system 16 is conducting electronic communication, and via a communication switch 30 ( Figure 1 The computer board 10 transmits and receives data from one or more computing systems that are part of the autonomous vehicle. Specifically, in the exemplary embodiment shown, the communication switch 30 of the computer board 10 is connected to the vehicle data bus 68. Figure 1 Electronic communication: The vehicle data bus 68 receives data from the camera electronic control unit (ECU) 32, radar ECU 34, inertial measurement unit (IMU) ECU 36, and global positioning system (GPS) ECU 38. The vehicle data bus 68 also transmits data from the computer board 10 to one or more vehicle propulsion and braking ECUs 40. However, it should be understood that... Figure 1 and Figure 2 This is essentially exemplary, and the computer board 10 can use any number of communication interfaces to communicate electronically with any number and type of computing systems.

[0037] In such Figure 1 In the non-limiting embodiment shown, the CPU 24 of each system-on-a-chip 14, which is part of the heterogeneous multi-core system 12, is divided into CPU groups 42, 44, 46, 48, and 50. Specifically, Figure 1The five physical CPU groups 42, 44, 46, 48, and 50 shown correspond to the five logical domains 52A, 52B, 54, 56, and 58 of the heterogeneous multi-core system 12, as follows: Figure 2 As shown. Reference Figure 1 and Figure 2 The first CPU group has 42 (see...) Figure 1 ) corresponds to the first logic domain 52A of the heterogeneous multi-core system 12 (see Figure 2 ). refer to Figure 2 The first logic domain 52A includes a camera driver 60 and a radar driver 62 that receive data from the camera ECU 32 and radar ECU 34, respectively. The second CPU group 44 (see...) Figure 1 ) corresponds to the second logic domain 52B (see Figure 2 ).like Figure 2 As shown, the second logic domain 52B includes an IMU driver 64 and a GPS driver 66 that receive data from IMU ECU 36 and GPS ECU 38, respectively. The third CPU group 46 (see...) Figure 1 ) corresponds to the third logical domain 54 (see Figure 2 It is based on camera driver 60, radar driver 62 and IMU driver 64 (see...) Figure 2 The received data is used to perform sensing functions. The fourth CPU group 48 (see...) Figure 1 ) corresponds to the fourth logical domain 56 (see Figure 2 ), and from IMU driver 64 and GPS driver 66 (see Figure 2 ) Receive data. Fifth CPU group 50 (see Figure 1 ) corresponds to the fifth logical domain 58 (see Figure 2 ), and from the lane tracking and fusion module 82 and object tracking and fusion module 84, which are part of the third logical domain 54, and the map service module 88 and positioning module 90 of the fourth logical domain 56 (see Figure 2 ) Receive data. It should be understood that, as Figure 1 and Figure 2 The embodiments shown are not inherently limiting, and the CPU 24 can be arranged in any number of logical domains based on the specific requirements of the real-time system 16.

[0038] The first CPU group 42 and the second CPU group 44 are both used to execute various device drivers 60, 62, 64, and 66. Specifically, the first CPU group 42 includes one CPU 24A. The camera driver 60 and the radar driver 62 are both executed by CPU 24A. The second CPU group 44 includes one CPU 24B, in which the IMU driver 64 and the GPS driver 66 are both executed by CPU 24B. The third CPU group 46 includes six CPUs 24C for executing the vision-based detector module 80, the lane tracking and fusion module 82, and the object tracking and fusion module 84. The fourth CPU group 48 includes three CPUs 24D for executing the vehicle dynamics module 86, the map service module 88, and the positioning module 90. The fifth CPU group 50 includes four CPUs 24E for executing the decision module 92 and the vehicle control module 94. Finally, as Figure 1 As shown, the system-on-a-chip 14A includes a reserved CPU 24F.

[0039] like Figure 2 As shown, the heterogeneous multi-core system 12 includes multiple modules 80, 82, 84, 86, 88, 90, 92, and 94, each module executing a scheduled task associated with one of logical domains 52A, 52B, 54, 56, and 58. For example, in Figure 2 In the illustrated embodiment, the real-time system 16 is an ADAS system, which includes a vision-based detector module 80, a lane tracking and fusion module 82, an object tracking and fusion module 84, a vehicle dynamics module 86, a map service module 88, a localization module 90, a decision module 92, and a vehicle control module 94 for performing various scheduling tasks associated with the ADAS system.

[0040] refer to Figure 2 The heterogeneous multicore system 12 may include one or more synchronizers 96. Specifically, in the illustrated non-limiting embodiment, the heterogeneous multicore system 12 includes six synchronizers 96A, 96B, 96C, 96D, 96E, 96F, and 96G. In one approach, synchronizer 96 connects two or more inputs 98 with different periodicities. Figure 3 The synchronizer 98 is synchronized together and transmits a synchronized set of two or more inputs 98 to one of modules 80, 82, 84, 86, 88, 90, 92, 94, which are part of the real-time system 16. Alternatively, the synchronizer 96 changes the periodicity or execution rate from a single input 98 and transmits the data to one of modules 80, 82, 84, 86, 88, 90, 92, 94, which are part of the heterogeneous multi-core system 12. Figure 2In the example shown, the heterogeneous multi-core system 12 includes a first synchronizer 96A, a second synchronizer 96B, a third synchronizer 96C, a fourth synchronizer 96D, a fifth synchronizer 96E, a sixth synchronizer 96F, and a seventh synchronizer 96G.

[0041] refer to Figure 3 Each synchronizer 96 includes one or more inputs 98, triggers 100, and outputs 102. It should be understood that each input 98 of the synchronizer 96 may include different execution rates or periodicities, as well as different delays. For example, in... Figure 2 In the illustrated embodiment, synchronizer 96D combines vehicle dynamics data 112 from vehicle dynamics module 86, including a 100 Hz frequency and a 30 ms delay, with GPS data 116 from GPS driver 66, including a 10 Hz frequency and a 150 ms delay, to create vehicle dynamics data 118 with a 10 Hz frequency. Therefore, when compared to input 98, the output 102 of synchronizer 96 ( Figure 3 () can include different execution rates.

[0042] refer to Figure 2 and Figure 3 Each synchronizer 96's trigger 100 represents a software event that prompts the synchronizer 96 to synchronize two or more inputs 98 with different periodicities at a predefined period based on one or more predefined strategies. Figure 3 They are synchronized together. Predefined strategies are application-specific and vary based on a specific instance of synchronizer 96. For example, a predefined strategy can indicate which inputs 98 are mandatory and which can be omitted. The predefined strategy can also indicate what actions need to be performed in the absence of mandatory inputs, or whether a specific input 98 is buffered to support backtracking. Trigger 100 is initiated by modules 80, 82, 84, 90, 92, 94 downstream of synchronizer 96, to which synchronizer 96 transmits data. For example, as... Figure 2 As shown, the trigger 100 of the first synchronizer 96A is activated by the vision-based detector module 80. (As...) Figure 3 As shown, synchronizer 96 generates a single output 102 that aggregates one or more inputs 98. In one embodiment, output 102 includes a status field message indicating the general state of output 102 and a field mapping indicating which inputs 98 are included in output 102.

[0043] In one embodiment, synchronizer 96 can be used to change the rate of data transmission. For example, synchronizer 96A, as part of a third CPU group 46, includes a single input 98 that receives raw camera data 104 from a camera driver 60, as part of a first CPU group 42. The raw camera data 104 includes a frequency of 30 Hz, and the trigger 100 includes a period of 100 milliseconds. The output 102 of synchronizer 96A is the raw camera data 104 with a frequency of 10 Hz. Furthermore, synchronizer 96B, also as part of the third CPU group 46, includes two inputs 98 and a trigger 100 with a period of 100 milliseconds. Synchronizer 96B receives camera detection data 106 with a frequency of 10 Hz from a vision-based detector module 80 and vehicle dynamics data 112 including a frequency of 100 Hz from a vehicle dynamics module 86. The output 102 of synchronizer 96B is the camera detection data 106 with a frequency of 10 Hz.

[0044] Continue to refer to Figure 2 and Figure 3 The synchronizer 96C, as part of the third CPU group 46, includes three inputs 98 and a trigger 100 with a 50 ms period. Specifically, the first input 98 of the synchronizer 96C receives camera detection data 106 at a frequency of 10 Hz from the vision-based detector module 80, the second input 98 of the synchronizer 96C receives radar detection data 108 at a frequency of 20 Hz from the radar driver 62, and the third input 98 of the synchronizer 96C receives vehicle dynamics data 112 at a frequency of 100 Hz from the vehicle dynamics module 86. The output 102 of the synchronizer 96C is object data 114 at a frequency of 20 Hz. The fourth synchronizer 96D includes two inputs 98 and a trigger 100 with a 100 ms period, wherein the first input 98 of the synchronizer receives vehicle dynamics data 112 from the vehicle dynamics module 86, and the second input 98 receives GPS data 116 at a frequency of 10 Hz from the GPS driver 66. The output 102 of the fourth synchronizer 96D is a combination of GPS and vehicle dynamics data 118 at a frequency of 10 Hz. The fifth synchronizer 96E includes a single input 98 and a trigger 100 with a period of 1000 milliseconds, wherein the input 98 of the fifth synchronizer 96E receives attitude data 126 with a frequency of 10 Hz from the positioning module 90. The output of the fifth synchronizer 96E is attitude data 126 with a frequency of 1 Hz.

[0045] The sixth synchronizer 96F includes four outputs and a trigger 100 with a 100-millisecond period. Specifically, the first input 98 of the sixth synchronizer 96F receives tracked lane data 120 at a frequency of 10 Hz from the lane tracking and fusion module 82, the second input 98 of the sixth synchronizer 96F receives tracked object data 122 at a frequency of 20 Hz from the object tracking and fusion module 84, the third input 98 receives map service data 124 at a frequency of 1 Hz from the map service module 88, and the fourth input 98 receives attitude data 126 at a frequency of 10 Hz from the positioning module 90. The output 102 of the sixth synchronizer 96F is state data 128 at a frequency of 10 Hz. The seventh synchronizer 96G includes two inputs 98 and a trigger 100 with a 10-millisecond period. The first output 102 of the seventh synchronizer 96G receives trajectory data 130 at a frequency of 10 Hz from the decision module 92, and the second output of the seventh synchronizer 96G receives vehicle dynamics data 112 from the vehicle dynamics module 86. The output 102 of the seventh synchronizer 96G is combined trajectory and vehicle dynamics data 132 with a frequency of 100 Hz.

[0046] Table 1 below illustrates an exemplary timer array for executing one of a plurality of scheduled tasks associated with one of the logical domains 52A, 52B, 54, 56, 58 of the real-time system 16. Refer to Table 1 and... Figure 2 Each module 80, 82, 84, 86, 88, 90, 92, 94 of the heterogeneous multi-core system 12 includes a unique periodicity. In the example shown in Table 3, modules 80, 82, 84, 86, 90, 92, 94 have an execution rate of 10 Hz, and the map service module 88 has an execution rate of 1 Hz; however, it should be understood that Table 3 is merely exemplary in nature. Furthermore, it should be understood that one or more scheduled tasks of the real-time system 16 execute at offset time 148 (see...). Figure 4 Specifically, multiple scheduled tasks are based on execution pipeline 150 (see...). Figure 4 ) is executed, and includes at least one offset scheduling task executed at offset time 148. Figure 4 ) and reference scheduled task. The reference scheduled task is located in execution phase 152 upstream of execution pipeline 150. Figure 4 The data is then transmitted to the offset scheduling task, with the offset time of 148 representing the time period measured relative to the reference scheduling task.

[0047] refer to Figure 2As shown in Table 1, the vision-based detector module 80 analyzes raw camera data 104 from the camera driver 60 to detect surrounding vehicles on the road. As shown in Table 1, the vision-based detector module 80 includes detection functionality and a rate of 10 Hz. The vehicle dynamics module 86 analyzes IMU data 110 from the IMU driver 64 and generates vehicle dynamics data 112 based on the IMU data 110. As shown in Table 1, the vehicle dynamics module 86 includes a rate of 10 Hz. Since both the vision-based detector module 80 and the vehicle dynamics module 86 are located in the execution pipeline 150 of the real-time system 16 (see Table 1), the detection process is designed to be efficient and efficient. Figure 4 The execution phase 152 is initiated, therefore there is no offset time 148 associated with the vision-based detector module 80 or the vehicle dynamics module 86.

[0048]

[0049]

[0050] Table 1

[0051] Lane tracking and fusion module 82 receives camera detection data 106 from second synchronizer 96B. For example... Figure 2 As shown, the lane tracking and fusion module 82 is located in the execution pipeline 150 ( Figure 4 This is downstream of the vision-based detector module 80. Therefore, in this example, the lane tracking and fusion module 82 performs the offset scheduling task, and the vision-based detector module 80 performs the reference scheduling task. As shown in Table 1, the lane tracking and fusion module 82 includes an offset time of 80 milliseconds, where the offset time is relative to the vision-based detector module 80.

[0052] Continue to refer to Table 1 and Figure 2 The object tracking and fusion module 84 receives object data 114 from the third synchronizer 96C. The object tracking and fusion module 84 analyzes the object data 114 and generates tracked object data 122 with a frequency of 20 Hz based on the object data 114. Since the object tracking and fusion module 84 is located in the execution pipeline 150... Figure 4 Downstream of the vision-based detector module 80, in this example, the object tracking and fusion module 84 performs the offset scheduling task, and the vision-based detector module 80 performs the reference scheduling task. As shown in Table 1, the object tracking and fusion module 84 includes an offset time of 30 milliseconds, where the offset time is relative to the vision-based detector module 80.

[0053] Positioning module 90 receives combined GPS and vehicle dynamics data 118 from fourth synchronizer 96D. Because positioning module 90 is located in execution pipeline 150... Figure 4Downstream of the vehicle dynamics module 86, the positioning module 90 performs an offset scheduling task, while the vehicle dynamics module 86 performs a reference scheduling task. As shown in Table 3, the positioning module 90 includes an offset time of 10 milliseconds, where the offset time is relative to the vehicle dynamics module 86.

[0054] The map service module 88 receives attitude data 126 from the fifth synchronizer 96E and generates map service data 124 with a frequency of 1 Hz. Since the map service module 88 is located in the execution pipeline 150... Figure 4 Downstream of the positioning module 90, the map service module 88 performs the offset scheduling task, and the visual positioning module 90 performs the reference scheduling task. As shown in Table 1, the map service module 88 includes an offset time of 200 milliseconds, where the offset time is relative to the positioning module 90. However, it should be understood that the output of the map service module 88 (i.e., map service data 124) is also received as input by the decision module 92. Therefore, the decision module 92 always consumes the latest data generated by the map service module 88. Therefore, the execution pipeline 150, which is described in more detail below, includes a guaranteed end-to-end latency of 210 milliseconds 156.

[0055] Decision module 92 receives tracking lane data 120 from the sixth synchronizer 96F. For example... Figure 4 As shown, the decision module 92 is located downstream of the execution pipeline 150 relative to the lane tracking and fusion module 82, the object tracking and fusion module 84, the map service module 88, and the positioning module 90. The decision module 92 includes an 80-millisecond offset time, where the offset time is relative to the object tracking and fusion module 84. Finally, the vehicle control module 94 receives combined trajectory and vehicle dynamics data 132 from the seventh synchronizer 96G and sends an output 160 to one or more vehicle propulsion and braking ECUs 40. The vehicle control module 94 is located downstream of the vehicle dynamics module 86 and the decision module 92 in the execution pipeline 150. As shown in Table 1, the vehicle control module 94 includes an 80-millisecond offset time, where the offset time 148 is relative to the decision module 92.

[0056] Figure 4 This is a diagram of the execution pipeline 150 of the real-time system 16, where T1 represents the first timer of the vision-based detector module 80, T2 represents the timer of the vehicle dynamics module 86, T3 represents the third timer of the object tracking and fusion module 84, T4 represents the fourth timer of the lane tracking and fusion module 82, T5 represents the timer of the map service module 88, T6 represents the timer of the localization module 90, T7 represents the timer of the decision-making module 92, and T8 represents the timer of the vehicle control module 94. (Reference) Figure 2 and Figure 4And as shown in Table 1, the first timer T1 of the vision-based detector module 80 includes an execution time of 80 milliseconds, and the third timer T3 of the object tracking and fusion module 84 includes an offset time 148 of 80 milliseconds relative to the start 140 of the first timer T1 of the vision-based detector module 80. Therefore, as Figure 4 As shown, the third timer T3 of the object tracking and fusion module 84 starts 80 milliseconds after the start 140 of the first timer T1. The fourth timer T4 of the lane tracking and fusion module 82 includes an offset time 148 of 80 milliseconds relative to the start 140 of the first timer T1 of the vision-based detector module 80. Therefore, as Figure 3 As shown, the fourth timer T4 of the lane tracking and fusion module 82 starts 80 milliseconds after the start 140 of the first timer T1. The fifth timer T5 of the map service module 88 includes an offset time 148 of 30 milliseconds relative to the start 142 of the second timer T2. Therefore, as Figure 3 As shown, the fifth timer T5 of the map service module 88 starts 30 milliseconds after the second timer T2 starts 142 seconds later.

[0057] The sixth timer T6 of the positioning module 90 includes an offset time 148 of 10 milliseconds relative to the start 140 of the second timer T2 of the vehicle dynamics module 86. Therefore, as... Figure 4 As shown, the sixth timer T6 of the positioning module 90 starts 10 milliseconds after the start 142 of the second timer T2. The seventh timer T7 of the decision module 92 includes an offset time 148 of 30 milliseconds relative to the start 144 of the third timer T3 of the tracking and fusion module 84. Therefore, as Figure 4 As shown, the seventh timer T7 of the decision module 92 starts 30 milliseconds after the start 144 of the third timer T3. The eighth timer T8 of the vehicle control module 94 includes an offset time 148 of 80 milliseconds relative to the start 146 of the seventh timer T7. Therefore, as Figure 4 As shown, the eighth timer T8 of the vehicle control module 94 starts 80 milliseconds after the start of the seventh timer T7, 146 milliseconds later.

[0058] The offset time 148 for the offset scheduling task is determined empirically by selecting an offset value that captures a predetermined percentage of the total runtime of all completed tasks. For example, in one embodiment, the lane tracing and fusion module 82 performs the offset scheduling task, and the vision-based detector module 80 performs the reference scheduling task. Therefore, the offset time 148 of the lane tracing and fusion module 82 is selected to allow the reference scheduling task to complete 90% of the time. Figure 2 and Figure 4As shown, the vision detector module 80 is located upstream of the vision-based detector module 80 at execution stage 152 and transmits data to the vision-based detector module 80. In this example, empirical data indicates that in 90% of cases, the reference scheduled task executed by the vision-based detector module 80 completes its execution within 80 seconds. This means that 10% of the time, the execution time exceeds 80 milliseconds. Therefore, as shown in Table 1, the offset time 148 of the lane tracking and fusion module 82 is 80 milliseconds.

[0059] like Figure 4 As shown, regardless of whether the task of the vision-based detector module 80 is executed, the third timer T3 of the lane tracking and fusion module 82 is started 80 milliseconds after the start 140 of the first timer T1. Therefore, it should be understood that increasing the offset time 148 reduces the probability of frame loss, while decreasing the offset time 148 increases the probability of frame loss. Therefore, it should also be understood that the heterogeneous multi-core system 12 compensates for lost frames.

[0060] Refer to Table 1 and Figure 4 Each stage 152 of the execution pipeline 150 of the real-time system 16 includes a guaranteed wait time based on an offset time 148. For example, since the offset time 148 of the third timer T3 is 80 milliseconds, the first timer T1 includes a guaranteed wait time of 80 milliseconds. Furthermore, as... Figure 4 As shown, since each timer T1-T8 starts at a specific time, the execution pipeline 150 includes a guaranteed end-to-end wait time 156. For example, in Figure 4 In the illustrated embodiment, the execution pipeline 150 includes a guaranteed end-to-end wait time 156 of 210 milliseconds. It should be understood that the guaranteed end-to-end wait time 156 (see...) Figure 4 The overall value of the end-to-end latency 156 is determined based on the specific application of the real-time system 16. For example, in the embodiment shown in the figure, the real-time system 16 is a safety-critical ADAS system, thus requiring a relatively short response time. However, if the real-time system 16 is used in applications where response time is less critical, such as robots moving at relatively low speeds and operating in a warehouse, the overall value of the end-to-end latency 156 may be larger.

[0061] Return to reference Figure 2 Each of the multiple modules 80, 82, 84, 86, 88, 90, 92, and 94 executes a scheduled task with a unique periodicity for one of the logical domains 52A, 52B, 54, 56, and 58. For example, in... Figure 2In the illustrated embodiment, the vision-based detector module 80 and lane tracking and fusion module 82 both include a 10 Hz periodicity, the object tracking and fusion module 84 includes a 20 Hz periodicity, the vehicle dynamics module 86 includes a 100 Hz periodicity, the map service module 88 includes a 1 Hz periodicity, the localization module 90 includes a 10 Hz periodicity, the decision module 92 includes a 10 Hz periodicity, and the vehicle control module 94 includes a 100 Hz periodicity. Therefore, each module 80, 82, 84, 86, 88, 90, 92, 94 of the disclosed real-time system 16 executes a scheduling task with a unique periodicity. Table 2 below is a priority table summarizing the runtime priorities of the scheduling tasks. Specifically, as shown in Table 2 below, the priority table assigns a runtime priority to each of one or more scheduling tasks, wherein the priority table is stored in the shared memory 20 of the system-on-a-chip 14 (see Table 2 below). Figure 1 ).

[0062] The runtime priority of each of one or more scheduled tasks is determined based on a unique periodicity. However, as described below, if two or more scheduled tasks that are part of the same logical domain 52A, 52B, 54, 56, 58 include the same periodicity, then in execution pipeline 150... Figure 4 Downstream scheduled tasks within a process are prioritized and upgraded. For example, in... Figure 2 In the embodiments shown in Table 2, the vision-based detector module 80 and the lane tracking and fusion module 82 each include the same 10 Hz periodicity and are located within the same logic domain 54. Therefore, since the scheduling task performed by the lane tracking and fusion module 82 is downstream of the scheduling task performed by the vision-based detector module 80, the scheduling task performed by the lane tracking and fusion module 82 is assigned a priority upgrade of +1. Therefore, in the case of... Figure 2 In the embodiments shown in Table 2, the vision-based detector module 80 includes a runtime priority of 10, and the lane tracking and fusion module 82 includes a runtime priority of 11. It should be understood that the execution pipeline 150 ( Figure 4 Downstream scheduling tasks take precedence over upstream scheduling tasks in the execution pipeline 150. Furthermore, since the object tracking and fusion module 84 includes a 20 Hz periodicity, it has a runtime priority of 21. Specifically, scheduling tasks executed by the object tracking and fusion module 84 include a priority upgrade of +1 because the vision-based detector module 80 is upstream of the object tracking and fusion module 84.

[0063] The fourth logical domain 56 includes a vehicle dynamics module 86 with a runtime priority of 30, a map service module 88 with a priority of 5, and a positioning module 90 with a priority of 10. Located in execution pipeline 150 ( Figure 4The vehicle dynamics module 86, downstream of the map service module 88 and positioning module 90, includes a highest runtime priority of 30. This is because the vehicle dynamics module 86 has a higher periodicity compared to the other two modules 88 and 90 in the fourth logical domain 56. The fifth logical domain 58 includes a decision module 92 and a vehicle control module 94, with the decision module 92 including a runtime priority of 10 and the vehicle control module 94 including a priority of 31.

[0064] 100 30 (highest) 30 25 20 20 15 15 10 10 1 5

[0065] Table 2

[0066] Table 3 and Figure 5 The diagram shows the logic domains 52A, 52B, 54, 56, and 58 (see...). Figure 2 A resource association table that allocates system resources between [various entities]. In [a specific context, such as...] Figure 5 In the non-limiting embodiment shown, each system-on-chip 14A, 14B includes a total of eight CPUs 24, with each CPU 24 assigned an identifier 0-7. As shown in Table 3, one of the CPUs 24 as part of the heterogeneous multi-core system 12 is assigned to each logical domain 52A, 52B, 54, 56, 58. Furthermore, each CPU 24 of each system-on-chip 14 is assigned one of the logical domains 52A, 52B, 54, 56, 58 of the heterogeneous multi-core system 12. Therefore, it should be understood that modifications and changes to one of the logical domains 52A, 52B, 54, 56, 58 will not introduce or cause frame dropping in the remaining logical domains 52A, 52B, 54, 56, 58.

[0067] 52A 0 0 52B 1 0 54 0 1、2、3、5、6、7 56 1 1、2、3 58 1 4、5、6、7

[0068] Table 3

[0069] Referring generally to the accompanying drawings, the disclosed heterogeneous multicore system offers various technical effects and benefits. Specifically, the disclosed heterogeneous multicore system includes a real-time scheduling system that provides guaranteed latency and guaranteed end-to-end latency for each stage of the execution pipeline. The guaranteed end-to-end latency described in this disclosure limits jitter and also provides a mechanism to ensure that the disclosed real-time system reacts within the allocated time. The disclosed real-time system can be used to support multiple system-on-a-chip (SoCs), each including multiple CPUs. Furthermore, the disclosed real-time scheduling system is scalable and efficient for multicore systems with shared hardware accelerators. The disclosed real-time scheduling system also supports multiple tasks executing at different rates without violating real-time performance. A synchronizer can synchronize two or more inputs with different periodicities, or, alternatively, the synchronizer can change the periodicity or execution rate from a single input. Finally, it should be understood that the disclosed real-time scheduling system utilizes embedded hardware in an optimized manner, which can reduce cost and power consumption compared to currently available embedded systems.

[0070] The description in this disclosure is merely exemplary in nature, and changes that do not depart from the essential points of this disclosure are intended to fall within its scope. Such changes should not be considered as departing from the spirit and scope of this disclosure.

Claims

1. A heterogeneous multi-core system for executing a real-time automotive system, the heterogeneous multi-core system comprising: Multiple system-on-a-chips that communicate electronically with each other, wherein each system-on-a-chip includes multiple central processing units (CPUs) arranged in multiple logical domains; Multiple scheduled tasks, based on pipelined execution and each executing a specific set of tasks in one of the logical domains, wherein the multiple scheduled tasks include: At least one offset-scheduled task is executed at an offset time, wherein the offset time of the offset-scheduled task is determined by selecting a predefined percentage of the total runtime of all completed tasks; and A reference scheduling task is located in an execution stage upstream of the execution pipeline relative to the offset scheduling task, wherein the reference scheduling task transmits data to the offset scheduling task, and the offset time represents a time period measured relative to the reference scheduling task, and wherein each stage of the execution pipeline of the real-time system includes a guaranteed waiting time based on the offset time of the offset scheduling task.

2. The heterogeneous multi-core system of claim 1, wherein the execution pipeline includes guaranteed end-to-end latency.

3. The heterogeneous multi-core system of claim 1, wherein each logical domain executes one or more scheduled tasks, and wherein each scheduled task is executed based on a unique periodicity.

4. The heterogeneous multi-core system according to claim 3, wherein each of the plurality of system-on-a-chip includes a corresponding shared memory.

5. The heterogeneous multi-core system of claim 4, wherein the shared memory stores a priority table indicating the runtime priority of each of the one or more scheduled tasks.

6. The heterogeneous multi-core system of claim 5, wherein the runtime priority of each of the one or more scheduled tasks is determined based on the unique periodicity.

7. The heterogeneous multi-core system of claim 1 further includes one or more synchronizers, wherein the one or more synchronizers synchronize two or more inputs with different periodicities together and transmit data from the two or more inputs to one of the plurality of scheduled tasks.

8. The heterogeneous multi-core system of claim 1, further comprising one or more synchronizers, wherein the one or more synchronizers alter the periodicity of a single input and transmit data from the single input to one of the plurality of scheduled tasks.

Images