Resource scheduling method, model generation method and device

Abstract

The invention provides a resource scheduling method, a model generation method and a model generation device, which are used for evaluating a scheduling strategy based on the operation delay of each task so as to obtain a more adaptive resource scheduling strategy for each function in a vehicle. The method comprises the steps that firstly, estimated resources corresponding to each task in at least one task to be achieved in a vehicle are obtained, the at least one task is used for achieving at least one function in the vehicle, the at least one function is achieved through computing resources in the vehicle, and the computing resources comprise multiple resources used for processing data in the vehicle; the estimated resource corresponding to each task is a resource used for running each task; obtaining the time delay of running each task in the estimated resource; obtaining an evaluation result according to the time delay of each task; a scheduling result is obtained according to the evaluation result, the scheduling result is used for indicating a strategy for achieving at least one task through the computing resources, and the scheduling result is used for generating a model for achieving at least one function.

Description

Technical Field

[0001] The present application relates to the field of smart cars, and in particular to a resource scheduling method, a model generation method and a device. Background Art

[0002] In recent years, with the rapid development of autonomous driving AI algorithms and technologies, a large number of traditional car manufacturers and new forces in autonomous driving have entered the market to build L4 high-level fully autonomous driving systems and applications. However, one of the most concerned issues in autonomous driving is whether it can ensure the safety of other vehicles and pedestrians.

[0003] Among them, a very important indicator for measuring safety is the end-to-end latency of the data flow link of the autonomous driving system, such as the latency from the acquisition of raw signals by sensors such as lidar and cameras to the issuance of braking and other instructions to the chassis domain controller. The running time of each node in the data flow link will directly have a significant impact. Lower end-to-end latency (such as less than 100ms) can greatly improve the safety factor of vehicles and pedestrians, and is one of the key goals for the later optimization of systems and applications.

[0004] When developing vehicle functions, different functions are usually developed by different developers. The end-to-end latency problem for vehicles may be difficult to achieve due to the lack of an integrated verification environment. Therefore, how to achieve end-to-end low latency for vehicles has become an urgent problem to be solved. Summary of the invention

[0005] The present application provides a resource scheduling method, a model generation method and a device for evaluating a scheduling strategy based on the runtime delay of each task, so as to obtain a more suitable resource scheduling strategy for each function in a vehicle.

[0006] In view of this, on the first aspect, the present application provides a resource scheduling method, the method comprising: first, obtaining estimated resources corresponding to each task in at least one task to be implemented in a vehicle, the at least one task being used to implement at least one function in the vehicle, the at least one function being implemented by computing resources in the vehicle, the computing resources including a plurality of resources in the vehicle for processing data, the estimated resources corresponding to each task being the resources used to run each task; obtaining the latency of running each task in the estimated resources; obtaining an evaluation result based on the latency of each task; obtaining a scheduling result based on the evaluation result, the scheduling result being used to indicate a strategy for using computing resources to implement at least one task, and the scheduling result being used to generate a model for implementing at least one function.

[0007] In an implementation manner of the present application, after allocating estimated resources in multiple dimensions to each task, the allocation plan can be evaluated based on the running delay of each task in the estimated resources, so that the end-to-end delay of each function in the vehicle can be evaluated, and the resource allocation plan that meets the delay requirements can be screened out, thereby achieving more adaptive resource scheduling for vehicle resources.

[0008] In addition, the resource scheduling method provided in the present application can be deployed in devices such as servers, terminals or cloud platforms. For the sake of distinction, the device deployed with the resource scheduling method provided in the present application is referred to as the first development device below.

[0009] In a possible implementation, each of the at least one task corresponds to at least one thread, and obtaining the latency of running each task in the estimated resources may include: running at least one thread corresponding to each task in the estimated resources; then, obtaining the latency corresponding to each task based on the running data of at least one thread of each task running in the estimated resources. Therefore, in the implementation of the present application, the running latency of each task can be collected by running the thread corresponding to each task in the estimated resources, so as to facilitate subsequent resource scheduling.

[0010] In a possible implementation, the aforementioned running at least one thread corresponding to each task in the estimated resources may include: simulating the running of at least one thread corresponding to each task in the estimated resources through a thread running simulator, the thread running simulator may simulate the running conditions of each task with threads as granularity, and then obtaining the delay corresponding to each task according to the running data of at least one thread of each task in the thread running simulator, that is, the running time of each task in the thread running simulator;

[0011] Alternatively, at least one thread corresponding to each task may be run in a device configured with the estimated resources, so that the runtime delay of each task may be calculated using actual hardware resources.

[0012] In the implementation mode of the present application, a thread operation simulator can be used to simulate the task operation at the thread granularity, so that the end-to-end delay can be accurately calculated, so as to facilitate a more accurate evaluation of the resource allocation scheme later. Alternatively, the running delay of each task can be calculated by actual hardware resources to obtain a more accurate delay.

[0013] In one possible implementation, if a thread running simulator is used to simulate the running of at least one thread corresponding to each task in the estimated resources, the aforementioned obtaining the delay corresponding to each task based on the running data of at least one thread of each function in the thread running simulator may include: in the process of running each task through the thread running simulator, obtaining the status data of each task, the status data of each task including the data generated when each task is running in the thread running simulator, such as the trigger time of the thread, the message queue and other data; according to the status data of each task, obtaining the delay corresponding to each task.

[0014] In the implementation of the present application, a state can be set for each task running in the thread running simulator, and the data generated during the task running process can be recorded through the state data, so that the running delay of each task in the simulator can be calculated based on the state data of each task. This is equivalent to setting a state machine for each task to simulate the task running process. Even if there are complex dependencies between tasks, the dependencies can be realized through the task state, thereby realizing the simulation of tasks with complex dependencies.

[0015] In one possible implementation, the status data of each of the aforementioned tasks includes at least one of the following: receiving status data, sending status data, computing status data or context status data, the receiving status data includes information of a message queue for receiving task data, the sending status data includes trigger information for sending task data, the computing status data includes data generated by triggering the execution of the task, and the context status data includes configuration data generated when indicating that the task enters a preset state when preset context conditions are met.

[0016] In the implementation mode of the present application, it is equivalent to dividing the state of each task into multiple categories, such as receiving state, sending state, computing state or context state, and setting corresponding state data for different states to record the data generated during the task being in different states. Therefore, the state change during the operation of each task can be simulated through the state of each task. Even if there are complex dependencies between tasks, the operation of tasks can be simulated through different states.

[0017] In a possible implementation, the aforementioned acquisition of status data of each task may include: if the first task is in a receiving state, updating the receiving status data of the first task, where the first task is any one of at least one task; if the trigger condition of the first task is met, triggering the execution of the first task, and updating the computing status data corresponding to the first task; if the trigger condition of the first task is met and the context condition of the first task is met, adjusting the context status data corresponding to the first task, and updating the computing status data corresponding to the first task based on the adjusted context data.

[0018] In the implementation manner of the present application, the state of the task can be switched through corresponding switching conditions, thereby realizing the operation simulation of various tasks.

[0019] In a possible implementation, the aforementioned method may further include:

[0020] If the thread running simulator does not include a running time probability model corresponding to the hardware platform indicated by the computing resource, an algorithm database is obtained, the algorithm database includes multiple algorithm models and a first running time probability model corresponding to a preset hardware platform, the first running time probability model includes the running time and corresponding probabilities of the multiple algorithm models running in the preset hardware platform, and the algorithm database is used by the thread running simulator when running at least one task;

[0021] If there is a real hardware platform adapted to the preset hardware platform, and multiple algorithm models can be run in the real hardware platform, then obtain the corresponding second runtime probability model in the real hardware platform, the second runtime probability model includes the runtime of multiple algorithm models running in the real hardware platform and the corresponding probabilities, and obtain the average runtime of multiple algorithms according to the ratio of the runtime of multiple algorithm models running in the preset hardware platform and the real hardware platform;

[0022] If there is a real hardware platform that is adapted to the preset hardware platform, and multiple algorithm models cannot be run in the real hardware platform, then the running time corresponding to multiple instructions in the algorithm database in the preset hardware platform is obtained, and the running time of multiple instructions in the real hardware platform is obtained. According to the ratio between the running time corresponding to the multiple instructions in the preset hardware platform and the running time in the real hardware platform, the corresponding third running time probability model in the real hardware platform is obtained.

[0023] In the implementation of the present application, when there is no hardware platform corresponding to the vehicle, the running time of each algorithm model or the running time of the assembly instruction can be measured based on the existing real hardware platform to estimate the possible running time of each algorithm model in the vehicle hardware platform. Therefore, even if the first development device side does not have the real vehicle hardware platform, the simulated operation of the thread can be realized, so as to calculate the accurate end-to-end delay.

[0024] In a possible implementation, the aforementioned obtaining of scheduling results based on the evaluation results may include: judging whether the convergence conditions are met based on the evaluation results; if so, taking the estimated resources corresponding to each task as the scheduling results; if not, adjusting the estimated resources corresponding to each task, and re-evaluating until the convergence conditions are met.

[0025] In the implementation manner of the present application, a better scheduling result can be obtained by iteration. In each iteration process, the resource allocation scheme can be evaluated based on the simulated end-to-end delay, and finally a resource scheduling result that meets the delay requirement can be obtained.

[0026] Optionally, the aforementioned convergence conditions may include but are not limited to one or more of the following: the delay of each task does not exceed the corresponding delay threshold, the utilization rate of computing resources when implementing each task is higher than a first threshold or lower than a second threshold, the delay change value of the task between multiple iterations does not exceed a preset change threshold, the number of iterations reaches a preset number of times or the iteration duration reaches a preset duration, etc., so that convergence can be identified from multiple dimensions.

[0027] In a possible implementation, the aforementioned obtaining the evaluation result according to the delay of each task may include: obtaining the evaluation result according to the delay of each task and the usage rate of computing resources when simulating the running of each task in the thread running simulation.

[0028] Therefore, in the implementation mode of the present application, the resource allocation scheme can also be evaluated in combination with the computing resource utilization rate of the vehicle when simulating the operation of each task, so as to evaluate the satisfiability of the vehicle's resources for the allocation scheme, so as to finally obtain a resource scheduling result that is adapted to the vehicle's available computing resources.

[0029] In a possible implementation, the aforementioned acquisition of estimated resources corresponding to each task in at least one task to be implemented in the vehicle may include: obtaining user input data, the user input data including information on computing resources and delay model information of at least one task, the delay model information of at least one task including at least one of thread running delay, running frequency, triggering relationship between tasks or number of threads; constructing a feasible solution space based on the computing resources and the delay model information of each task; searching for feasible solutions from the feasible solution space to obtain the estimated resources corresponding to each task.

[0030] In the implementation mode of the present application, user input data can be collected, and the user input data set may include the available computing resources of the vehicle and the delay model information of each function to be implemented. Therefore, resources can be allocated according to the needs input by the user, and a resource scheduling solution that meets the user's needs can be obtained.

[0031] In a possible implementation, the aforementioned computing resources may include but are not limited to the following: central processing unit CPU resources, graphics processing unit GPU resources, memory resources, network card resources, hardware accelerators or storage resources. Therefore, in the implementation of the present application, resource scheduling can be performed from multiple dimensions, so that the final scheduling scheme can reasonably utilize various types of resources of the vehicle and improve the performance of the vehicle finally realized.

[0032] In a possible implementation, the aforementioned method may further include: generating a timing visualization analysis result according to the scheduling result, the timing visualization analysis result including a list of tasks corresponding to the tasks respectively executed on various types of resources in the computing resources according to the priorities included in the scheduling result. In the implementation of the present application, a timing diagram for allocating computing resources may be generated based on the scheduling result, indicating the situation of each task to be executed in each resource, so that the user or when developing functions can refer to the scheduling result more intuitively.

[0033] In a possible implementation, the aforementioned method may further include: sending a scheduling result to a second development device to instruct the second development device to perform function development according to the scheduling result, that is, to develop a behavior model running in the vehicle; and then receiving at least one behavior model sent by the second development device, at least one behavior model being generated by the second development device according to the scheduling result, and at least one behavior model being used to implement at least one function.

[0034] In an implementation manner of the present application, after obtaining the scheduling result, the scheduling result can be sent to the second development device, so that the second development device can perform function development based on the scheduling result and obtain a behavior model adapted to the scheduling result to realize the vehicle function.

[0035] In one possible implementation, the aforementioned scheduling results include at least one of the following: an operating cycle, a running time probability model, a memory bandwidth, or a network bandwidth. The operating cycle includes the running time range of each function, the running time probability model includes the running time of each function as the probability of each running time, the memory bandwidth is the bandwidth occupied by the memory used during the running of each function, and the network bandwidth is the bandwidth of the communication network used by each function during the running.

[0036] In a possible implementation, the method may further include: generating one or more behavior models according to the scheduling result, and the one or more behavior models may be used to implement one or more functions in the vehicle. Therefore, in addition to resource scheduling, the first development device side may also perform function development, so that the vehicle function implementation may be completed more efficiently.

[0037] In a second aspect, the present application provides a model generation method, comprising: first, obtaining a scheduling result, the scheduling result including a strategy indicating the use of computing resources in a vehicle to implement at least one task, and then obtaining at least one behavior model based on the scheduling result, the at least one behavior model being used to implement at least one function in the vehicle; and then sending information of at least one behavior model to a first development device.

[0038] In the implementation mode of the present application, after obtaining the scheduling result based on the end-to-end delay obtained by simulating the task operation of the vehicle, function development can be performed according to the available resources allocated to each task in the scheduling result, thereby generating one or more behavior models to implement one or more functions of the vehicle. Thus, a behavior model adapted to the vehicle computing resources is obtained.

[0039] In a possible implementation, the above-mentioned acquisition of at least one behavior model based on the scheduling results may include: generating at least one model based on the scheduling results; if at least one model meets the constraint conditions, then at least one model is used as at least one behavior model, and the constraint conditions are determined based on the scheduling results. For example, the scheduling results may include the running time or running frequency of each task, etc., and the constraint conditions may be generated based on the running time or running frequency of each task to constrain the generated behavior model.

[0040] Therefore, in the implementation manner of the present application, the behavior model can be constrained based on the constraints generated by the scheduling results, so as to obtain a behavior model that meets the requirements.

[0041] In one possible implementation, the aforementioned scheduling results include at least one of the following: an operating cycle, a running time probability model, a memory bandwidth, or a network bandwidth. The operating cycle includes the running time range of each function, the running time probability model includes the running time of each function as the probability of each running time, the memory bandwidth is the bandwidth occupied by the memory used during the running of each function, and the network bandwidth is the bandwidth of the communication network used by each function during the running.

[0042] In a possible implementation, the aforementioned computing resources may include but are not limited to the following: central processing unit CPU resources, graphics processing unit GPU resources, memory resources, network card resources, hardware accelerators or storage resources. Therefore, in the implementation of the present application, resource scheduling can be performed from multiple dimensions, so that the final scheduling scheme can reasonably utilize various types of resources of the vehicle and improve the performance of the vehicle finally realized.

[0043] In addition, the model generation method provided in the present application can be deployed in devices such as servers, terminals or cloud platforms. For the sake of distinction, the device on which the model generation method provided in the present application is deployed is referred to as the second development device.

[0044] In a third aspect, the present application provides a resource scheduling device, the device comprising:

[0045] a screening module, used to obtain estimated resources corresponding to each of at least one task to be implemented in the vehicle, wherein the at least one task is used to implement at least one function in the vehicle, the at least one function is implemented by computing resources in the vehicle, the computing resources include a plurality of resources in the vehicle for processing data, and the estimated resources corresponding to each task are resources used to run each task;

[0046] The latency management module is used to obtain the latency of running each task in the estimated resources;

[0047] An evaluation module, used to obtain evaluation results based on the latency of each task;

[0048] A tuning module is used to obtain a scheduling result based on the evaluation result, the scheduling result is used to indicate a strategy for using computing resources to implement at least one task, and the scheduling result is used to generate a model that implements at least one function.

[0049] Among them, the effects achieved by the third aspect or any optional implementation of the third aspect can be found in the description of the aforementioned first aspect or any optional implementation of the first aspect, and will not be repeated below.

[0050] In a possible implementation, the latency management module is specifically used to: run at least one thread corresponding to each task in the estimated resources; and obtain the latency corresponding to each task based on the running data of at least one thread of each task running in the estimated resources.

[0051] In one possible implementation, the latency management module is specifically used to: simulate the running of at least one thread corresponding to each task in the estimated resources through a thread running simulator, and obtain the latency corresponding to each task based on the running data of at least one thread of each task in the thread running simulator; or, run at least one thread corresponding to each task through a device configured with estimated resources.

[0052] In one possible implementation, if at least one thread corresponding to each task is run in the estimated resources through a thread running simulator, the delay management module is specifically used to: obtain the status data of each task in the process of running each task through the thread running simulator, the status data of each task including the data generated when each task is running in the thread running simulator; and obtain the delay corresponding to each task based on the status data of each task.

[0053] In one possible implementation, the status data of each task includes at least one of the following: receiving status data, sending status data, computing status data or context status data, the receiving status data includes information of a message queue for receiving data of the task, the sending status data includes trigger information for sending data of the task, the computing status data includes data generated by triggering the execution of the task, and the context status data includes configuration data generated when indicating that the task enters a preset state when preset context conditions are met.

[0054] In one possible implementation, the latency management module is specifically used to: if the first task is in a receiving state, update the receiving state data of the first task, where the first task is any one of at least one task; if the trigger condition of the first task is met, trigger the execution of the first task and update the computing state data corresponding to the first task; if the trigger condition of the first task is met and the context condition of the first task is met, adjust the context state data corresponding to the first task, and update the computing state data corresponding to the first task based on the adjusted context data.

[0055] In one possible implementation, the device includes: a model migration module, which is used to: if the thread running simulator does not include a runtime probability model corresponding to the hardware platform indicated by the computing resources, then obtain an algorithm database, the algorithm database includes multiple algorithm models and a first runtime probability model corresponding to the preset hardware platform, the first runtime probability model includes the runtime of multiple algorithm models running in the preset hardware platform and the corresponding probabilities, the algorithm database is used by the thread running simulator when running at least one task; if there is a real hardware platform adapted to the preset hardware platform, and multiple algorithm models can be run in the real hardware platform, then obtain the corresponding second runtime probability model in the real hardware platform, the second The running time probability model includes the running time and corresponding probabilities of multiple algorithm models running in the real hardware platform, and obtains the average running time of multiple algorithms based on the ratio of the running time of multiple algorithm models in the preset hardware platform and the real hardware platform; if there is a real hardware platform adapted to the preset hardware platform, and multiple algorithm models cannot be run in the real hardware platform, then the corresponding running time of multiple instructions in the algorithm database in the preset hardware platform is obtained, and the running time of multiple instructions in the real hardware platform is obtained; according to the ratio between the running time corresponding to the multiple instructions in the preset hardware platform and the running time in the real hardware platform, the corresponding third running time probability model in the real hardware platform is obtained.

[0056] In a possible implementation, the tuning module is specifically used to: determine whether the convergence condition is met according to the evaluation result; if so, use the estimated resources corresponding to each task as the scheduling result; if not, adjust the estimated resources corresponding to each task.

[0057] In a possible implementation, the evaluation module is specifically used to obtain an evaluation result according to the delay of each task and the usage rate of computing resources when simulating the running of each task in the thread running simulation.

[0058] In one possible implementation, the screening module is specifically used to: obtain user input data, where the user input data includes information about computing resources and delay model information of at least one task, where the delay model information of at least one task includes at least one of thread running delay, running frequency, triggering relationship between tasks, or number of threads; construct a feasible solution space based on computing resources and the delay model information of each task; search for feasible solutions from the feasible solution space to obtain estimated resources corresponding to each task.

[0059] In a possible implementation, the computing resources include, but are not limited to, the following: central processing unit (CPU) resources, graphics processing unit (GPU) resources, memory resources, network card resources, hardware accelerators, or storage resources.

[0060] In a possible implementation, the device further includes: a visualization analysis module for generating a timing visualization analysis result based on the scheduling result, wherein the timing visualization analysis result includes a list of corresponding tasks that are run on multiple types of resources in the computing resources according to the priority included in the scheduling result.

[0061] In a possible implementation, the above-mentioned device also includes: a transceiver module, used to: send a scheduling result to a second development device; receive at least one behavior model sent by the second development device, at least one behavior model is generated by the second development device according to the scheduling result, and at least one behavior model is used to implement at least one function.

[0062] In one possible implementation, the scheduling result includes at least one of the following: an operating cycle, a running time probability model, a memory bandwidth, or a network bandwidth. The operating cycle includes the running time range of each function, the running time probability model includes the running time of each function as the probability of each running time, the memory bandwidth is the bandwidth occupied by the memory used during the running of each function, and the network bandwidth is the bandwidth of the communication network used by each function during the running.

[0063] In a fourth aspect, the present application provides a model generation device, comprising:

[0064] A transceiver module is used to obtain a scheduling result, wherein the scheduling result includes a strategy for using computing resources in the vehicle to implement at least one task.

[0065] A generating module, used for acquiring at least one behavior model according to the scheduling result, wherein the at least one behavior model is used for realizing at least one function in the vehicle;

[0066] The transceiver module is also used to send information of at least one behavior model to the first development device.

[0067] The effects achieved by the third aspect and any optional implementation manner of the third aspect can be found in the description of the aforementioned second aspect or any optional implementation manner of the second aspect, and will not be repeated here.

[0068] In a possible implementation, the generation module is specifically used to: generate at least one model according to the scheduling result; if the at least one model meets the constraint condition, use the at least one model as at least one behavior model, and the constraint condition is determined according to the scheduling result.

[0069] In a fifth aspect, an embodiment of the present application provides a resource scheduling device, comprising: a processor and a memory, wherein the processor and the memory are interconnected via a line, and the processor calls a program code in the memory to execute the processing-related functions in any of the methods shown in the first aspect above. Optionally, the resource scheduling device can be a chip.

[0070] In the sixth aspect, an embodiment of the present application provides a model generation device, which can also be called a digital processing chip or chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is used to perform functions related to processing as described in the first aspect or any optional embodiment of the first aspect.

[0071] In a seventh aspect, an embodiment of the present application provides a model generation device, comprising: a processor and a memory, wherein the processor and the memory are interconnected via a line, and the processor calls a program code in the memory to execute the processing-related functions in the second aspect or any optional implementation of the second aspect. Optionally, the model generation device may be a chip.

[0072] In the eighth aspect, an embodiment of the present application provides a model generation device, which may also be referred to as a digital processing chip or chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is used to perform functions related to processing as described in the second aspect or any optional embodiment of the second aspect.

[0073] In a ninth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enables the computer to execute a method in any optional implementation of the first aspect or the second aspect above.

[0074] In the tenth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute a method in any optional implementation of the first aspect or the second aspect above.

[0075] In the eleventh aspect, an embodiment of the present application provides a vehicle function configuration system, including a first development device and a second development device, wherein the first development device can be used to implement the method in the aforementioned first aspect or any optional embodiment of the first aspect, and the second development device is used for the method in the aforementioned second aspect or any optional embodiment of the second aspect.

[0076] In the twelfth aspect, an embodiment of the present application provides a vehicle, which includes the device as described in the third aspect or the fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 A schematic diagram of the structure of a vehicle provided for this application;

[0078] Figure 2 A schematic diagram of the structure of another vehicle provided for this application;

[0079] Figure 3 A schematic diagram of the system architecture of a CP classic platform deployed on a vehicle provided in this application;

[0080] Figure 4 A schematic diagram of the system architecture of an AP platform deployed on a vehicle provided in this application;

[0081] Figure 5 A schematic diagram of the architecture of a vehicle function configuration system provided in this application;

[0082] Figure 6 A schematic diagram of a resource scheduling method provided in this application;

[0083] Figure 7 A schematic diagram of a flow chart of a model generation method provided in this application;

[0084] Figure 8 A schematic diagram of the architecture of another vehicle function configuration system provided by the present application;

[0085] Fig. 9 A schematic diagram of the architecture of another vehicle function configuration system provided by the present application;

[0086] Fig.10 A flowchart of another resource scheduling method and model generation method provided in this application;

[0087] Fig.11 A schematic diagram of an existing thread operation process provided for this application;

[0088] Fig.12 A schematic diagram of a thread operation process provided for this application;

[0089] Fig.13 Another thread operation flow diagram provided for this application;

[0090] Fig.14 A task and thread mapping diagram provided for this application;

[0091] Fig.15 A schematic diagram of the structure of a thread running simulator provided for this application

[0092] Fig.16 A simulation operation process of a module provided for this application;

[0093] Fig.17 A schematic diagram of end-to-end delay provided for this application;

[0094] Fig.18 A schematic diagram of an end-to-end delay optimization architecture provided for this application;

[0095] Fig.19 A schematic diagram of an end-to-end delay optimization process provided for this application;

[0096] Fig. 20 A runtime visualization diagram provided for this application;

[0097] Fig.21 A schematic diagram of the execution flow of a model migration provided for this application;

[0098] Fig. 22 A schematic diagram of the structure of a resource scheduling device provided in this application;

[0099] Fig.23 A schematic diagram of the structure of a model generation device provided in this application;

[0100] Fig.24 A schematic diagram of the structure of another resource scheduling device provided by the present application;

[0101] Fig.25 A schematic diagram of the structure of another model generation device provided in this application;

[0102] Fig.26 A schematic diagram of a chip structure provided for this application. DETAILED DESCRIPTION

[0103] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0104] The method provided in this application can be applied to various scenarios of equipment reliability verification. Reliability verification is crucial in the fields of smart cars, aviation, industrial control, or rail transit. For example, verifying the reliability of aircraft in the aviation field, verifying the reliability of industrial equipment in the industrial control field, and verifying the reliability of rail transit vehicles in the rail transit field. This application exemplarily introduces the reliability verification of a vehicle as an example, and the vehicles mentioned below can also be replaced by other equipment that needs to be verified.

[0105] In order to facilitate understanding of this solution, the present application embodiment first combines Figure 1 The structure of the vehicle provided in this application is introduced.

[0106] Please read first Figure 1 , Figure 1 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application, Figure 1 1 is a functional block diagram of a vehicle 100 provided in an embodiment of the present application. The vehicle 100 may be configured in a fully or partially autonomous driving mode. For example, the vehicle 100 may obtain environmental information about its surroundings through the perception system 120, and obtain an autonomous driving strategy based on the analysis of the surrounding environmental information to achieve fully autonomous driving, or present the analysis results to the user to achieve partially autonomous driving.

[0107] The vehicle 100 may include various subsystems, such as an infotainment system 110, a perception system 120, a decision control system 130, a drive system 140, and a computing platform 150. Optionally, the vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and component of the vehicle 100 may be interconnected by wire or wirelessly.

[0108] In some embodiments, the infotainment system 110 may include a communication system 111 , an entertainment system 112 , and a navigation system 113 .

[0109] The communication system 111 may include a wireless communication system 111, which can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 111 can use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. The wireless communication system 111 can communicate with a wireless local area network (WLAN) using WiFi. In some embodiments, the wireless communication system 146 can communicate directly with the device using an infrared link, Bluetooth, or ZigBee. The wireless communication system 111 may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communications between vehicles and / or roadside stations.

[0110] The entertainment system 112 may include a central control screen, a microphone and a speaker. The user can listen to the radio and play music in the car based on the entertainment system 112; or connect the mobile phone to the vehicle and realize the projection of the mobile phone on the central control screen. The central control screen may be touch-sensitive, and the user can operate by touching the screen. In some cases, the user's voice signal can be obtained through the microphone, and the user can control certain aspects of the vehicle 100 based on the analysis of the user's voice signal, such as adjusting the temperature in the car. In other cases, music can be played to the user through the speaker.

[0111] The navigation system 113 may include a map service provided by a map provider, thereby providing navigation of the driving route for the vehicle 100. The navigation system 113 may be used in conjunction with the vehicle's global positioning system 121 and inertial measurement unit 122. The map service provided by the map provider may be a two-dimensional map or a high-precision map.

[0112] The perception system 120 may include several sensors that sense information about the environment around the vehicle 100. For example, the perception system 120 may include a global positioning system 121 (the global positioning system may be a GPS system, or a Beidou system or other positioning systems), an inertial measurement unit (IMU) 122, a laser radar 123, a millimeter wave radar 124, an ultrasonic radar 125, and a camera 126. The perception system 120 may also include sensors of the internal systems of the monitored vehicle 100 (for example, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, direction, speed, etc.). Such detection and recognition are key functions for the safe operation of the vehicle 100.

[0113] The global positioning system 121 may be used to determine the geographic location of the vehicle 100 .

[0114] The inertial measurement unit 122 is used to sense the position and orientation changes of the vehicle 100 based on inertial acceleration. In some embodiments, the inertial measurement unit 122 may be a combination of an accelerometer and a gyroscope.

[0115] The LiDAR 123 may utilize laser light to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the LiDAR 123 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0116] The millimeter wave radar 124 may utilize radio signals to sense objects in the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the millimeter wave radar 124 may also be used to sense the speed and / or heading of the objects.

[0117] The ultrasonic radar 125 may sense objects around the vehicle 100 using ultrasonic signals.

[0118] The camera device 126 may be used to capture image information of the surrounding environment of the vehicle 100. The camera device 126 may include a monocular camera, a binocular camera, a structured light camera, a panoramic camera, etc. The image information acquired by the camera device 126 may include static image information or video stream information.

[0119] The decision control system 130 includes a computing system 131 for analyzing and making decisions based on the information obtained by the perception system 120. The decision control system 130 also includes a vehicle controller 132 for controlling the power system of the vehicle 100, and a steering system 133, a throttle 134 and a braking system 135 for controlling the vehicle 100.

[0120] The computing system 131 may process and analyze various information acquired by the perception system 120 in order to identify targets, objects and / or features in the surrounding environment of the vehicle 100. The targets may include pedestrians or animals, and the objects and / or features may include traffic signals, road boundaries and obstacles. The computing system 131 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking and other technologies. In some embodiments, the computing system 131 may be used to map the environment, track objects, estimate the speed of objects, and so on. The computing system 131 may analyze the various information acquired and derive a control strategy for the vehicle.

[0121] The vehicle controller 132 can be used to coordinate and control the vehicle's power battery and engine 141 to improve the power performance of the vehicle 100.

[0122] The steering system 133 may be used to adjust the forward direction of the vehicle 100. For example, in one embodiment, it may be a steering wheel system.

[0123] The throttle 134 is used to control the operating speed of the engine 141 and thus the speed of the vehicle 100 .

[0124] The braking system 135 is used to control the deceleration of the vehicle 100. The braking system 135 can use friction to slow down the rotation speed of the wheel 144. In some embodiments, the braking system 135 can convert the kinetic energy of the wheel 144 into electric current. The braking system 135 can also take other forms to slow down the rotation speed of the wheel 144 to control the speed of the vehicle 100.

[0125] The drive system 140 includes components that provide power movement for the vehicle 100. In one embodiment, the drive system 140 may include an engine 141, an energy source 142, a transmission system 143, and wheels 144. The engine 141 may be an internal combustion engine, an electric motor, an air compression engine, or a combination of other types of engines, such as a hybrid engine composed of a gasoline engine and an electric motor, or a hybrid engine composed of an internal combustion engine and an air compression engine. The engine 141 converts the energy source 142 into mechanical energy.

[0126] Examples of energy source 142 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 142 may also provide energy to other systems of vehicle 100.

[0127] The transmission system 143 can transmit mechanical power from the engine 141 to the wheels 144. The transmission system 143 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission system 143 may also include other devices, such as a clutch. Among them, the drive shaft may include one or more shafts that can be coupled to one or more wheels 144.

[0128] Some or all functions of the vehicle 100 are controlled by a computing platform 150. The computing platform 150 may include at least one processor 151 that may execute instructions 153 stored in a non-transitory computer-readable medium such as a memory 152. In some embodiments, the computing platform 150 may also be a plurality of computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

[0129] The processor 151 may be any conventional processor, such as a commercially available CPU. Alternatively, the processor 151 may also include a graphics processing unit (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application-specific integrated circuit (ASIC), or a combination thereof. The processor 151 may be located on a device remote from the vehicle and communicate wirelessly with the vehicle.

[0130] In some embodiments, the memory 152 may include instructions 153 (e.g., program logic) that may be executed by the processor 151 to perform various functions of the vehicle 100. The memory 152 may also include additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the infotainment system 110, the perception system 120, the decision control system 130, and the drive system 140.

[0131] In addition to instructions 153, memory 152 may also store data such as road maps, route information, the vehicle's location, direction, speed, and other similar vehicle data, as well as other information. Such information may be used by vehicle 100 and computing platform 150 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0132] The computing platform 150 may control functions of the vehicle 100 based on input received from various subsystems, such as the drive system 140, the perception system 120, and the decision control system 130. For example, the computing platform 150 may utilize input from the decision control system 130 in order to control the steering system 133 to avoid an obstacle detected by the perception system 120. In some embodiments, the computing platform 150 may be operable to provide control over many aspects of the vehicle 100 and its subsystems.

[0133] Alternatively, one or more of the above-mentioned components may be installed or associated separately from the vehicle 100. For example, the memory 152 may exist partially or completely separately from the vehicle 100. The above-mentioned components may be communicatively coupled together in a wired and / or wireless manner.

[0134] Optionally, the above components are only examples. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 1 It should not be understood as limiting the embodiments of the present application.

[0135] An autonomous vehicle traveling on a road, such as vehicle 100 above, can identify objects in its surrounding environment to determine adjustments to the current speed. The objects can be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently, and based on the object's respective characteristics, such as its current speed, acceleration, spacing from the vehicle, etc., can be used to determine the speed to be adjusted by the autonomous vehicle.

[0136] Optionally, the vehicle 100 or a sensing and computing device associated with the vehicle 100 (e.g., computing system 131, computing platform 150) can predict the behavior of the identified object based on the characteristics of the identified object and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so all the identified objects can also be considered together to predict the behavior of a single identified object. The vehicle 100 can adjust its speed based on the predicted behavior of the identified object. In other words, the autonomous vehicle can determine what stable state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the object. In this process, other factors can also be considered to determine the speed of the vehicle 100, such as the lateral position of the vehicle 100 in the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, etc.

[0137] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device may also provide instructions to modify the steering angle of vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0138] In the method provided in the present application, one or more functions in the vehicle to be implemented may include but are not limited to the functions in the various systems shown in the aforementioned vehicle, such as implementing various functions in the infotainment system 110, or implementing the tracking perception function in the perception system 120, and so on.

[0139] The vehicle 100 may be a car, a truck, a motorcycle, a bus, a ship, an airplane, a helicopter, an amusement vehicle, an amusement park vehicle, construction equipment, a tram, a golf cart, a train, etc., and the embodiments of the present application do not make any particular limitation.

[0140] Figure 2A system architecture diagram of a vehicle 100 based on some embodiments is shown, where the vehicle 100 includes multiple vehicle integration units (VIU) 11, a telematic box (T-BOX) 12, a cockpit domain controller (CDC), a mobile data center (MDC) 14, and a vehicle domain controller (VDC) 15.

[0141] The vehicle 100 also includes multiple types of sensors disposed on the vehicle body, including: a laser radar 21, a millimeter wave radar 22, an ultrasonic radar 23, and a camera 24. Each type of sensor may include multiple sensors. It should be understood that although Figure 2 FIG. 1 shows the location layout of different sensors on the vehicle 100 , but Figure 2 The number and position layout of the sensors in the figure are only for illustration, and those skilled in the art can reasonably select the type, number and position layout of the sensors according to their needs.

[0142] exist Figure 2 Four VIUs are shown, it should be understood that Figure 2 The number and position of the VIUs are only an example, and those skilled in the art can select the appropriate number and position of the VIUs according to actual needs.

[0143] The vehicle integrated unit VIU 11 provides part or all of the data processing functions or control functions required by the vehicle components for the vehicle components. The VIU may have one or more of the following functions.

[0144] 1. Electronic control function, that is, VIU is used to implement the electronic control function provided by the electronic control unit (ECU) inside some or all vehicle parts. For example, the control function required by a certain vehicle part, or the data processing function required by a certain vehicle part.

[0145] 2. The same functions as the gateway, that is, the VIU may also have some or all of the same functions as the gateway, such as protocol conversion function, protocol encapsulation and forwarding function, and data format conversion function.

[0146] 3. The function of processing data across vehicle components, that is, processing and calculating data obtained from actuators of multiple vehicle components.

[0147] It should be noted that the data involved in the above functions may include the operating data of the actuator in the vehicle parts, such as the motion parameters of the actuator, the working status of the actuator, etc. The data involved in the above functions may also be data collected by the data collection unit (e.g., sensitive element) of the vehicle parts, such as the road information collected by the sensitive element of the vehicle, or weather information, etc., which is not specifically limited in the embodiments of the present application.

[0148] exist Figure 2 In the example of the vehicle 100, the vehicle 100 can be divided into multiple domains, each of which has an independent domain controller. Specifically, Figure 2 In the figure, two domain controllers are shown: a cockpit domain controller CDC13 and a vehicle domain controller VDC 15.

[0149] The cockpit domain controller CDC 13 can be used to implement functional control of the cockpit area of ​​the vehicle 100. Vehicle components in the cockpit area may include a head-up display (HUD), an instrument panel, a radio, a central control screen, a navigation, a camera, etc.

[0150] The vehicle domain controller VDC 15 may be used to coordinate and control the vehicle's power battery and engine 141 to improve the power performance of the vehicle 100. In some embodiments, Figure 1 The vehicle controller 132 can realize various functions of VDC.

[0151] Figure 2 The figure also shows a vehicle networking device T-BOX 12 and a mobile data center MDC 13. T-BOX 12 can be used to realize the communication connection between the vehicle 100 and the internal and external devices of the vehicle. T-BOX can obtain the data of the in-vehicle devices through the bus of the vehicle 100, and can also communicate with the user's mobile phone through a wireless network. In some embodiments, T-BOX 12 can be included in Figure 1 The mobile data center MDC 13 is used to output driving, transmission, steering and braking execution control instructions based on core control algorithms such as environment perception positioning, intelligent planning decision and vehicle motion control to achieve automatic control of the vehicle 100, and can also achieve human-computer interaction of vehicle driving information through a human-computer interaction interface. In some embodiments, Figure 1 The computing platform 150 in can implement various functions of MDC 13.

[0152] exist Figure 2The four VIUs 11 in the network form a ring topology connection network, each VIU 11 is connected to the sensor in its neighboring position, and the T-BOX 12, CDC 13, MDC 14 and VDC 15 are connected to the ring topology connection network of the VIU. The VIU 11 can obtain information from each sensor and report the obtained information to the CDC 13, MDC 14 and VDC 15. Through the ring topology network, the T-BOX 12, CDC 13, MDC 14 and VDC 15 can also communicate with each other.

[0153] It should be understood that the above ring topology network connection is only an illustration, and those skilled in the art can select other appropriate VIU connection methods according to requirements.

[0154] The connection between VIUs can adopt, for example, Ethernet, the connection between VIU and T-BOX 12, CDC 13, MDC 14 and VDC 15 can adopt, for example, Ethernet or peripheral component interconnect express (PCIe) bus, and the connection between VIU and sensors can adopt, for example, controller area network (CAN), local interconnect network (LIN), FlexRay, multimedia data transmission (MOST) and the like.

[0155] The AUTOSAR of vehicles can be divided into AP AUTOSAR and CP AUTOSAR, and CP AUTOSAR can also be called classic AUTOSAR. Among them, CP has been widely used in traditional automotive embedded ECUs, and can meet the needs of scenarios with high functional safety and real-time requirements but low computing power requirements, including engine controllers and brake controllers. AP is a new architecture that has emerged to meet the high performance and high computing power requirements of the next generation of intelligent connected vehicles, as well as the needs of continuous connection and integration in complex scenarios such as autonomous driving and cloud computing. It will be widely used in autonomous driving vehicles in the future.

[0156] Figure 3The schematic diagram of the CP AUTOSAR architecture is shown. CP AUTOSAR can be divided into three layers from top to bottom: application software layer, runtime environment layer, and basic software layer. CP AUTOSAR runs on a microcontroller unit.

[0157] Application Software (ASW): software that implements specific application functions. It can contain multiple software components (SWC).

[0158] Runtime Environment (RTE): It separates the application software layer from the basic software layer and provides a running environment for the application layer software, such as process time slice scheduling, data exchange between application layer modules and between the application layer and the basic software layer.

[0159] Basic Software (BSW): The basic software layer consists of the following four parts:

[0160] Microcontroller Abstraction Layer (MCAL): It is the driver software directly related to the hardware, such as the operation of memory, communication registers, IO ports, etc.

[0161] ECU Abstraction Layer (ECUAL): unifies the basic functions and interfaces of the controller, such as parsing the content of CAN messages, forwarding gateway messages, controlling the memory read and write processes, etc.

[0162] Services Layer: Provides various background services for the application layer, such as network management, memory management, bus communication management services, and operating systems.

[0163] Complex Device Drivers (CDD): Provides users with the possibility to write their own special device driver software.

[0164] Figure 4 The schematic diagram of AP AUTOSAR architecture is shown in Figure 4As shown, from top to bottom, AP AUTOSAR may include a user layer (User Application), which includes multiple adaptive applications (Adaptive Application, AA); an AUTOSAR runtime adaptive application layer (Runtime for Adaptive Application, ARA), which includes at least one functional module; a POSIX layer; a virtual machine / container / operating system layer and the bottom hardware layer, which may include but is not limited to hardware devices such as CPU / SoC / FPGA / NPU / GPU.

[0165] ARA can include at least one functional unit, each of which provides a service. The services provided by ARA are called Adaptive Platform Services, including:

[0166] ara::exec Execution Management(EM)

[0167] The main function of EM is to manage the life cycle of AP platforms and applications, including starting and stopping processes (meeting process dependencies), setting resource limits (CPU and memory usage), configuring core binding, setting scheduling priorities and scheduling policies, etc. When the system starts, the OS will initialize EM as the initial process, and EM will load other functional groups. For example, setting the status of a functional group to state up means that the functional group has been started. After the basic components of the platform are running, EM will start platform applications (such as platform functional modules) and ordinary applications (such as user-installed apps) in sequence according to the Machine Manifest and Execution Manifest configuration files.

[0168] ara::sm State Management(SM)

[0169] EM is responsible for loading, starting, and shutting down FC and AA, while SM is responsible for when to load and start, and when to shut down. In addition, the main function of SM is to manage the operating mode of the normal operation of the Machine, which is similar to the operating mode management of the ECU in the CP. The relationship between SM and EM in AP is similar to the relationship between BswM and EcuM in CP.

[0170] ara::phm Platform Health Management(PHM)

[0171] The health status of the application is managed, which mainly includes multiple error detection functions: Alive Supervision, Deadline Supervision, Logical Supervision, and Health Status, and can trigger corresponding recovery actions for the errors found.

[0172] For the aforementioned vehicle architecture, such as the CP platform or the AP platform, when implementing vehicle functions, such as the throttle, brake, steering or entertainment system, one or at least one function in the vehicle can usually be implemented through one or more behavioral models, and each function can be implemented through one or more behavioral models. Usually, in order to improve the reliability of vehicle equipment, the models of various functions of the vehicle can be verified.

[0173] The present application relates to the field of smart cars. To facilitate understanding, some concepts related to vehicle development are explained below.

[0174] Message passing: A general term for a type of data communication method between processes or software components in a computer system. It abstracts and encapsulates the data to be communicated into "messages". The two or more parties involved in the communication can transfer messages between processes or components by calling primitives such as message sending and receiving, thereby completing data communication.

[0175] Data flow diagram: A graphical data structure that reflects the design principles and implementation process of computing logic by expressing the flow direction and computing relationship of data in computing logic.

[0176] Data flow graph parameters: In a data flow graph, parameters refer to the data carried by the connecting edges of the computing nodes on the graph, which are used for computing node processing or feedback by the computing nodes.

[0177] Autonomous driving intelligent operating system: An operating system based on the POSIX standard, suitable for the high-performance computing and high-bandwidth communication required for autonomous driving, providing functions such as vehicle environment perception, intelligent decision-making and path planning.

[0178] Application node: In the autonomous driving intelligent operating system, an application with specific autonomous driving functions (such as perception, fusion and planning) encapsulated by communication middleware (such as ROS, AUTOSAR CM, etc.), usually a collection of single processes and multiple threads. Two or more nodes implement the transmission of messages and service calls between processes or components by calling the communication middleware's message sending and receiving APIs.

[0179] Message channel (Topic): In the autonomous driving intelligent operating system, a communication channel designated between application nodes, which consists of the sender's sending port, the receiver's receiving port, the channel name, the message format, etc., and can be used to receive or send messages.

[0180] Formal Method: In the field of computer science and software engineering, formal methods are special techniques based on mathematics, suitable for the description, development and verification of software and hardware systems. The use of formal methods in software and hardware design is expected to improve the reliability and robustness of the design by using appropriate mathematical analysis, just like other engineering disciplines. Generally, formal methods are expensive and are generally only used to develop highly integrated systems that focus on safety.

[0181] Formal Verification: A verification method based on formal methods. In the design process of computer hardware (especially integrated circuits) and software systems, the meaning of formal verification is to use mathematical methods to prove the correctness or incorrectness according to one or more formal specifications or properties.

[0182] Model Checking: In computer science, model checking is a method of checking whether the finite state model of a system satisfies a given specification (also called correctness). In simple terms, it is a method of exhaustively traversing the finite state space.

[0183] Traditional autonomous driving OEMs focus on implementing the AUTOSAR CP function set of suppliers at all levels, but performance issues (such as insufficient computing power and bandwidth) are often discovered only after integrating the applications of various suppliers. For example, a complete function is distributed to TIER1 manufacturers to complete. Then the TIER 1 decomposes the function into multiple sub-functions and distributes them to several TIER 2 manufacturers for implementation. TIER 2 manufacturers independently develop their own sub-modules, and deliver them to TIER 1 after verifying that they can implement the functions required by TIER 1; TIER 1 also integrates and verifies the sub-function modules delivered by all TIER 2 manufacturers, and delivers them to the OEM after meeting the functional requirements proposed by the OEM. Among the many TIER 1 delivered function modules received by the OEM, if some function modules use the same MCU hardware, the OEM may discover performance issues during integration verification, such as insufficient overall computing power and unschedulable tasks on the CPU. However, this problem is difficult to discover during the development of TIER 1 and TIER 2, because they are only responsible for the functional implementation of their own modules, and lack an integrated verification environment, so it is impossible to verify whether there are performance issues.

[0184] In the above issues, the AUTOSAR AP field faces greater challenges, because the design, architecture, and functional logic of the operating system and middleware in the AP field are much more complex than those in the CP field. For example, most AP-side applications run on the Linux operating system, which usually has a large number of dynamic memory requests. When using multi-threaded parallel computing optimization, they will face complex scheduling and load balancing operations of the operating system. In addition, the use of a large number of uncertain and random AI algorithms leads to unstable application running time and frequency, making it difficult to apply the traditional method of using the worst-case runtime delay to optimize scheduling and scheduling, and even more difficult to evaluate and optimize performance issues.

[0185] For example, in an existing solution, the solution includes an automated development framework and a load orchestration system. The automated development framework aims to shorten the development cycle by integrating the development process. In addition, the running time of the application on the embedded platform is estimated based on the learning and performance inference program, and the program is mapped to the hardware resources based on the application automatic mapping algorithm. The load orchestration system is based on the solution of simulated annealing algorithm and EDF scheduling simulation, which can generate static task scheduling table and task binding core scheme at the same time, and can better meet the real-time requirements of the task than the traditional method. The load orchestration system of this solution provides a method of using data-driven scheduling strategy combined with static task scheduling and orchestration to maximize CPU resource utilization and reduce waste problems, but lacks optimization considerations for other hardware resources such as GPU, ISP and other resources, and lacks optimization considerations for the default fair scheduling algorithms (such as CFS scheduling) used by Linux, etc., making it difficult to accurately estimate the delay of the current AI algorithm-driven autonomous driving system applications developed by the new forces of autonomous driving, and provide accurate end-to-end delay optimization capabilities for applications running on complex systems such as Linux at the design stage.

[0186] For another example, in an existing solution, in order to make full use of the service resources of the autonomous driving vehicle itself and the service resources of the surrounding nodes, the service resources are optimized for task scheduling in the constructed task scheduling model, and the autonomous driving vehicle schedules the corresponding tasks according to the optimized task queue. The use process of this solution is to obtain the service resources of the autonomous driving vehicle itself and the service resources of the surrounding nodes; optimize the task scheduling for the service resources in the task scheduling model to obtain the optimized task queue; and send the task queue to the autonomous driving vehicle so that the autonomous driving vehicle calls the corresponding surrounding nodes to perform tasks in sequence based on the task queue. Make full use of the computing resources of other computing nodes or autonomous driving vehicle nodes in the surrounding network, and provide better scheduling optimization for all service resources through the constructed task scheduling model to improve the overall performance and reliability. However, this technology does not lack consideration of task scheduling and arrangement on autonomous driving vehicle chips and systems, especially the current autonomous driving vehicle chips have sufficient computing power to support the independent operation of autonomous driving services without the assistance of external computing resources. In addition, the relationship between autonomous driving applications is complex, and how to more efficiently expose and optimize these problems in advance in the design stage is also something that this solution cannot do simply by patching.

[0187] For another example, in some existing solutions, the Linux scheduler running in kernel state is ported to the user state application, and the Linux scheduler operation is simulated to provide users with a better means of observing and debugging the scheduler. The simulated scheduling results are highly similar to the actual operation results of the Linux scheduler. Because the Linux scheduler code is ported and simulated in user state, the scheduling results will be very similar to Linux and have high fidelity. However, this technology is highly dependent on kernel code, and each new version of Linux needs to update and port new code, which requires a lot of manpower; further, this technology cannot simulate scheduling mechanisms that Linux does not support, and it is very difficult to add new scheduling mechanisms because the Linux scheduler code and other kernel modules are highly coupled, and one move can affect the whole body. In addition, this solution only supports high-fidelity Linux scheduling, lacks simulation of end-to-end latency of key mission links of autonomous driving, and cannot simply provide an end-to-end latency optimization solution, let alone an optimization solution in the design stage.

[0188] For example, some existing solutions provide a formal framework for calculating the end-to-end delay of multiple data links for the AUTOSAR autonomous driving system, including the maximum fastest response delay and the longest data life. The solution provides several practical implementation examples to demonstrate the significance and effect of end-to-end delay calculation. However, it mainly considers the end-to-end delay in the AUTOSAR CP architecture system (i.e., OSEK system). Compared with general operating systems such as Linux, which are more used by high-level autonomous driving manufacturers, the system is relatively simple, especially its scheduling system is simple, without considering CFS fair scheduling, load balancing scheduling, etc. The time window algorithm used by CPA is difficult to support the end-to-end delay calculation of Linux scheduling scenarios; in addition, the AUTOSAR CP autonomous driving application targeted by the solution is relatively simple and fixed than the AUTOSAR AP autonomous driving application, because the AP application based on AI computing and data-driven is highly uncertain and random, and the CPA input delay model cannot support the characteristics of dynamic running time and running cycle of AP applications, and it is difficult to apply on the AP platform.

[0189] Based on the existing vehicle development solutions, the problems solved by this application include: how to enable the traditional CP-side thread running simulator for static running scenarios and simple OS to more accurately simulate the running process of AP-side applications for AI algorithms and data-driven, rich-ecosystem OS, and calculate the corresponding end-to-end latency, which is a very important point; how to achieve early E2E latency optimization of autonomous driving services in the design stage from the development process, and provide an optimization method that seamlessly connects the design stage to the development state and the running state, reducing user usage and learning costs and improving optimization efficiency.

[0190] First of all, the method provided in the present application can be deployed in a device that performs vehicle function planning or development. Among them, the functions for vehicle function planning and development can be deployed in different devices. For example, it can usually be that the original equipment manufacturer (vehicle manufacturer) (original equipment manufacturer, OEM) puts forward requirements to the first-tier supplier (Tier-1), Tier-1 integrates the configuration, generates the dynamic code of the configuration, and then generates the bottom software static library. Tier-1 provides interface files to the OEM, and the OEM develops related business to compile and generate the OEM static library. The bottom software static library and the OEM static library are linked into an executable file, and after the reliability of each function is verified, it can be further burned into the vehicle's microcontroller unit (MCU).

[0191] For example, the system provided by the present application can be Figure 5As shown, the system provided by the present application may include OEM and Tier-1, and the number of Tier-1 may include one or more. In addition, it may also include Tier-2 or Tier-3, etc., which will not be described in detail in the present application.

[0192] A wired or wireless communication connection is established between the OEM and Tier-1 for data transmission.

[0193] The OEM can first determine the functions to be implemented, and plan the end-to-end latency of the tasks to implement each function based on software and hardware resources, generate actual demand data for each function based on the latency, and send it to one or more Tier-1s.

[0194] Tier-1 can generate behavioral models corresponding to each function based on the OEM's requirements and send them to the OEM after verifying that they meet the OEM's requirements.

[0195] OEMs can receive models from one or more Tier-1s and evaluate whether each function meets the requirements. If the evaluation results are passed, they can be integrated into the vehicle architecture.

[0196] It should be noted that the method provided in this application may include multi-party interaction. This application exemplarily introduces OEM (also known as the first development equipment) and Tier-1 (also known as the second development equipment) as examples.

[0197] In the method provided in the present application, during the process of the OEM generating demand data, the delay of the tasks corresponding to one or more functions to be implemented can be simulated and calculated, that is, the end-to-end delay for implementing each function can be calculated, and the end-to-end delay for implementing each function can be planned based on the computing resources of the vehicle to be deployed, that is, the resources corresponding to the tasks of each function are scheduled, and the demand data is generated based on the scheduling results. The demand data can be used to generate a behavioral model for implementing one or more functions.

[0198] First, the method flow provided by the present application is summarized, for example, the process of OEM planning resources for implementing various functions is summarized.

[0199] See also Figure 6 , a flow chart of a task scheduling method provided in this application is as follows.

[0200] 601. Obtain estimated resources corresponding to each task in at least one task to be implemented in a vehicle.

[0201] Generally, the OEM can determine one or more functions of the vehicle to be implemented, each of which can be implemented by one or more tasks, or in other words, the functions in the vehicle can be implemented by executing one or more tasks. Therefore, after determining one or more functions in the vehicle to be implemented, the one or more functions can be decomposed into the tasks to be performed.

[0202] For example, each function can be decomposed into one or more tasks, such as functions that need to be calculated, functional fragments in autonomous driving, etc., which is equivalent to decomposing the functions into smaller-grained tasks that are easier to implement.

[0203] In a possible scenario, before step 601, user input data may be obtained, and the user input data may include the computing resources and delay model information of the vehicle. Among them, the functions in the vehicle can be implemented by the computing resources available in the vehicle, and the computing resources may specifically include but are not limited to: CPU resources, GPU resources, memory resources, network card resources, hardware accelerators or storage resources, such as the CPU Soc model, memory size, network module model, NPU model, GPU model or key delay link information, so as to support the implementation of one or more functions of the vehicle through these computing resources. The delay model information may specifically include but is not limited to: one or more of thread runtime delay, operating frequency, trigger relationship between tasks or number of threads, etc.

[0204] In addition, optionally, the user input data may also include an algorithm database, which may include a variety of algorithm models. The multiple algorithm models can be used to implement a variety of tasks, which may generally include tasks in one or more of the aforementioned functions to be implemented.

[0205] After determining one or more tasks corresponding to one or more functions to be implemented (this application takes the implementation of multiple tasks as an example), in order to facilitate the implementation of the function of latency and vehicle hardware adaptation, the resources for implementing each function can be planned, and the estimated resources corresponding to each task can be output. The estimated resources are the resources used to run each task.

[0206] In a possible scenario, a feasible solution space can be constructed based on computing resources and delay model information of each task; feasible solutions are searched from the feasible solution space to obtain estimated resources corresponding to each task.

[0207] For example, the estimated resources allocated to each task may include but are not limited to one or more of the following: operation cycle, operation time probability model, memory bandwidth or network bandwidth. The operation cycle includes the operation time range of each function, the operation time probability model includes the operation time of each function as the probability of each operation time, the memory bandwidth is the bandwidth occupied by the memory used by each function during operation, and the network bandwidth is the bandwidth of the communication network used by each function during operation. Therefore, resources of various dimensions can be allocated to each task, so that the use of resources for each task can be planned more comprehensively later.

[0208] Optionally, the estimated resources allocated to each task may also include at least one of the priority of each task or the core binding strategy. The core binding strategy is the core of the CPU running the task. Usually, the priority or core binding strategy of each task can be adjusted in the next iteration process to obtain a resource scheduling result that is more suitable for the computing resources.

[0209] 602. Obtain the latency of running each task in the estimated resources.

[0210] After allocating estimated resources to each task, the delay of running each task in the estimated resources may be calculated, or in other words, the delay of each task when running in the corresponding estimated resources.

[0211] Specifically, at least one thread corresponding to each task may be run in the estimated resources, so as to obtain the delay corresponding to each task according to the running data generated by the at least one thread running in the estimated resources.

[0212] Optionally, when running each task in the estimated resources, the task can be run through a simulator or through a device configured with the estimated resources. The specific selection can be based on the actual application scenario, and this application does not limit this. For example, the estimated resources can be configured in a simulator, and each task can be run in the simulator configured with the estimated resources; or, when hardware resources are available, the estimated resources can be allocated to each task in the hardware resources and the tasks can be run, that is, the aforementioned device configured with the estimated resources can be a hardware resource configured corresponding to the estimated resources, or the estimated resources can be allocated separately for the tasks that need to be run in hardware with more hardware resources.

[0213] In the following embodiments of the present application, for example, running each task in a simulator is used for introduction. The running of tasks through a simulator mentioned below can also be replaced by running each task in hardware resources, which will not be repeated below.

[0214] Optionally, the present application provides a thread running simulator, in which each task can be mapped to one or more threads in the thread running simulator, that is, the thread corresponding to each task can be simulated in the thread running simulator, and the delay corresponding to each task can be calculated based on the running data of each thread of each task in the thread running simulator, that is, the data generated during the running process.

[0215] Usually, an algorithm database is set up in the OEM, and the algorithm database may include multiple algorithm models, and the algorithm database may include multiple algorithm models and corresponding runtime probability models, and the runtime probability model may be used to represent the probabilities corresponding to the runtimes of the multiple algorithm models included in the algorithm database when they are run in a specific hardware platform. The runtime probability model may be used to determine the possible runtime of each thread when simulating the thread corresponding to each task in the thread running simulator, so as to calculate the delay of each task.

[0216] Optionally, when running the thread corresponding to each task in the thread running simulator, status data can be set for each task. The status data of each task can include data generated when each task is running in the thread running simulator. For example, the data generated when each task is triggered to execute in the thread running simulator can be recorded. Based on the status data of each task, the delay model information corresponding to each task can be obtained.

[0217] Optionally, the state data of each task may include one or more of the following: receiving state data, sending state data, computing state data or context state data, wherein the receiving state data may include information of a message queue of task receiving data, the sending state data includes trigger information of task sending data, the computing state data includes data generated by triggering the execution of the task, and the context state data includes configuration data generated when indicating that the task enters a preset state when preset context conditions are met. In the implementation manner of the present application, the data during the task triggering or running process can be simulated by setting state data for each task, and even the interaction between complex tasks can be simulated through the state data, so that the runtime delay of the task can be simulated more accurately.

[0218] In a possible scenario, the process of running tasks in a thread running simulator may include: taking any one of the tasks as an example, for the sake of distinction, it is called the first task, if the first task is in a receiving state, that is, a state of receiving data, then the receiving state data of the first task is updated; if the triggering conditions of the first task are met, such as the completion of receiving data or being triggered by the task it depends on, etc., then the first task can be triggered to execute, and the computing state data of the first task is updated after the first task is executed; if the triggering conditions of the first task are met and the context conditions of the first task are also met, then the context state data corresponding to the first task can be adjusted, and the first task can also be executed according to the adjusted context state data, and the computing state data corresponding to the first task is updated after the first task is executed.

[0219] Therefore, in the implementation of the present application, different state data can be used to simulate or record the running process of the task, so as to achieve a more accurate estimate of the delay of each task. Even for tasks with complex dependencies, the state data provided by the present application can be used to simulate the running of each task, so as to calculate a more accurate delay.

[0220] In addition, in one possible scenario, the OEM may not have a vehicle hardware platform corresponding to the computing resources in the input data, or the specific hardware platform on the OEM side, that is, the hardware platform corresponding to the runtime probability model in the algorithm database, is different from the hardware platform of the vehicle to be developed. At this time, in order to facilitate the subsequent calculation of the delay of each subsequent task, the algorithm model can be migrated. That is, the OEM's existing hardware platform is used to estimate the possible running time and corresponding probability of each algorithm model in the vehicle hardware platform, that is, the runtime probability model that the vehicle hardware platform may correspond to.

[0221] Specifically, the above process can be called the model migration process, which can be expressed as:

[0222] If the thread running simulator does not include a running time probability model corresponding to the hardware platform indicated by the computing resource, an algorithm database is obtained, the algorithm database includes multiple algorithm models and a first running time probability model corresponding to a preset hardware platform, the first running time probability model includes the running time and corresponding probabilities of the multiple algorithm models running in the preset hardware platform, and the algorithm database is used by the thread running simulator when running at least one task;

[0223] If there is a real hardware platform adapted to the preset hardware platform, such as the real hardware platform is the same as the preset hardware platform or the same Soc, and multiple algorithm models can be run in the real hardware platform, then obtain the corresponding second runtime probability model in the real hardware platform, the second runtime probability model includes the runtime of multiple algorithm models running in the real hardware platform and the corresponding probabilities, and according to the ratio of the running time of multiple algorithm models in the preset hardware platform and the real hardware platform, obtain the average running time of multiple algorithms;

[0224] If there is a real hardware platform that is adapted to the preset hardware platform, and multiple algorithm models cannot be run in the real hardware platform, then the running time corresponding to multiple instructions in the algorithm database in the preset hardware platform is obtained, and the running time of multiple instructions in the real hardware platform is obtained. According to the ratio between the running time corresponding to the multiple instructions in the preset hardware platform and the running time in the real hardware platform, the corresponding third running time probability model in the real hardware platform is obtained.

[0225] Therefore, in the implementation manner of the present application, even if the OEM does not have a hardware platform compatible with the vehicle, the runtime probability model that may correspond to the vehicle hardware platform can be calculated by model migration, so that the runtime delay of each task can be calculated more accurately.

[0226] 603. Obtain evaluation results according to the latency of each task.

[0227] After obtaining the time delay of each task, the strategies of allocating estimated resources to each task can be evaluated based on the time delay of each task to obtain an evaluation result. The evaluation result can be understood as the result of evaluating the strategy of allocating estimated resources to each task in the current solution.

[0228] In a possible scenario, the evaluation result can also be generated by combining the usage rate of computing resources when simulating the running of each task in the thread running simulation, such as the CPU usage rate or the GPU usage rate. For example, the latency of each task and / or the usage rate of computing resources when simulating the running of each task in the thread running simulation can be used as the evaluation result.

[0229] 604. Obtain a scheduling result according to the evaluation result, where the scheduling result is used to indicate a strategy for using computing resources to implement at least one task.

[0230] After obtaining the evaluation result, it can be determined whether the current resource allocation strategy meets the requirements based on the evaluation result, so as to determine the final scheduling result based on the judgment result.

[0231] In the implementation of the present application, the delay of each task in running in the allocated estimated resources can be calculated, the strategy of allocating resources to the task can be evaluated according to the running delay of each task, and the final resource scheduling result can be obtained according to the evaluation result. Thus, a scheduling result adapted to the computing resources of the vehicle can be obtained.

[0232] In one possible implementation, it can be determined based on the evaluation results whether the convergence condition is met; if so, the estimated resources corresponding to each task are used as the scheduling result; if not, the estimated resources corresponding to each task are adjusted, such as adjusting the task priority, operation cycle or core binding strategy, and the new estimated resources are evaluated until the convergence condition is met and the final scheduling result is output. It can be understood that steps 601 to 604 can be repeated until the convergence condition is met. Therefore, in the implementation of the present application, the scheduling strategy can be adjusted in an iterative manner, that is, the strategy for allocating available resources to each task, so that through multiple optimizations, a scheduling result that is more adapted to the vehicle's computing resources can be obtained.

[0233] Optionally, the aforementioned convergence conditions may include but are not limited to one or more of the following: the delay of each task does not exceed the corresponding delay threshold, the utilization rate of computing resources when implementing each task is higher than a first threshold or lower than a second threshold, the delay change value of the task between multiple iterations does not exceed a preset change threshold, the number of iterations reaches a preset number of times or the iteration duration reaches a preset duration, etc., so that convergence can be identified from multiple dimensions.

[0234] It can be understood that, in the process of resource scheduling, a feasible solution space can be constructed based on the computing resources and the delay model information corresponding to each task, and the feasible solution space can include computing resources, one or more tasks, and the delay model information corresponding to each task; then the feasible solution space is searched iteratively multiple times to obtain the final converged scheduling result, which can include the priority of each task and the resources used to run each task. Therefore, in the implementation of the present application, a feasible solution space can be constructed based on the computing resources of the vehicle and the delay model information corresponding to each task, and a scheduling result that meets the requirements can be obtained through iterative search in the feasible solution space.

[0235] Optionally, taking any iterative search as an example, the specific process of iterative search may include: searching for the current feasible solution from the feasible solution space, each feasible solution can be understood as: allocating corresponding delays and resources to be used for each task; generating simulator configuration data according to the current feasible solution, such as allocating computing resources for each task and configuring them in the simulator; configuring and running the simulator according to the simulator configuration data, so as to obtain the evaluation result corresponding to the current feasible solution according to the running result of the simulator; if the evaluation result of the current feasible solution meets the convergence condition, the current feasible solution is used as the scheduling result; if the evaluation result of the current feasible solution does not meet the convergence condition, the feasible solution of the next iteration is searched again from the feasible solution space, such as the current feasible solution can be adjusted to obtain the next iteration. Therefore, in the implementation manner of the present application, the feasible solution of each iteration can be evaluated by iterative search and running in the simulator, that is, simulating the running of the feasible solution in the vehicle, so that it can be identified whether the feasible solution meets the delay adapted to the computing resources of the vehicle according to the evaluation result.

[0236] Optionally, when configuring and running the simulator according to simulator configuration data generated by the feasible solution, the data generated when the simulator runs each feasible solution can be collected, such as the corresponding resource usage when the simulator is running, so as to evaluate the feasible solution based on the usage of computing resources. For example, the resource usage rate when running the simulator can be obtained, such as CPU usage rate, GPU usage rate or NPU usage rate, etc., and the evaluation result can be generated according to the resource usage rate. So that it can be judged whether it has converged according to the evaluation result later.

[0237] In a possible implementation, the aforementioned scheduling results may specifically include: operating cycle, operating time probability model, memory bandwidth or network bandwidth, etc., wherein the operating cycle is the operating time range of each function or each task, the operating time probability model represents the probability of each function or each task being each operating time, the memory bandwidth is the bandwidth occupied by the memory used during the operation of each function or each task, and the network bandwidth is the bandwidth of the communication network available during the operation of each function or task. Therefore, resources are allocated to multiple dimensions of each function or each task, and resource scheduling is achieved from multiple dimensions. This is equivalent to more comprehensively implementing the scheduling of available resources in the vehicle, obtaining more accurate scheduling results, and making the final vehicle functions more compatible with the vehicle's computing resources.

[0238] In addition, in a possible implementation, a time series visualization analysis result can also be generated based on the scheduling result, and the time series visualization analysis result includes a list of tasks that are run on various types of resources in the computing resources according to the priorities included in the scheduling result. It can be understood that the scheduling result includes the priorities corresponding to each task and the resources allocated to it, and a visualization analysis result of running each task in each resource can be generated, so that subsequent users can know the running time of the tasks corresponding to the vehicle function to be developed based on the visualization analysis result.

[0239] In one possible implementation, after obtaining the scheduling result, the OEM may continue to develop the function, that is, the OEM may generate a behavior model for realizing one or more functions of the vehicle, or the OEM may send the scheduling result to Tier-1, and Tier-1 may develop the function.

[0240] If Tier-1 is developing functions, after Tier-1 generates one or more corresponding behavior models, it will send the information of the one or more behavior models to the OEM. After receiving the one or more behavior models sent by Tier-1, the OEM can integrate the one or more behavior models for subsequent deployment in the vehicle.

[0241] For example, the process of feature development by Tier-1 can be as follows: Figure 7 As shown, the process of the model generation method provided by this application can be as follows Figure 7 As shown below.

[0242] 701. The OEM obtains estimated resources corresponding to each task.

[0243] 702. The OEM obtains the latency of running each task in the estimated resources.

[0244] 703. The OEM root obtains evaluation results according to the latency of each task.

[0245] 704. The OEM obtains a scheduling result according to the evaluation result.

[0246] Among them, steps 701 to 704 can refer to the relevant descriptions of the aforementioned steps 601 to 604, which will not be repeated here.

[0247] 705. The OEM sends a scheduling result to Tier-1, where the scheduling result includes a strategy for instructing to use computing resources in the vehicle to implement at least one task.

[0248] After the OEM obtains the scheduling result, it can send the scheduling result to Tier-1. The scheduling result can refer to the relevant description in the above step 603, which will not be repeated here.

[0249] 706. Tier-1 obtains at least one behavior model according to the scheduling result.

[0250] Tier-1 can generate one or more behavior models based on the scheduling results, and the one or more behavior models can be used to implement one or more functions in the vehicle, wherein one behavior model can be used to perform one or more tasks in each function.

[0251] Optionally, Tier-1 may directly generate one or more behavior models based on the scheduling results, or may first generate an initial model and use the model that passes the evaluation as one or more behavior models.

[0252] Specifically, at least one model can be generated according to the scheduling results, and the delay of running the at least one model can be obtained. If the delay of running the at least one model meets the constraints, the at least one model can be used as a behavior model. The constraints can be determined specifically based on the scheduling results. For example, the scheduling results may include the operating cycle of each task, the operating time probability model, the memory bandwidth or the network bandwidth, etc., so that the operating delay range of each behavior model, that is, the constraints, can be determined based on the data carried in the scheduling results. If the at least one model does not meet the constraints, at least one new model can be generated and evaluated until a behavior model that meets the constraints is obtained.

[0253] 707. Tier-1 sends information of at least one behavior model to the OEM.

[0254] After Tier-1 obtains one or more behavior models, it can send information about the one or more behavior models to the OEM, so that the OEM can integrate the received one or more behavior models and verify them in the vehicle.

[0255] In the implementation mode of the present application, the OEM can perform scheduling by calculating the end-to-end delay of each task, assigning corresponding operation cycles and resources to each task, and sending the scheduling results to Tier-1 for function development. Tier-1 can generate a behavior model that meets the requirements based on the scheduling results and feed it back to the OEM to complete the function development of the vehicle.

[0256] In a possible implementation, after receiving one or more behavioral models sent by one or more Tier-1s, the OEM may perform a model check on the model implementing the vehicle, thereby verifying the reliability of the vehicle. For example, the model check may be performed through formal verification to obtain a vehicle with higher reliability.

[0257] The above is an introduction to the method flow provided by the present application. The following is a more detailed introduction to the steps in the method provided by the present application in combination with the system provided by the present application.

[0258] See also Figure 8 , a schematic diagram of a system architecture provided in this application.

[0259] Among them, the architecture can include multiple modules, such as Figure 8 The architecture design system 81, function development system 82 and vehicle hardware and software platform 83 shown in the figure.

[0260] The architecture design system 81 can be deployed on the OEM side, such as in a server, terminal, cloud server or other device.

[0261] The function development system 82 can be deployed on the OEM side or the Tier-1 side, such as in a server, terminal, cloud server, etc. The number of Tier-1s can be one or more. This application uses one Tier-1 as an example for illustrative introduction. The Tier-1 mentioned below can also be replaced by multiple Tier-1s. The number of Tier-1s can be determined according to the actual application scenario, and this application does not limit this.

[0262] The architecture design system 81 and the function development system 82 can be deployed in the same device or in different devices. The specific deployment can be determined based on the actual application scenario, and this application does not limit this.

[0263] The vehicle hardware and software platform 83 can be a vehicle or a simulated vehicle.

[0264] Generally, the development of vehicle functions can be divided into a design phase and an operation phase. The design phase is to design and develop the implementation methods of each function of the vehicle, and the operation phase is to integrate and deploy the developed functions in the vehicle so that the vehicle can operate based on the developed functions. In this application, the architecture design system and the function development system are to develop the functions of the vehicle in the design phase. The vehicle hardware and software platform can be used to verify the developed functions in the design phase, and can also be operated based on the functions implemented in the design phase in the operation phase.

[0265] Among them, the architecture design system can specifically include architecture modeling tools 811, vehicle-level optimization tools 812, and analysis and verification tools 813; the function development system 82 can include behavior modeling tools 821, model algorithm database 822, and measurement and verification tools 823; the software part of the vehicle hardware and software platform 83 can be divided into autonomous driving applications 831 and autonomous driving operating system (OS) 832, and the hardware part can specifically include CPU833, GPU834, memory 835, storage 836 or network card 837 and other hardware.

[0266] The architecture design system 81 can plan and optimize the architecture of the vehicle, generate function development requirements and send them to the function development system 82, and perform verification after receiving the model sent by the function development system 82.

[0267] Specifically:

[0268] Architecture Modeling Tool 811: Responsible for the design of system-level architecture models based on the AUTOSAR development process, including running entities, software component (SWC) function definition, software interface definition, and software performance definition;

[0269] Vehicle-level optimization tool 812: responsible for automatically analyzing and optimizing the utilization rate and overall performance of vehicle computing resources based on the architecture model data in the design phase;

[0270] Analysis and Verification Tool 813: Responsible for analyzing and verifying vehicle resource satisfiability and performance issues during the design phase.

[0271] The function development system 82 can develop functions based on the requirements generated by the architecture design system 81 and send them to the architecture design system after verification.

[0272] Behavior Modeling Tool 821: responsible for the design, implementation, import and export of internal behavior models of software components based on the AUTOSAR development process, as well as automatic code generation;

[0273] Model Algorithm Database 822: Responsible for providing models and generated code library implementations of AI algorithms commonly used in model-based development;

[0274] Measurement Verification Tool 823: Responsible for verifying whether the program meets the performance definition in the architecture model after behavioral model development and code automatic generation.

[0275] The vehicle hardware and software platform 83 can be used to run or verify the functions implemented by the architecture design system and the function development system. This application is illustratively introduced by taking the vehicle as an autonomous driving vehicle as an example. The autonomous driving vehicle or function mentioned in this application can also replace other vehicles or corresponding functions, such as driving assistance functions or other driving functions.

[0276] Autonomous driving application 831: Responsible for calculating the autonomous driving vehicle control signals or instructions based on the original signals and data collected by the autonomous driving hardware through algorithms including but not limited to perception, fusion, prediction, planning and control;

[0277] Autonomous driving OS 832: Responsible for abstracting and managing the hardware resources of autonomous driving vehicles, providing resources and services for upper-level autonomous driving applications. For example, the architecture of the autonomous driving OS can be as described above. Figure 3 CP platform as shown or Figure 4The AP platform shown;

[0278] CPU 833, GPU 834: provide necessary computing resources for autonomous driving systems and applications;

[0279] Memory 835: Provides memory resources necessary for computing for autonomous driving systems and applications;

[0280] Network card 836: provides network communication resources for autonomous driving systems and applications;

[0281] Memory 837: Provides general storage resources for autonomous driving systems and applications.

[0282] It can be understood that the architecture modeling tool designs and outputs the autonomous driving system-level architecture model, provides the vehicle-level optimization tool with analysis and search for the optimal performance optimization configuration solution, and verifies its effectiveness and operational feasibility through analysis and verification tools; the behavior model tool designs the internal behavior model of each software component based on the system-level architecture model and the algorithm model library, automatically generates the code and binary of the autonomous driving application, and then verifies it with the measurement verification tool; the autonomous driving application obtains the CPU, GPU, memory, network card, storage and other resources on the autonomous driving hardware platform that are necessary for the normal operation of the autonomous driving through the autonomous driving operation OS.

[0283] Combined with the above system architecture, the steps performed by each part can be referred to Fig. 9 .

[0284] 1. Architecture Design System

[0285] OEMs can use architecture modeling tools to generate an overall architecture model that complies with the AUTOSAR specification, which contains planned behavioral model attributes, including but not limited to the expected functional task running time and frequency, which can be exported as an ARXML file. The behavioral model attributes can also be called the estimated resources allocated to each task, or the allocation plan for allocating resources to each task.

[0286] The vehicle-level simulation analysis and optimization system includes multiple tools, such as a domain-specific thread operation simulator (such as the thread operation simulator mentioned in the aforementioned step 603) and a task scheduling and orchestration optimizer. The vehicle architecture model and the planned behavior model attributes are input, and the system performance is analyzed by simulating the task thread operation. At the same time, a performance optimization strategy for the design state is generated.

[0287] The computing and communication satisfiability analysis module and the performance optimization strategy verification module can be used to analyze and verify whether the current architecture model, the planned behavior model attributes and the performance optimization strategy are feasible. If the verification fails (such as not being able to run normally), it is necessary to return to the architecture modeling tool for adjustment. For example, the computing and communication satisfiability analysis module, which can also be called an evaluation module, can be used to evaluate the planned behavior model attributes, and identify whether the behavior model attributes are compatible with the vehicle's resources based on the simulation results output by the thread running simulator. The performance optimization strategy verification module can be used to schedule and orchestrate the planned behavior model attributes, such as adjusting the behavior model attributes until the behavior model attributes that are analyzed as being able to run normally by the computing and communication satisfiability analysis module are obtained.

[0288] It can be understood that the computing and communication satisfiability analysis module is used to evaluate the planned behavior model attributes based on the simulation results of the thread running simulator. If the evaluation fails, the behavior model attributes are adjusted. For example, the scheduling method of each task can be optimized through the task scheduling optimizer, and the simulation and evaluation are performed again through the thread running simulator until the behavior model attributes that pass the evaluation are output, that is, the strategy for allocating resources to each task.

[0289] 2. Function Development System

[0290] TIER suppliers obtain the ARXML files belonging to their respective TIER suppliers exported from the architecture design system by the OEM, and use the ARXML files as input to the behavioral modeling tool to implement the functional logic of each specific software module. The commonly used algorithm models in the algorithm database provided in this application can be used to implement the required behavioral models.

[0291] The behavior model can be realized through performance measurement tools and algorithm database selection tools. Among them, the performance measurement tool will give an estimate of the running time on different heterogeneous hardware platforms; if the measured running time on the vehicle hardware platform does not meet the planned behavior model attributes, then TIER needs to return to the behavior modeling tool for re-modification and generate a new behavior model until the behavior model attributes that meet the planned running time on the vehicle hardware platform are obtained.

[0292] For example, taking the method provided in this application deployed in the modeling development system HAS Studio as an example, it usually includes multiple development systems: such as an architecture design system or a function development system.

[0293] In the architecture design system, HAS Studio Architect provides OEMs and other developers with the ability to develop architecture modeling. Developers can add behavioral model attributes that comply with the AUTOSAR specification in the Architect tool and export them together with the architecture model as AUTOSAR ARXML files. The domain-specific thread operation simulator and task scheduling and orchestration optimizer of HAS Studio DET can simulate the operation of all software modules on computing resources and generate the best performance optimization solution; HAS Studio DET derivative modules can provide the functions of computing and communication satisfiability analysis module and performance optimization strategy verification module.

[0294] In the function development system, HAS Studio Behavior can provide modeling developers with the ability to develop and implement the internal behavior of software functions, as well as formal verification, automatic code generation and other capabilities. The algorithm database selection tool can provide the most suitable algorithm database for the behavior model implementation, ensuring that the functional logic and required performance required by the algorithm used in the model are provided. The performance measurement tool can help function developers check whether the internal behavior of the implemented behavior model meets the planned behavior model attributes.

[0295] 3. Vehicle Software and Hardware Platform

[0296] The autonomous driving OS will abstract the underlying hardware, including but not limited to CPU, GPU, memory, network card and storage resources, and provide management and usage APIs for use by upper-level autonomous driving applications.

[0297] For example, this application takes the MDC 610, a software and hardware platform for autonomous driving vehicles, as an example. The hardware of the autonomous driving software and hardware platform can adopt the MDC 610 hardware platform, in which AICPU and AICore provide CPU and GPU computing resources respectively, LPDDR provides the memory resources necessary for calculation, Eth network card provides Gigabit network card communication capability, and UFS provides large-capacity storage resources; its software is based on the autonomous driving application ADS App and the intelligent driving operating system AOS.

[0298] OEMs integrate autonomous driving applications provided by different TIERs. These applications mainly come from binary programs automatically generated and compiled by the function development system. The interaction between applications theoretically conforms to the definition of the architecture model. The application performance (such as runtime latency) on the autonomous driving vehicle hardware platform should be consistent with the planned behavior model attributes and the measured behavior model attributes, that is, the scheduling results.

[0299] The scheduling and orchestration deployment module will deploy the performance optimization strategies generated in the design phase (such as the generated behavior models and the corresponding resource scheduling results) directly on the autonomous driving OS and applications. OEMs can verify whether the expected optimization goals are achieved.

[0300] The following is an introduction to the method flow provided by this application in combination with the aforementioned architecture.

[0301] See also Fig.10 , a flow chart of another resource scheduling method provided by the present application is as follows.

[0302] First of all, it should be noted that in the process of OEM scheduling and arranging vehicles based on end-to-end latency optimization, the architecture design can be iteratively performed until the functions that meet the requirements are integrated for the vehicle. It can be understood that the OEM can estimate the initial estimated resources required for each application based on the autonomous driving application to be implemented, and then iterative optimization can be performed, that is, steps 1001 to 1006 can be iteratively executed. The following only takes one of the optimization processes as an example for introduction, and the iterative process is not repeated.

[0303] 1001. OEM plans initial behavior model attributes.

[0304] OEMs can use the aforementioned architecture design system to build an architecture model of the required application based on the autonomous driving application (application can also be called function) to be implemented. The architecture model can be understood as a functional architecture planned for the vehicle as a whole, which can include multiple functions in the vehicle. And based on the algorithm model in the algorithm database and the computing resources available for the autonomous driving vehicle, the initial estimated resources required for each application are estimated.

[0305] Among them, the algorithm database may include multiple algorithm models and corresponding runtime probability models. The multiple algorithm models can be understood as multiple algorithms pre-set to realize vehicle functions. When performing function development or simulation operation, various tasks can be realized or simulated based on the multiple algorithm models. The runtime probability model may include various runtimes and corresponding probabilities of the multiple algorithm models running on a specific hardware platform. Usually, when simulating the algorithm model runtime in a thread running simulator, the runtime probability model can be combined to calculate the possible runtime of the running algorithm model, so as to facilitate the subsequent calculation of the runtime corresponding to each task, that is, the runtime delay.

[0306] For example, the initial operation cycle, operation time probability model, DDR bandwidth or network bandwidth and other information are estimated for each application, and recorded as the initial behavior model attribute of the plan, that is, the estimated resources. For example, each application may include multiple tasks, each task may be a calculation function or a functional fragment in the application, etc. For each task, the initial estimated resources may be allocated, such as the corresponding operation cycle operation time probability model, DDR bandwidth or network bandwidth and other information, so as to obtain the operation cycle operation time probability model, DDR bandwidth or network bandwidth and other information corresponding to each application.

[0307] 1002. OEM estimates end-to-end latency.

[0308] After allocating initial estimated resources to each application, the OEM can estimate the end-to-end latency of each application based on the allocated initial estimated resources, that is, the latency required to implement each function.

[0309] In order to achieve the estimation of the end-to-end delay of each task, in an embodiment of the present application, the operation process of each task can be simulated by a thread operation simulator. In addition, the embodiment of the present application can simulate tasks with complex dependencies by setting multiple states for each task. It can be understood that the present application also provides a task / thread state machine for simulating the states of tasks or threads with dependencies.

[0310] The task / thread state machine and thread operation simulator provided by this application are introduced below.

[0311] 1. Task / Thread State Machine

[0312] Typically, each application may include one or more tasks, each task may be a computing function or functional fragment in the application, etc., each task may be mapped to one or more threads, and the threads may have dependencies or trigger relationships.

[0313] In the thread operation simulator provided by the present application, in order to support more complex dependencies between tasks, state data can be set for the thread. For example, a thread may have multiple receiving channels or multiple sending channels, or a communication delay needs to be set, or the context needs to be modified according to the running state of the model.

[0314] In the present application, the state of a thread can be divided into multiple types, such as receiving state, sending state, computing state or context state, etc. Accordingly, the state data set for a thread may include but is not limited to one or more of the following state data: receiving state data, sending state data, computing state data or context state data, the receiving state data includes information of a message queue for task receiving data, the sending state data includes triggering information for task sending data, the computing state data includes data generated by triggering the execution of a task, and the context state data includes configuration data generated when indicating that a task enters a preset state when a preset context condition is met.

[0315] Furthermore, the aforementioned status data is introduced in more detail below.

[0316] 1. Receiving status data: The recorded data includes the message queue of each Topic, the sequence number ID of each message, or supports modifying the depth and size of the recorded message queue.

[0317] 2. Computing status data: It can record the last triggering time of time-triggered computing tasks, the receiving Topic status of event-triggered computing tasks, the last triggering time and receiving Topic status of mixed-triggered computing tasks, or the computing time fluctuation status of computing tasks, that is, the computing time fluctuation of AI and data-driven tasks can be jointly characterized through probability models and state transitions.

[0318] 3. Sending status data: records the triggering time of the task sending, that is, the end of calculation or periodic sending; sequence number ID, which can inherit the sequence number of the specified receiving Topic, or specify the Topic sequence number customized and maintained within the task.

[0319] 4. Context state data: The thread running simulator provided in this application supports tasks entering a specific state under certain context conditions (such as meeting relevant thresholds such as time, CPU, memory, bandwidth, latency, etc.), such as changing task attributes such as task running cycle, running time probability model, frequency, etc. according to the configuration file; and changing or exiting the context state under certain context conditions, such as entering the calculation state.

[0320] In addition, in the embodiments of the present application, each task can be classified according to the trigger type, such as time-triggered computing tasks, event-triggered computing tasks, or mixed-triggered computing tasks, etc. Among them, the time-triggered computing tasks are computing tasks that need to be triggered within a preset time or tasks that have a time dependency relationship with other tasks, the event-triggered computing tasks are tasks that can be triggered after the event on which they depend is completed, or tasks that have data interaction or data dependency with other tasks, and the mixed-triggered computing tasks are tasks that have both time dependency and event dependency, and are usually triggered to perform computing tasks after the time trigger condition and / or event trigger condition are met.

[0321] It should be understood that in the embodiments of the present application, the context state is set taking into account the running time of the actual task (AP task or CP task), that is, the running time of some tasks does not fluctuate randomly, but fluctuates regularly with the context of business scenarios, time, etc. For example, in the autonomous driving ADS App, the radar perception node enters a specific state when the running delay exceeds a certain threshold, the average value of the running time model increases by 20% to 30%, and exits the state after tens of seconds in this state, and the running time model is restored to its original state accordingly. At this point, the task can be set to enter the context state to adapt to the running time of the actual application scenario.

[0322] For example, to facilitate understanding, the Topic state of the task provided by the existing solution is compared with the task / thread state machine provided by this application. The Topic state in the existing solution can be as follows: Fig.11 As shown, the task / thread state machine provided by this application can be as follows Fig.12 As shown in the figure. For example, in the AP-oriented program tasks, there may be a state, such as waiting for Topic 1 to receive data once, Topic 2 to receive data three times in a row, and then merging these data before performing the calculation of Task 1 and finally publishing Topic 3. In the existing simulator, Fig.11 The simulator shown in cannot support the simulation of the AP program logic because it cannot further simulate the operation of all nodes on the key link and ultimately calculate the end-to-end delay of the key link.

[0323] The task / thread state machine provided by this application is as follows Fig.12The task / thread state machine shown in the figure can support complex logic judgment by setting multiple states, which is more in line with the actual usage scenario where there are dependencies between tasks. For example, the operation logic of each Topic includes: only after receiving messages from Topic1 and Topic2 can computing task 1 be executed and Topic4 be sent; and it must be triggered by Topic2, that is, computing task 1 can be triggered only after the Topic2 message arrives; Topic3 ​​must receive 3 messages before computing task 2 can be triggered. After computing task 2 is completed, the message will be sent out in the latest fixed sending cycle of Topic5.

[0324] Generally, the state change of each task can be that it first enters the receiving state, that is, the state of receiving data, and updates the receiving state data when receiving data to record the data reception of the task. After the reception is completed, it can be determined whether the task is triggered to enter the calculation state, that is, to trigger the task to run. Before the task runs or during the task running, it can be judged whether the context conditions are met, such as task suspension, hardware resource adjustment, and too long task waiting time. If the context conditions are met, the context state can be entered, and the configuration of the running task can be adjusted, such as adjusting the running time of the task, the running time probability model, the running frequency and other attributes, and the context state data can also be updated. After the context state ends, or after the running task is triggered, the calculation state can be entered. After obtaining the calculation result, the calculation state data is updated. The calculation result can also be sent to other tasks, that is, enter the sending state. Of course, if the calculation result does not need to be sent to other tasks, it is also possible not to enter the sending state and complete the simulation running of the task. In the process of simulating the running of the task, the running time of the task, that is, the running time of the thread, can be calculated according to the data recorded in the data of each state, so as to calculate the end-to-end delay of each application.

[0325] For scenarios with more complex dependencies between tasks, take the processing flow of one Topic as an example. Fig.13 shown.

[0326] 1301. In the receiving state, wait to receive messages sent by other topics.

[0327] 1302. Simulate the delay of receiving the Topic and record it in the receiving status data.

[0328] 1303. After receiving the Topic message, record it in the Topic message queue.

[0329] 1304. Determine whether the receiving status of the Topic message queue can trigger the calculation status. If not, continue to wait for receiving messages sent by other Topics. If so, the task can be triggered to enter the calculation status 1305.

[0330] 1306. In addition, during the thread running process, such as for the thread of the time-triggered type computing task, it can be determined whether to trigger the computing state of the task according to the timer time.

[0331] 1307. During the task running process or before the task running, that is, when in the computing state, it is possible to determine whether to enter the context state according to the running state of the task to correct data such as the task running time.

[0332] 1308. If the context state is entered, the simulation task can continue to enter the calculation state according to the corrected time.

[0333] 1309. After the task is completed, the operation result can be updated to the calculation status data, and then the calculation status can be ended.

[0334] 1310. After the calculation state ends, or according to time determination, the task can be triggered to enter the sending state.

[0335] 1311. You can send messages to the topics that have dependencies with it, so as to complete the simulation operation of the current topic.

[0336] 1312. During the thread running process, the data generated during the thread running process can be recorded, such as the starting time of each state or the starting time difference of the timer, so that the running delay of each thread can be calculated according to the data of each state of each thread, and the end-to-end delay calculation of each task can be realized.

[0337] 2. Thread running simulator (or domain-specific thread running simulator)

[0338] Specifically, the present application provides a thread-level timing scheduling simulation simulator, that is, a thread running simulator, which can support complex scheduling strategies such as cgroup group scheduling, SMP scheduling, load balancing or multi-threaded computing. In addition, it also supports CPU, GPU heterogeneous mixed scheduling or custom scheduling.

[0339] Generally, CP-side work cannot support complex Linux scheduling strategies in reality, such as multi-threading, load balancing, CFS scheduling, group scheduling, etc. Fig.14 The autonomous driving application node data flow shown in the figure is used to simulate task operation and estimate end-to-end delay. This is because the basic scheduling unit of most current CP-side work is Task, while the basic scheduling unit of the Linux operating system used in actual autonomous driving services is Thread. There is a big difference between the two in autonomous driving services. Direct mapping will lead to an inaccurate service delay model, and the delay of the simulated operation and the final end-to-end delay will be quite different from the actual one.

[0340] Therefore, the embodiment of the present application decomposes each Task task in the autonomous driving service data flow into different Thread-level subtasks, which is equivalent to mapping to an OS-like task control block (TCB) structure and corresponding to each Thread, filling the gap between tasks and OS threads in the data flow diagram and realizing complex scheduling strategies. For example, Fig.14 A simple data flow diagram shows how tasks are decomposed into TCBs and then mapped to Threads. For example, the composition of each TCB is shown in Table 1 TCB data structure.

[0341]

[0342] Table 1

[0343] The following is an introduction to the implementation architecture of the thread-level timing scheduling simulation simulator provided in this application.

[0344] like Fig.15 As shown, this is a structural diagram of a thread running simulator provided by the present application.

[0345] The thread running simulator may include a timer module, a task generator, a task TCB manager, a thread manager, a group manager, a scheduler, a computing resource module (including but not limited to CPU, GPU, etc.), a task executor and a latency analyzer.

[0346] The scheduler supports a variety of common scheduling strategies, including but not limited to round robin (RR), first in first out (FIFO), completely fair scheduler (CFS), earliest deadline first (EDF) and time partition (TP). At the same time, the scheduler also provides a basic scheduling interface for users to expand and write scheduling algorithms and dynamically load them into the scheduler.

[0347] Computing resources support heterogeneous hardware. In this embodiment, the AICPU on the MDC 610 hardware platform is abstracted as a CPU, including 16 Cores, and each Core has its own multiple run queues (such as Ready queue, Pending queue, Running queue, Complete queue, etc.); the AICore is abstracted as a GPU, including 10 Cores, and each Core also has its own multiple run queues.

[0348] The task TCB specifies on which heterogeneous computing resources the task can run through CPU affinity.

[0349] The function of Latency Analyzer is to analyze the runtime latency of each task and the end-to-end latency of the critical task chain. It does this by obtaining the runtime results of all task TCBs from the Complete queues of all computing resources, finding the dependencies between tasks based on the task data flow graph, and then analyzing the end-to-end latency of the critical task chain.

[0350] For example, Fig.16 The simulation operation process of each module shown is described as follows.

[0351] Each time a system clock interrupt is sent from the Timer, the Executor receives it and triggers the execution of execution_for_tick() (indicating the execution tick), simulating the running Tick time of the Running queue tasks on all current computing resource modules.

[0352] Executor calls update_runtime() (used to update the runtime), which will update the data of the task TCB in each queue. If it is CFS scheduling, vruntime (indicating virtual runtime) will also be updated in the following way: vruntime = actual_runtime (actual runtime) * NICE_VALUE (indicating CFS priority) / load_weight (indicating load weight).

[0353] Executor calls update_config() (used for configuration update) to refresh whether the task TCB meets the triggering conditions, and calls update_cfg_vruntime() (used to update the virtual runtime in CFS scheduling) to refresh the vruntime of the newly created task TCB for CFS scheduling, and dynamically determines the runtime of the triggered new task through the task TCB probability model and its status.

[0354] The Scheduler obtains the running status of all current groups and task TCBs from the Group Manager and TCB Manager, and checks for each core whether a new schedule needs to be initiated based on the current scheduling algorithm.

[0355] Executor checks each computing resource from the Scheduler's scheduling results to determine whether task switching is required. If task switching is required, it calls manipulate_task() to schedule the switch. If CFS scheduling occurs and core migration occurs (how to explain, the thread migrates to another CPU for execution), then update_cfg_runtime() is called to update the vruntime time of the task TCB after the scheduling switch.

[0356] After this embodiment supports cgroup, the scheduling entity includes multiple types: Group scheduling entity, or Task scheduling entity. A Task scheduling entity corresponds to a thread, and the Group scheduling entity is directly mapped to a scheduling group. The scheduler starts searching from the root cfs_rq. If the selected scheduling entity with the smallest vruntime is a Group scheduling entity, it will continue to find the scheduling entity with the smallest vruntime from a sub-cfs_rq corresponding to this Group scheduling entity until the scheduling entity is a Task scheduling entity, which is also the task selected for continued execution in this scheduling.

[0357] For example, Fig.17 As shown in the figure, the calculated end-to-end delays can be Fig.17 For example, the end-to-end delay of the data processing task from the camera to the camera sensor is 10ms, the end-to-end delay of the data processing task from the radar to the radar sensor is 100ms, and so on.

[0358] 1003. OEM performs scheduling and arrangement.

[0359] After simulating the thread operation through the thread operation simulator provided by this application and calculating the end-to-end delay of each task, scheduling and arranging can be performed based on the end-to-end delay of each task. For example, the initial estimated resources of each task are optimized, that is, the operation cycle, operation time probability model, DDR bandwidth or network bandwidth of each task are adjusted to obtain a scheduling result that is more suitable for the vehicle's computing resources.

[0360] It can be understood that after estimating the end-to-end delay of each task, the resources allocated to each task can be scheduled and optimized in combination with the vehicle's computing resources, thereby obtaining a scheduling result that is more suitable for the vehicle's computing resources.

[0361] For example, an end-to-end latency optimization architecture can be Fig.18 As shown in the figure, it can be specifically divided into input and output modules, evaluation modules, tuning modules and thread operation simulators. When in use, you need to input multiple types of data: one part is software and hardware information, including SoC model, memory size, network module, key delay link and other information; the other part is delay model data (richer data), which can be manually entered by the user. The required operation model data (thread operation delay, frequency, trigger relationship, etc.) can also be directly provided using an existing algorithm database. The model data migrated to the new platform can be estimated based on the performance model known in the algorithm database.

[0362] Correspondingly, combined with the above Fig.18 Given the architecture shown, the process performed by the OEM may be as follows Fig.19 shown.

[0363] First, you need to provide hardware and software information (such as SoC model, memory size, network module, key latency links, etc.) and latency model information (such as thread running time, frequency, trigger relationship, etc.) and input them into the evaluation module in HAS Studio DET.

[0364] The evaluation module estimates the size of the entire feasible solution space based on the input information, and uses the feasible solution screening module to find the next feasible solution to be verified from the feasible solution queue.

[0365] After the configuration conversion module converts the feasible solution into the simulator configuration (such as the scheduling configuration), it starts a domain-specific thread through the delay simulation manager to run the simulator and pass the configuration; the scheduling configuration of this embodiment can be the priority Priority and the core binding strategy CPU affinity of each task TCB.

[0366] The domain-specific thread running simulator is initialized first, runs for a specified time according to the input configuration, and then ends. It analyzes and calculates the running evaluation results (such as end-to-end delay) under the configuration and feeds back to the evaluation module.

[0367] After receiving feedback from the evaluation module, the tuning module adjusts the search space or strategy (such as changing task priority, binding cores, time slice size, or running weight, etc.) based on the currently searched feasible solutions, searches for the next batch of feasible solutions, and sends them to the evaluation module.

[0368] Determine whether the tuning process converges to an optimal solution or a near-optimal solution, or whether the predetermined number of searches has been exhausted, etc. If so, continue; otherwise, continue searching for the next feasible solution.

[0369] The tuning module will converge to an optimal solution or a near-optimal solution, or when the predetermined number of searches is exhausted, the search process stops, and the evaluation module converts the current optimal solution into a scheduling strategy output; the scheduling strategy of this embodiment includes the priority Priority of each task TCB and the CPU affinity strategy.

[0370] Each time the domain-specific thread is called to run the simulator, the simulator automatically performs CPU satisfiability analysis and task running timing analysis after the run ends. The CPU satisfiability analysis mainly analyzes the time that the task TCB in the Complete queue on all computing resources occupies on each computing resource during the entire simulation process, thereby calculating the utilization rate of each core of the computing resource per unit time window, such as the CPU single core occupancy rate. If the occupancy rate of a single CPU core exceeds a certain threshold (such as 99%) for a period exceeding the specified time window (such as 1s) (such as too many tasks bound to the core), then it is determined that the CPU core is not schedulable under the scheduling arrangement, and other CPU cores can be scheduled at this time.

[0371] In addition, task runtime analysis can not only provide data analysis such as the average, maximum, probability distribution, and standard deviation of end-to-end latency, but also provide a visual analysis of the simulated runtime of the task on each computing resource. Fig. 20 Shown with Fig.17 The task TCB simulation runtime visualization diagram using the autonomous driving data flow diagram as an example. The data flows with the same color have the same sequence number. The production method is to record the task TCB in the Complete queue on each computing resource core according to its start and end time of scheduling and running on a single computing resource core, print out the task TCB Task ID and SeqID and draw it according to the timeline to obtain the runtime visualization analysis diagram.

[0372] 1004. OEM sends a scheduling file to Tier-1.

[0373] The OEM can export the optimized scheduling results that meet the requirements as an AUTOSAR ARXML file and send it to Tier-1 to instruct Tier-1 to develop functions according to the scheduling results.

[0374] 1005. Tier-1 performs functional development to generate behavioral models that meet OEM planning.

[0375] TIER suppliers obtain the AUTOSAR ARXML files of the software function models of their respective software modules from the OEM, use behavioral modeling tools to develop relevant software functions, and use performance measurement tools to ensure that the developed software functions meet the behavioral model attributes planned by the OEM.

[0376] 1006. Tier-1 sends the behavioral model to the OEM.

[0377] As Tiers continue to deliver their respective software modules, OEMs continue to integrate them into the system, timely update the latest performance data of the application through SIL / HIL testing, and then iteratively update the scheduling and orchestration optimization configuration.

[0378] In addition, in order to improve the reliability of vehicles undergoing functional development, the OEM integrated system can also be verified in the vehicle hardware and software platform. For example, the behavioral model in the system can be model checked to verify the reliability of the vehicle.

[0379] In some possible scenarios, there may be hardware specified in the architecture model or the hardware has not yet been developed. At this time, it may be difficult to accurately estimate the runtime delay on the hardware through experience. Therefore, this application provides a function of migrating the algorithm database operation model, that is, the algorithm database runtime delay model measurement platform is different from the user-specified platform, which can assist users in more accurately estimating the calculation delay of functional modules in the architecture model in the design state.

[0380] For example, Fig.21 As shown, the execution process of the model migration of the algorithm database provided in this application can be as follows Fig.21 shown.

[0381] 2101. Construct an algorithm database (such as called Benchmark), and test the API runtime probability model in the algorithm database on a specific hardware platform through a hardware tracer such as Lauderbach or a software tracer module, or the first runtime probability model; and measure the runtime probability models under a large number of different autonomous driving scenarios and record them in the algorithm database.

[0382] 2102. Analyze the commonly used hot assembly instructions in the Benchmark. Of course, it can also be all the assembly instructions. Use hardware tracers such as Lauderbach or software tracer modules to measure the running time of commonly used assembly instructions on a specific hardware platform and record them in the algorithm database.

[0383] 2103. Determine whether the OEM has a hardware platform with the same configuration as the specific hardware platform. If so, execute step 2104; if not, execute step 2105.

[0384] 2104. Determine whether the OEM can fully run the algorithm database on the real hardware platform. If so, execute step 2107; if not, execute step 2109.

[0385] 2105. Determine whether the OEM's user hardware platform is the same as the Soc of the specific hardware platform. If so, execute step 2104; if not, execute step 2106.

[0386] 2106. Prompt the user to input runtime delay model information. This is equivalent to estimating the runtime probability model of each algorithm model when running in the vehicle hardware platform through the hardware platform on the OEM side, which may be different, and the user needs to provide it after measurement.

[0387] 2107. Under specified autonomous driving scenario data, the API runtime probability model of its algorithm database is measured on the user hardware platform through a hardware tracer such as Lauderbach or a software tracer module, or is called a second runtime probability model.

[0388] 2108. By comparing the running time ratio of the algorithm database API on the specific hardware and the user hardware platform in the execution scenario, the weight corresponding to each time length is calculated, and the running time of each algorithm model in the algorithm database is estimated based on the weight.

[0389] 2109. Analyze the top N assembly instructions with the highest usage rate when users call the algorithm database.

[0390] 2110. Use a hardware tracer such as Lauderbach or a software tracer module to measure the runtime of the top N assembly instructions with the highest usage rate in the user's hardware platform.

[0391] 2111. Calculate the time ratios of the first N assembly instructions recorded in the algorithm database on the specific hardware platform and the user hardware platform, and estimate the running time probability model of the algorithm database on the user hardware platform by weighted average, or called the third running time probability model.

[0392] Therefore, in the implementation manner of the present application, even if the hardware platform on which the algorithm database runs the delay model measurement is different from the user-specified hardware platform, the model migration provided by the present application can be used to estimate the running time of the algorithm model running on different hardware, thereby achieving accurate end-to-end delay calculation even if the hardware resources of the specified hardware platform are not available.

[0393] In the method provided in the present application, support for simulation tasks running on computing resources other than the CPU (such as GPU) is considered, and scheduling of various types of resources can be realized; support for complex, dynamic, and high-utilization scheduling algorithms (such as CFS) and load balancing, group scheduling and other scheduling mechanisms used on OS such as Linux on which current autonomous driving AP applications run, laying a solid foundation for satisfying users' use of different scheduling; support for computing tasks that are AI algorithm and data-driven, with strong cycle and running time volatility, which is more in line with the characteristics of autonomous driving AP references, and is more universal and general; this embodiment can better support high-fidelity simulation of end-to-end delay of key links of autonomous driving applications; and this embodiment can better support optimization of end-to-end delay of autonomous driving applications in the design state.

[0394] The present application provides a task / thread state machine, which supports the trigger relationship between complex dependency tasks (multiple receipts), the timing of complex sending (multiple sending), heterogeneous computing resources, etc. by setting a variety of state information, and can effectively support the complex application logic of AP-oriented autonomous driving application nodes, as well as the complex dependencies and trigger relationships between nodes. For example, by setting a variety of state information, it supports the trigger relationship between complex dependency tasks, the timing of complex sending, communication delay settings, etc., and even supports the operation model to make relevant corrections according to the operation state context. Through a domain-specific autonomous driving application delay simulator, including a task / thread state machine and a thread-level timing scheduling simulation simulator, it is possible to simulate the scheduling and running process of autonomous driving tasks on heterogeneous resources in a lightweight and scalable manner, and provide an analysis report on the usage of computing resources and the operation of the task chain. Therefore, through the traditional CP-side delay simulator for static operation scenarios and simple OS, it is possible to more accurately simulate the operation process of AP-side applications for AI algorithms and data-driven, rich ecosystem OS, and calculate the corresponding end-to-end delay. Through the running time probability model, computing state and context state in the task model, the period fluctuation and runtime delay fluctuation of AI algorithms and data-driven autonomous driving applications are effectively and accurately characterized. It also provides a thread-level timing scheduling simulation simulator, which supports the simulation of complex scheduling algorithms (such as CFS and RR) and scheduling mechanisms (load balancing scheduling and group scheduling) of modern operating systems with high fidelity through reasonable and effective architecture and process abstraction, such as task TCB, Group and computing resource design.

[0395] Based on the above description, the CPU core occupancy rate, GPU occupancy rate, and average latency, WCET, WCRT, jitter, etc. of the task link and each task can be simulated and calculated with high fidelity, and users do not need to worry about the specific details of the measurement; it can support fine-grained thread-level task simulation operation, support complex scheduling strategies such as cgroup group scheduling, SMP scheduling, load balancing, multi-threaded computing, and support CPU and GPU heterogeneous resource mixed scheduling, thereby greatly improving the accuracy and fidelity of the simulation, and more realistically restoring the operating status of the autonomous driving business; a white box delay simulation design can restore the CPU, GPU status and system thread running status at each moment, with high analyzability and explainability; it is flexible and scalable, can implement a variety of complex scheduling algorithms, and support user-defined algorithms.

[0396] The method provided in this application realizes a design-state latency optimization method and architecture for self-driving applications as a whole. In particular, the evaluation module, the tuning module and the domain-specific thread running simulator cooperate with each other, which can efficiently and accurately automatically optimize the most suitable scheduling and orchestration configuration for autonomous driving applications in the design state.

[0397] It can be understood that this application solves the problem of how to achieve early E2E latency optimization of autonomous driving business in the design state from the development process, and provides an optimization method that seamlessly connects from the design state to the development state and the operation state, reduces the user's use and learning costs, and improves the optimization efficiency. In the design state, the simulator provides OEM users with the resource and latency estimation and modeling capabilities of a single autonomous driving application through a domain-specific thread running simulator, as well as the end-to-end latency and resource satisfiability analysis of different scheduling and scheduling strategies. The optimal task scheduling and scheduling strategy can be automatically generated to expose the problem of insufficient resources early and reduce the end-to-end latency of the entire system. Therefore, a method is provided to users to simulate and calculate CPU occupancy, E2E latency, and optimize the scheduling and scheduling of tasks before generating code and binary in the model design and development stage; the end-to-end latency can be quickly calculated using simulation, which can greatly shorten the optimization time and improve development efficiency compared to the real vehicle real environment tuning. The application performance data delivered by the supplier can be continuously imported into the current latency model, and the optimal task scheduling and scheduling can be quickly updated and its effect verified.

[0398] The above is an introduction to the method flow provided by the present application. The following is an introduction to the device structure provided by the present application in combination with the above method flow.

[0399] See also Fig. 22 The present application provides a resource scheduling device for implementing the above Figure 6-Figure 21 The steps performed by the first development device or the functions implemented by the architecture design system, the device includes:

[0400] A screening module 2201 is used to obtain estimated resources corresponding to each task in at least one task to be implemented in the vehicle, wherein the at least one task is used to implement at least one function in the vehicle, and the at least one function is implemented by computing resources in the vehicle, and the computing resources include a plurality of resources in the vehicle for processing data, and the estimated resources corresponding to each task are resources used to run each task;

[0401] The delay management module 2202 is used to obtain the delay of running each task in the estimated resources;

[0402] Evaluation module 2203, used to obtain evaluation results according to the delay of each task;

[0403] The tuning module 2204 is used to obtain a scheduling result based on the evaluation result, the scheduling result is used to indicate a strategy for using computing resources to implement at least one task, and the scheduling result is used to generate a model for implementing at least one function.

[0404] In a possible implementation, the delay management module 2202 is specifically used to: run at least one thread corresponding to each task in the estimated resources; and obtain the delay corresponding to each task based on the running data of at least one thread of each task running in the estimated resources.

[0405] In one possible implementation, the delay management module 2202 is specifically used to: simulate the running of at least one thread corresponding to each task in the estimated resources through a thread running simulator, and obtain the delay corresponding to each task based on the running data of at least one thread of each task in the thread running simulator; or, run at least one thread corresponding to each task through a device configured with estimated resources.

[0406] In one possible implementation, if at least one thread corresponding to each task is run in the estimated resources through a thread running simulator, the delay management module 2202 is specifically used to: obtain the status data of each task in the process of running each task through the thread running simulator, the status data of each task including the data generated when each task is running in the thread running simulator; and obtain the delay corresponding to each task based on the status data of each task.

[0407] In one possible implementation, the status data of each task includes at least one of the following: receiving status data, sending status data, computing status data or context status data, the receiving status data includes information of a message queue for receiving data of the task, the sending status data includes trigger information for sending data of the task, the computing status data includes data generated by triggering the execution of the task, and the context status data includes configuration data generated when indicating that the task enters a preset state when preset context conditions are met.

[0408] In one possible implementation, the delay management module 2202 is specifically used to: if the first task is in a receiving state, update the receiving state data of the first task, where the first task is any one of at least one task; if the trigger condition of the first task is met, trigger the execution of the first task and update the computing state data corresponding to the first task; if the trigger condition of the first task is met and the context condition of the first task is met, adjust the context state data corresponding to the first task, and update the computing state data corresponding to the first task based on the adjusted context data.

[0409] In a possible implementation, the device includes: a model migration module 2205, which is used to: if the thread running simulator does not include a runtime probability model corresponding to the hardware platform indicated by the computing resources, then obtain an algorithm database, the algorithm database includes multiple algorithm models and a first runtime probability model corresponding to the preset hardware platform, the first runtime probability model includes the runtime of multiple algorithm models running in the preset hardware platform and the corresponding probability, the algorithm database is used by the thread running simulator when running at least one task; if there is a real hardware platform adapted to the preset hardware platform, and multiple algorithm models can be run in the real hardware platform, then obtain the corresponding second runtime probability model in the real hardware platform, The second runtime probability model includes the runtime of multiple algorithm models running in the real hardware platform and the corresponding probabilities, and obtains the average runtime of multiple algorithms based on the ratio of the runtime of multiple algorithm models running in the preset hardware platform and the real hardware platform; if there is a real hardware platform adapted to the preset hardware platform, and multiple algorithm models cannot be run in the real hardware platform, then the runtime corresponding to multiple instructions in the algorithm database in the preset hardware platform is obtained, the runtime of multiple instructions in the real hardware platform is obtained, and based on the ratio between the runtime corresponding to multiple instructions in the preset hardware platform and the runtime in the real hardware platform, the corresponding third runtime probability model in the real hardware platform is obtained.

[0410] In a possible implementation, the tuning module 2204 is specifically used to: determine whether the convergence condition is met according to the evaluation result; if so, use the estimated resources corresponding to each task as the scheduling result; if not, adjust the estimated resources corresponding to each task.

[0411] In a possible implementation, the evaluation module 2203 is specifically used to obtain an evaluation result according to the delay of each task and the usage rate of computing resources when simulating the running of each task in the thread running simulation.

[0412] In one possible implementation, the screening module 2201 is specifically used to: obtain user input data, where the user input data includes information about computing resources and delay model information of at least one task, where the delay model information of at least one task includes at least one of thread running delay, running frequency, triggering relationship between tasks, or number of threads; construct a feasible solution space based on the computing resources and the delay model information of each task; search for feasible solutions from the feasible solution space to obtain estimated resources corresponding to each task.

[0413] In a possible implementation, the computing resources include, but are not limited to, the following: central processing unit (CPU) resources, graphics processing unit (GPU) resources, memory resources, network card resources, hardware accelerators, or storage resources.

[0414] In a possible implementation, the device further includes: a visualization analysis module 2206, which is used to generate a timing visualization analysis result based on the scheduling result, and the timing visualization analysis result includes a list of corresponding tasks that are run on multiple types of resources in the computing resources according to the priority included in the scheduling result.

[0415] In a possible implementation, the above-mentioned device also includes: a transceiver module 2207, which is used to: send a scheduling result to a second development device; receive at least one behavior model sent by the second development device, at least one behavior model is generated by the second development device according to the scheduling result, and at least one behavior model is used to implement at least one function.

[0416] In one possible implementation, the scheduling result includes at least one of the following: an operating cycle, a running time probability model, a memory bandwidth, or a network bandwidth. The operating cycle includes the running time range of each function, the running time probability model includes the running time of each function as the probability of each running time, the memory bandwidth is the bandwidth occupied by the memory used during the running of each function, and the network bandwidth is the bandwidth of the communication network used by each function during the running.

[0417] See also Fig.23 , a schematic diagram of a structure of a model generation device provided in the present application, the model generation device can be used to perform the aforementioned Figure 6-Figure 21 The steps performed by the second development device or the functions of the function development system, the device comprising:

[0418] The transceiver module 2301 is used to obtain a scheduling result, where the scheduling result includes a strategy indicating the use of computing resources in the vehicle to implement at least one task;

[0419] A generating module 2302, configured to obtain at least one behavior model according to the scheduling result, wherein the at least one behavior model is used to implement at least one function in the vehicle;

[0420] The transceiver module 2301 is further configured to send information of at least one behavior model to the first development device.

[0421] In a possible implementation, the generation module 2302 is specifically used to: generate at least one model according to the scheduling result; if the at least one model meets the constraint condition, use the at least one model as at least one behavior model, and the constraint condition is determined according to the scheduling result.

[0422] like Fig.24 , which is a schematic diagram of the hardware structure of a resource scheduling device 240 provided in an embodiment of the present application. The computer device 240 can be used to implement the function of the first development device, such as OEM.

[0423] Fig.24 The computer device 240 shown may include: a processor 2401 , a memory 2402 , a communication interface 2403 , and a bus 2404 . The processor 2401 , the memory 2402 , and the communication interface 2403 may be connected via the bus 2404 .

[0424] The processor 2401 is the control center for generating the computer device 240, and may be a general-purpose central processing unit (CPU) or other general-purpose processors, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0425] As an example, the processor 2401 may include one or more CPUs, such as Fig.24 CPU 0 and CPU 1 are shown in .

[0426] The memory 2402 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0427] In one possible implementation, the memory 2402 may exist independently of the processor 2401. The memory 2402 may be connected to the processor 2401 via a bus 2404 and used to store data, instructions, or program codes. When the processor 2401 calls and executes the instructions or program codes stored in the memory 2402, the resource scheduling method provided in the embodiment of the present application can be implemented.

[0428] In another possible implementation, the memory 2402 may also be integrated with the processor 2401 .

[0429] The communication interface 2403 is used for connecting the computer device 240 with other devices through a communication network, and the communication network may be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 2403 may include a receiving unit for receiving data and a sending unit for sending data.

[0430] The bus 2404 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.24 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0431] It should be pointed out that Fig.24 The structure shown in the figure does not constitute a limitation on the computer device 240, except Fig.24 In addition to the components shown, computer device 240 may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0432] Optionally, the aforementioned Fig.24 The resource scheduling device shown in FIG. 1 is a chip.

[0433] like Fig.25 , which is a schematic diagram of the hardware structure of a model generation device 250 provided in an embodiment of the present application. The model generation device 250 can be used to implement the functions of a second development device, such as a Tier-1 or functional development system.

[0434] Fig.25The computer device 250 shown may include: a processor 2501 , a memory 2502 , a communication interface 2503 , and a bus 2504 . The processor 2501 , the memory 2502 , and the communication interface 2503 may be connected via a bus 2504 .

[0435] The processor 2501 is the control center of the computer device 250, and can be a general-purpose central processing unit (CPU) or other general-purpose processors, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0436] As an example, the processor 2501 may include one or more CPUs, such as Fig.25 CPU 0 and CPU 1 are shown in .

[0437] The memory 2502 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0438] In one possible implementation, the memory 2502 may exist independently of the processor 2501. The memory 2502 may be connected to the processor 2501 via a bus 2504 for storing data, instructions, or program codes. When the processor 2501 calls and executes the instructions or program codes stored in the memory 2502, the model generation method provided in the embodiment of the present application can be implemented.

[0439] In another possible implementation, the memory 2502 may also be integrated with the processor 2501 .

[0440] The communication interface 2503 is used for connecting the computer device 250 with other devices through a communication network, and the communication network may be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 2503 may include a receiving unit for receiving data and a sending unit for sending data.

[0441] The bus 2504 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.25 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0442] It should be pointed out that Fig.25 The structure shown in the figure does not constitute a limitation on the computer device 250, except Fig.25 In addition to the components shown, computer device 250 may include more or fewer components than shown, or combine certain components, or arrange components differently.

[0443] Optionally, the aforementioned Fig.25 The model generating device shown in is a chip.

[0444] The embodiment of the present application also provides a digital processing chip. The digital processing chip integrates a circuit and one or more interfaces for implementing the functions of the above-mentioned processor 2401, processor 2501, or processor 2401, processor 2501. When the digital processing chip integrates a memory, the digital processing chip can complete the method steps of any one or more of the above-mentioned embodiments. When the digital processing chip does not integrate a memory, it can be connected to an external memory through a communication interface. The digital processing chip implements the actions performed by the resource scheduling device or the model generation device in the above-mentioned embodiment according to the program code stored in the external memory.

[0445] The present application also provides a computer program product which, when executed on a computer, enables the computer to execute the above Figure 6-Figure 21 The illustrated embodiments describe steps in a method.

[0446] The resource scheduling device or model generation device provided in the embodiment of the present application may be a chip, and the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit. The processing unit may execute the computer execution instructions stored in the storage unit, and the chip may execute the above Figure 6-Figure 21Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0447] Specifically, the aforementioned processing unit or processor may be a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0448] For example, see Fig.26 , Fig.26 A schematic diagram of the structure of a chip provided in an embodiment of the present application, the chip can be expressed as a neural network processor NPU 260, NPU 260 is mounted on the host CPU (Host CPU) as a coprocessor, and the host CPU assigns tasks. The core part of the NPU is the operation circuit 2603, which is controlled by the controller 2604 to extract matrix data from the memory and perform multiplication operations.

[0449] In some implementations, the operation circuit 2603 includes multiple processing units (process engines, PEs) inside. In some implementations, the operation circuit 2603 is a two-dimensional systolic array. The operation circuit 2603 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 2603 is a general-purpose matrix processor.

[0450] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the corresponding data of matrix B from the weight memory 2602 and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 2601 and performs matrix operation with matrix B, and the partial result or final result of the matrix is ​​stored in the accumulator 2608.

[0451] The unified memory 2606 is used to store input data and output data. The weight data is directly transferred to the weight memory 2602 through the direct memory access controller (DMAC) 2605. The input data is also transferred to the unified memory 2606 through the DMAC.

[0452] The bus interface unit (BIU) 2610 is used for the interaction between the AXI bus and the DMAC and instruction fetch buffer (IFB) 2609 .

[0453] The bus interface unit 2610 (BIU) is used for the instruction fetch memory 2609 to obtain instructions from the external memory, and is also used for the storage unit access controller 2605 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0454] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 2606 or to transfer weight data to the weight memory 2602 or to transfer input data to the input memory 2601.

[0455] The vector calculation unit 2607 includes multiple operation processing units, and further processes the output of the operation circuit when necessary, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, upsampling of feature planes, etc.

[0456] In some implementations, the vector calculation unit 2607 can store the processed output vector to the unified memory 2606. For example, the vector calculation unit 2607 can apply a linear function and / or a nonlinear function to the output of the operation circuit 2603, such as linear interpolation of the feature plane extracted by the convolution layer, and then, for example, a vector of accumulated values ​​to generate an activation value. In some implementations, the vector calculation unit 2607 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 2603, for example, for use in a subsequent layer in a neural network.

[0457] An instruction fetch buffer 2609 connected to the controller 2604 is used to store instructions used by the controller 2604;

[0458] Unified memory 2606, input memory 2601, weight memory 2602 and instruction fetch memory 2609 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0459] Among them, the operations of each layer in the recurrent neural network can be performed by the operation circuit 2603 or the vector calculation unit 2607.

[0460] The processor mentioned in any of the above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the above Figure 6-Figure 21 The method of procedure.

[0461] The present application also provides a vehicle, which includes the above Fig. 22 Or the device shown in 23.

[0462] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0463] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0464] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0465] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0466] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0467] Finally, it should be noted that the above is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A resource scheduling method, characterized in that: The method comprises: Obtaining estimated resources corresponding to each of at least one task to be implemented in the vehicle, wherein the at least one task is used to implement at least one function in the vehicle, the at least one function is implemented by computing resources in the vehicle, the computing resources include a plurality of resources in the vehicle for processing data, and the estimated resources corresponding to each task are resources used to run each task; Obtaining a delay of running each of the tasks in the estimated resources; Obtaining an evaluation result according to the delay of each task; A scheduling result is obtained according to the evaluation result, wherein the scheduling result is used to indicate a strategy for using the computing resources to implement the at least one task, and the scheduling result is used to generate a model for implementing the at least one function.

2. The method according to claim 1, characterized in that: Each of the at least one task corresponds to at least one thread, and obtaining the latency of running each of the tasks in the estimated resources includes: Running at least one thread corresponding to each task in the estimated resources; The delay corresponding to each task is obtained according to the running data of at least one thread of each task running in the estimated resources.

3. The method according to claim 2, characterized in that The running of at least one thread corresponding to each task in the estimated resources includes: The at least one thread corresponding to each task is simulated to run in the estimated resources by a thread running simulator; or the at least one thread corresponding to each task is run by a device configuring the estimated resources.

4. The method according to claim 2, characterized in that: If the thread running simulator is used to simulate the running of at least one thread corresponding to each task in the estimated resources, obtaining the delay corresponding to each task according to the running data of at least one thread of each task running in the estimated resources includes: In the process of running each task through the thread running simulator, obtaining status data of each task, wherein the status data of each task includes data generated when each task runs in the thread running simulator; According to the status data of each task, the delay corresponding to each task is obtained.

5. The method according to claim 4, characterized in that The status data of each task includes at least one of the following: receiving status data, sending status data, computing status data or context status data, the receiving status data includes information of a message queue for task receiving data, the sending status data includes trigger information for task sending data, the computing status data includes data generated by triggering the execution of the task, and the context status data includes configuration data generated when indicating that the task enters a preset state when preset context conditions are met.

6. The method according to claim 5, characterized in that The obtaining of the status data of each task includes: If the first task is in a receiving state, updating the receiving state data of the first task, the first task being any one of the at least one task; If the triggering condition of the first task is met, the first task is triggered to be executed, and the computing status data corresponding to the first task is updated; If the trigger condition of the first task is met and the context condition of the first task is met, the context state data corresponding to the first task is adjusted, and the computing state data corresponding to the first task is updated according to the adjusted context data.

7. The method according to any one of claims 2 to 6, characterized in that: If the thread running simulator is used to simulate running at least one thread corresponding to each task in the estimated resources, the method includes: If the thread running simulator does not include a running time probability model corresponding to the hardware platform indicated by the computing resource, then obtaining an algorithm database, wherein the algorithm database includes a plurality of algorithm models and a first running time probability model corresponding to a preset hardware platform, wherein the first running time probability model includes the running time and corresponding probability of the plurality of algorithm models running in the preset hardware platform, and the algorithm database is used by the thread running simulator when running the at least one task; If there is a real hardware platform adapted to the preset hardware platform, and the multiple algorithm models can be run in the real hardware platform, then obtain the corresponding second runtime probability model in the real hardware platform, the second runtime probability model includes the runtime of the multiple algorithm models running in the real hardware platform and the corresponding probability, and according to the ratio of the duration of the multiple algorithm models running in the preset hardware platform and the real hardware platform, obtain the average runtime of the multiple algorithms; If there is a real hardware platform adapted to the preset hardware platform, and the multiple algorithm models cannot be run on the real hardware platform, then the running time corresponding to the multiple instructions in the algorithm database in the preset hardware platform is obtained, and the running time of the multiple instructions in the real hardware platform is obtained; according to the ratio between the running time corresponding to the multiple instructions in the preset hardware platform and the running time in the real hardware platform, the corresponding third running time probability model in the real hardware platform is obtained.

8. The method according to any one of claims 1 to 7, characterized in that The obtaining of the scheduling result according to the evaluation result includes: Determining whether a convergence condition is met according to the evaluation result; If yes, the estimated resources corresponding to each task are used as the scheduling result; If not, the estimated resources corresponding to each task are adjusted.

9. The method according to claim 8, characterized in that The obtaining of the evaluation result according to the time delay of each task includes: The evaluation result is obtained according to the delay of each task and the usage rate of the computing resources when simulating the running of each task in the thread running simulation.

10. The method according to any one of claims 1 to 9, characterized in that The obtaining of estimated resources corresponding to each task in at least one task to be implemented in the vehicle includes: Acquire user input data, wherein the user input data includes information about the computing resources and delay model information of at least one task, wherein the delay model information of at least one task includes at least one of thread running delay, running frequency, trigger relationship between tasks, or number of threads; Constructing a feasible solution space according to the computing resources and the delay model information of each task; A feasible solution is searched from the feasible solution space to obtain estimated resources corresponding to each task.

11. The method according to any one of claims 1 to 10, characterized in that The computing resources include the following: Central processing unit (CPU) resources, graphics processing unit (GPU) resources, memory resources, network card resources, hardware accelerators or storage resources.

12. The method according to any one of claims 1 to 11, characterized in that The method further comprises: A time series visualization analysis result is generated according to the scheduling result, and the time series visualization analysis result includes a list of corresponding tasks that are respectively run on multiple types of resources in the computing resources according to the priorities included in the scheduling result.

13. The method according to any one of claims 1 to 12, characterized in that The method further comprises: Sending the scheduling result to the second development device; Receive at least one behavior model sent by the second development device, where the at least one behavior model is generated by the second development device according to the scheduling result, and the at least one behavior model is used to implement the at least one function.

14. The method according to any one of claims 1 to 13, characterized in that The scheduling result includes at least one of the following: An operation cycle, a operation time probability model, a memory bandwidth or a network bandwidth, wherein the operation cycle includes the operation time range of each function, the operation time probability model includes the probability of the operation time of each function being each operation time, the memory bandwidth is the bandwidth occupied by the memory used when each function is allocated to the operation, and the network bandwidth is the bandwidth of the communication network used by each function when the function is running.

15. A model generation method, characterized in that: include: Obtaining a scheduling result, the scheduling result including a strategy indicating the use of computing resources in the vehicle to implement at least one task Acquire at least one behavior model according to the scheduling result, wherein the at least one behavior model is used to implement at least one function in the vehicle; The information of the at least one behavior model is sent to the first development device.

16. The method according to claim 15, characterized in that The acquiring at least one behavior model according to the scheduling result includes: generating at least one model according to the scheduling result; If the at least one model meets the constraint condition, the at least one model is used as the at least one behavior model, and the constraint condition is determined according to the scheduling result.

17. A resource scheduling device, characterized in that: The device comprises: a screening module, configured to obtain estimated resources corresponding to each of at least one task to be implemented in the vehicle, wherein the at least one task is used to implement at least one function in the vehicle, the at least one function is implemented by computing resources in the vehicle, the computing resources include a plurality of resources in the vehicle for processing data, and the estimated resources corresponding to each task are resources used to run each task; A delay management module, used to obtain the delay of running each task in the estimated resources; An evaluation module, used for obtaining an evaluation result according to the delay of each task; A tuning module is used to obtain a scheduling result based on the evaluation result, wherein the scheduling result is used to indicate a strategy for using the computing resources to implement the at least one task, and the scheduling result is used to generate a model for implementing the at least one function.

18. The device according to claim 17, characterized in that The delay management module is specifically used for: Running at least one thread corresponding to each task in the estimated resources; The delay corresponding to each task is obtained according to the running data of at least one thread of each task running in the estimated resources.

19. The device according to claim 18, characterized in that The delay management module is specifically used for: Simulate, by a thread running simulator, running at least one thread corresponding to each task in the estimated resources; Alternatively, at least one thread corresponding to each task is run by configuring the device for the estimated resources.

20. The device according to claim 19, characterized in that If the thread running simulator is used to simulate running at least one thread corresponding to each task in the estimated resources, the latency management module is specifically used to: In the process of running each task through the thread running simulator, obtaining status data of each task, wherein the status data of each task includes data generated when each task runs in the thread running simulator; According to the status data of each task, the delay corresponding to each task is obtained.

21. The device according to claim 20, characterized in that The status data of each task includes at least one of the following: receiving status data, sending status data, computing status data or context status data, the receiving status data includes information of a message queue for task receiving data, the sending status data includes trigger information for task sending data, the computing status data includes data generated by triggering the execution of the task, and the context status data includes configuration data generated when indicating that the task enters a preset state when preset context conditions are met.

22. The device according to claim 21, characterized in that The delay management module is specifically used for: If the first task is in a receiving state, updating the receiving state data of the first task, the first task being any one of the at least one task; If the triggering condition of the first task is met, the first task is triggered to be executed, and the computing status data corresponding to the first task is updated; If the trigger condition of the first task is met and the context condition of the first task is met, the context state data corresponding to the first task is adjusted, and the computing state data corresponding to the first task is updated according to the adjusted context data.

23. The device according to any one of claims 19 to 22, characterized in that If the thread running simulator is used to simulate running at least one thread corresponding to each task in the estimated resources, the device includes: a model migration module, which is used to: If the thread running simulator does not include a running time probability model corresponding to the hardware platform indicated by the computing resource, then obtaining an algorithm database, wherein the algorithm database includes a plurality of algorithm models and a first running time probability model corresponding to a preset hardware platform, wherein the first running time probability model includes the running time and corresponding probability of the plurality of algorithm models running in the preset hardware platform, and the algorithm database is used by the thread running simulator when running the at least one task; If there is a real hardware platform adapted to the preset hardware platform, and the multiple algorithm models can be run in the real hardware platform, then obtain the corresponding second runtime probability model in the real hardware platform, the second runtime probability model includes the runtime of the multiple algorithm models running in the real hardware platform and the corresponding probability, and according to the ratio of the duration of the multiple algorithm models running in the preset hardware platform and the real hardware platform, obtain the average runtime of the multiple algorithms; If there is a real hardware platform adapted to the preset hardware platform, and the multiple algorithm models cannot be run on the real hardware platform, then the running time corresponding to the multiple instructions in the algorithm database in the preset hardware platform is obtained, and the running time of the multiple instructions in the real hardware platform is obtained; according to the ratio between the running time corresponding to the multiple instructions in the preset hardware platform and the running time in the real hardware platform, the corresponding third running time probability model in the real hardware platform is obtained.

24. The device according to any one of claims 17 to 23, characterized in that The tuning module is specifically used for: Determining whether a convergence condition is met according to the evaluation result; If yes, the estimated resources corresponding to each task are used as the scheduling result; If not, the estimated resources corresponding to each task are adjusted.

25. The device according to claim 24, characterized in that The evaluation module is specifically used for: The evaluation result is obtained according to the delay of each task and the usage rate of the computing resources when simulating the running of each task in the thread running simulation.

26. The device according to any one of claims 17 to 25, characterized in that The screening module is specifically used for: Acquire user input data, wherein the user input data includes information about the computing resources and delay model information of at least one task, wherein the delay model information of at least one task includes at least one of thread running delay, running frequency, trigger relationship between tasks, or number of threads; Constructing a feasible solution space according to the computing resources and the delay model information of each task; A feasible solution is searched from the feasible solution space to obtain estimated resources corresponding to each task.

27. The device according to any one of claims 17 to 26, characterized in that The computing resources include the following: Central processing unit (CPU) resources, graphics processing unit (GPU) resources, memory resources, network card resources, hardware accelerators or storage resources.

28. The device according to any one of claims 17 to 27, characterized in that The device also includes: A visualization analysis module is used to generate a timing visualization analysis result according to the scheduling result, wherein the timing visualization analysis result includes a list of corresponding tasks that are respectively run on multiple types of resources in the computing resources according to the priorities included in the scheduling result.

29. The device according to any one of claims 17 to 28, characterized in that The device further comprises a transceiver module, configured to: Sending the scheduling result to the second development device; Receive at least one behavior model sent by the second development device, where the at least one behavior model is generated by the second development device according to the scheduling result, and the at least one behavior model is used to implement the at least one function.

30. The device according to any one of claims 17 to 29, characterized in that The scheduling result includes at least one of the following: An operation cycle, a operation time probability model, a memory bandwidth or a network bandwidth, wherein the operation cycle includes the operation time range of each function, the operation time probability model includes the probability of the operation time of each function being each operation time, the memory bandwidth is the bandwidth occupied by the memory used when each function is allocated to the operation, and the network bandwidth is the bandwidth of the communication network used by each function when the function is running.

31. A model generation device, characterized in that: include: A transceiver module, used to obtain a scheduling result, wherein the scheduling result includes a strategy for instructing to use computing resources in the vehicle to implement at least one task a generating module, configured to obtain at least one behavior model according to the scheduling result, wherein the at least one behavior model is used to implement at least one function in the vehicle; The transceiver module is further used to send information of the at least one behavior model to the first development device.

32. The device according to claim 31, characterized in that The generating module is specifically used for: generating at least one model according to the scheduling result; If the at least one model meets the constraint condition, the at least one model is used as the at least one behavior model, and the constraint condition is determined according to the scheduling result.

33. A vehicle function configuration system, characterized in that: include: A first development device and a second development device, wherein the first development device is used to execute the method according to any one of claims 1-14, and the second development device is used to execute the method according to any one of claims 15-16.

34. A resource scheduling device, characterized in that: The method comprises a processor, wherein the processor is coupled to a memory, wherein the memory stores a program, and when the program instructions stored in the memory are executed by the processor, the method according to any one of claims 1 to 14 is implemented.

35. A model generation device, characterized in that: The invention comprises a processor, the processor is coupled to a memory, the memory stores a program, and when the program instructions stored in the memory are executed by the processor, the method described in claims 15-16 is implemented.

36. A computer-readable storage medium comprising a program, which, when executed by a processing unit, performs the method according to any one of claims 1 to 14 or 15 to 16.

37. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 14 or 15 to 16 is implemented.

38. A vehicle, characterized in that: Comprising an apparatus as claimed in claim 34 or 35.

Images