Vehicle control device and vehicle control method

Abstract

The present invention involves: a memory for storing a main application that executes vehicle control processing as tasks and a shadow application that executes vehicle control processing for verification as a task; a calculation time estimation unit for estimating, from input data of the shadow application, the processing time required for the shadow application to process the task using an accelerator; a permitted execution time determination unit for determining a permitted execution time, which is a time during which the task of the shadow application can use the accelerator; and an accelerator arbitration unit for granting accelerator usage permission to the shadow application when the estimated processing time is shorter than the permitted execution time.

Description

Vehicle control device and vehicle control method

[0001] The present invention relates to a vehicle control device and a vehicle control method.

[0002] Vehicle control systems installed in automobiles and other vehicles control the vehicle through the execution of their installed programs. These programs are updated periodically to add new control functions, correct bugs, and so on. Before updating a vehicle control system program, it is necessary to verify that the updated program (new program) will allow the vehicle control system to properly control the vehicle without malfunctions.

[0003] To verify the operation of a new program, it is preferable to implement the verification program in the vehicle control unit and test whether it operates without problems in a realistic driving environment. To test the verification program in a realistic driving environment, for example, the verification program can be added to a vehicle control unit that already has an existing program installed. Then, test processing using the verification program to control the vehicle can be performed in parallel with the processing using the existing program to control the vehicle. This allows verification of whether the verification program can correctly control the vehicle when driving on actual roads, etc.

[0004] Patent Document 1 describes a technique in which, when an electronic control device performs multiple arithmetic processes, it extracts characteristic features for each execution task, predicts the task processing time corresponding to the characteristic features, and allocates available processing space for each task according to the predicted task processing time. In the technique described in Patent Document 1, the prediction of the task processing time corresponding to the characteristic features is performed by estimating the execution time from statistical values ​​obtained from logs, etc.

[0005] Japanese Patent Publication No. 2019-125093

[0006] When running a verification program in a vehicle control device that already has an existing program implemented, the verification program will be executed during the idle time of the existing program's calculation processing. In this case, it is necessary to predict the processing time of the task that executes each program, for example by applying the technology described in Patent Document 1, and to secure a period for executing the verification program based on the predicted task processing time.

[0007] However, since the verification program is an unknown program, there is a high possibility that the task processing time cannot be accurately predicted. For example, the process of estimating execution time from statistical values ​​obtained from logs as described in Patent Document 1 cannot accurately predict the task processing time of an unknown program. If it is not possible to secure a period for running the verification program, there is a possibility that vehicle control using the existing program may not be performed properly, which is undesirable.

[0008] The present invention has been made in view of the above circumstances, and aims to provide a vehicle control device and a vehicle control method that can appropriately implement and operate a verification program in a vehicle control device without affecting the execution of an existing program.

[0009] To solve the above problems, for example, the configuration described in the claims is adopted. The present invention includes multiple means for solving the above problems, but to give one example, the vehicle control device includes a memory that stores a main application that executes vehicle control processing as an accelerator task and a shadow application that executes verification vehicle control processing as an accelerator task, a calculation time estimation unit that estimates the estimated processing time when the accelerator processes the task from the input data of the shadow application, an execution permission time determination unit that determines the execution permission time which is the time during which the shadow application task can use the accelerator, and an accelerator arbitration unit that grants permission to use the accelerator to the shadow application when the estimated processing time is shorter than the execution permission time, and the shadow application executes a task that uses the accelerator when it has permission to use the accelerator from the accelerator arbitration unit.

[0010] According to the present invention, by estimating the execution time based on the input data handled when a task is executed, accelerator tasks can be appropriately scheduled so as not to affect the processing of the main application. Other issues, configurations, and effects will be clarified by the following description of embodiments.

[0011] This is a block diagram showing an example of the overall configuration of a vehicle control device according to a first embodiment of the present invention. This is a block diagram showing an example of the hardware configuration of a vehicle control device according to a first embodiment of the present invention and an example of a processing unit configured in memory. This is a flowchart showing an example of the execution of a main application by a vehicle control device according to a first embodiment of the present invention. This is a flowchart showing an example of the execution of a shadow application by a vehicle control device according to a first embodiment of the present invention. This is a flowchart showing an example of the processing of the accelerator arbitration unit of a vehicle control device according to a first embodiment of the present invention. This is a flowchart showing an example of the processing of the execution permission time determination unit of a vehicle control device according to a first embodiment of the present invention (receiving a notification that the main application has started using the accelerator). This is a flowchart showing an example of the processing of the execution permission time determination unit of a vehicle control device according to a first embodiment of the present invention (receiving a notification that the main application has finished using the accelerator). This is a flowchart showing an example of the processing of the execution permission time determination unit of a vehicle control device according to a first embodiment of the present invention (receiving a notification that the shadow application has started using the accelerator). This is a flowchart showing an example of the processing of the execution permission time determination unit of a vehicle control device according to a first embodiment of the present invention (receiving a notification that the shadow application has finished using the accelerator). This is a flowchart showing an example of the processing of the accelerator arbitration unit of a vehicle control device according to a second embodiment of the present invention. This is a flowchart showing an example of the processing of the accelerator arbitration unit of a vehicle control device according to a third embodiment of the present invention. This is a time chart showing an example of processing division performed by a vehicle control device according to a third embodiment of the present invention. This is a flowchart showing an example of a shadow application executed by a vehicle control device according to a fourth embodiment of the present invention. This is a block diagram showing an example of the configuration of a vehicle control device according to a fourth embodiment of the present invention. This is a time chart showing an example of processing division performed by a vehicle control device according to a fourth embodiment of the present invention.

[0012] <First Embodiment Example> Hereinafter, a vehicle control device and a vehicle control method according to the first embodiment example of the present invention will be described with reference to Figures 1 to 9.

[0013] [Example of Vehicle Control Device Configuration] Figure 1 shows the overall configuration of a vehicle control device 100 according to a first embodiment of the present invention. The vehicle control device 100 in this embodiment is an electronic device called an ECU (Electronic Control Unit). One or more vehicle control devices 100 are mounted on a vehicle such as an automobile and control various parts of the vehicle. For example, the vehicle control device 100 controls various devices mounted on the vehicle, such as the engine, brakes, steering, and lights, for driving assistance such as autonomous driving and collision avoidance. The vehicle control device 100 is composed of a semiconductor called a SoC (System on a chip), for example. The configuration of the vehicle control device 100 shown in Figure 1 is a configuration viewed from the perspective of the functions performed by the vehicle control device 100.

[0014] The vehicle control device 100 shown in Figure 1 comprises an input data holding unit 101, a main application holding unit 102, a shadow application holding unit 103, a calculation time estimation unit 104, and an execution permission time determination unit 105. The vehicle control device 100 also includes an accelerator 130 that executes processing using application programs held by the main application holding unit 102 and the shadow application holding unit 103. In the drawing, the accelerator 130 is labeled "ACC".

[0015] The input data holding unit 101 holds input data for the vehicle control device 100. The input data includes various types of data, such as detection data from various sensors installed on the vehicle, image data from cameras, and operation data from the vehicle driver. Here, the image data includes not only the image data itself captured by the camera, but also data obtained by analyzing the image data, such as the number of objects contained in the image data.

[0016] The main application holder 102 holds the main application. The main application is an application program for controlling the vehicle. The vehicle control device 100 is controlled by executing this main application. The shadow application holder 103 holds the shadow application. The shadow application is also an application program for controlling the vehicle, similar to the main application, but it is an application for checking the execution status of the main application in the vehicle control device 100.

[0017] This shadow application is, for example, an updated version of the main application. During the execution of the shadow application, no control is performed on any part of the vehicle; instead, it performs tasks such as analyzing the execution status of the main application. The shadow application runs during idle time while the main application is running.

[0018] The calculation time estimation unit 104 estimates the processing time required when the accelerator 130 processes a task based on the input data of the shadow application. The input data of the shadow application is the input data held in the input data holding unit 101.

[0019] The execution permission time determination unit 105 determines the time during which the shadow application task can use the accelerator 130, i.e., the execution permission time (execution permission time determination process). Here, the execution permission time determination unit 105 determines the time (deadline) during which the shadow application can use the accelerator 130 based on the execution status of the main application held in the main application holding unit 102.

[0020] The accelerator arbitration unit 106 grants permission to use the accelerator to the shadow application if the estimated processing time obtained by the calculation time estimation unit 104 is within the time slot of the execution permission time (interval) obtained by the execution permission time determination unit 105, and the main application is not currently running.

[0021] Figure 2 shows an example of the hardware configuration of the vehicle control device 100 and an example of the processing unit configured in the memory 110. The vehicle control device 100 includes a memory 110, a CPU (Central Processing Unit) 120, and an accelerator 130. The CPU 120 can configure various processing units in the memory 110 by executing programs stored in the memory 110.

[0022] The accelerator 130 executes various control processes based on the control of the CPU 120. The vehicle control device 100 receives data from various sensors 140 installed on the vehicle. The various sensors 140 include, for example, cameras that photograph the area around the vehicle, and the image data captured by the cameras is supplied to the vehicle control device 100.

[0023] Furthermore, the vehicle control unit 100, which is an ECU, is capable of exchanging data with another ECU 150 installed in the vehicle. For example, it can transmit the processing results from the accelerator 130 to the other ECU 150. The ECU 150 can then perform various processes based on instructions from the vehicle control unit 100.

[0024] The memory 110 comprises various processing units, including an input data holding unit 101, a main application holding unit 102, a shadow application holding unit 103, a calculation time estimation unit 104, an execution permission time determination unit 105, and an accelerator arbitration unit 106. These processing units include all the processing units described in Figure 1.

[0025] Furthermore, the memory 110 includes an output data holding unit 111, an accelerator input data holding unit 112, an accelerator output data holding unit 113, a deadline holding unit 114, a main application execution counter 115, and a shadow application execution counter 116. In the following description, the data holding units for input data, output data, applications, etc., will be omitted and simply referred to as input data 101, main application 102, shadow application 103, output data 111, deadline 114, etc.

[0026] The following describes each data and processing unit shown in Figure 2. Input data 101 is input data to the main application 102 or shadow application 103. This input data 101 includes, for example, input data from various sensors 140. Output data 111 is output data obtained by executing the main application 102 or shadow application 103. Output data 111 is output to, for example, another ECU 150.

[0027] The main application 102 is a program that executes vehicle control processing, and performs processing on the CPU 120 and processing on the accelerator 130. The shadow application 103 is a program that executes verification processing, and performs processing on the CPU 120 and processing on the accelerator 130.

[0028] Accelerator input data 112 is data transferred from the main application 102 or shadow application 103 to the accelerator 130. That is, accelerator input data 112 is transferred from the memory 110 of the CPU 120 to the memory (not shown) of the accelerator 130. Accelerator output data 113 is data transferred from the accelerator 130 to the main application 102 or shadow application 103. That is, accelerator output data 113 is transferred from the memory of the accelerator 130 to the memory 110 of the CPU 120.

[0029] As explained in Figure 1, the calculation time estimation unit 104 is a processing unit that calculates the time for which the shadow application 103 can use the accelerator 130. The execution permission time determination unit 105 is a processing unit that calculates the deadline, which is the time for which the shadow application 103 can use the accelerator 130. The calculation of the deadline in the execution permission time determination unit 105 is performed under the control of the CPU 120.

[0030] The accelerator mediation unit 106 is a processing unit that receives an accelerator usage request from the shadow application 103 and determines whether it can be used. Note that the calculation time estimation unit 104, the execution permission time determination unit 105, and the accelerator mediation unit 106 are each configured by a program, and these processes are executed under the control of the CPU 120.

[0031] The main application execution counter 115 is a counter that indicates whether the main application 102 is being executed. For example, if the count value is 1 or more, it indicates that the main application 102 is being executed, and if the count value is 0, it indicates that the main application 102 has not been executed.

[0032] The shadow application execution counter 116 is a counter that indicates whether the shadow application 103 is being executed. For example, if the count value is 1 or more, it indicates that the shadow application 103 is being executed, and if the count value is 0, it indicates that the shadow application 103 has not been executed. The deadline 114 indicates information on the deadline when the shadow application 103 can use the accelerator 130.

[0033] [Execution example of main application] Fig. 3 is a flowchart showing an execution example of the main application 102 by the vehicle control device 100. In the flowchart shown in Fig. 3, the left column is the process by the CPU 120, and the right column is the process by the accelerator 130. That is, the example in Fig. 3 shows one cycle of the process of the main application 102.

[0034] First, when the process of the main application 102 starts, the preprocessing starts in the CPU 120 (step S11). As the preprocessing, the CPU 120 receives the input data 101 and formats the input data 101. That is, in step S11, a process of converting it into accelerator input data 112 in a format that can be supplied for the control operation of the accelerator 130 is performed.

[0035] Next, the CPU 120 notifies the start (step S12). Here, the CPU 120 notifies the execution permission time determination unit 105 of the start of using the accelerator 130. After that, the CPU 120 transfers the accelerator input data 112 to the accelerator 130 (step S13). The accelerator 130 receives the accelerator input data 112 transferred in step S13 (step S21), and executes the processing of the main application 102 based on the received data (step S22). The processing in this step S22 is the main processing of the control calculation in the accelerator 130.

[0036] After the execution of the processing in step S22, the accelerator 130 transfers the accelerator output data 113 to the CPU 120 (step S23). The CPU 120 receives the accelerator output data 113 transferred in step S23 (step S14), and notifies the execution permission time determination unit 105 of the end of using the accelerator 130 (step S15). Then, the CPU 120 performs post-processing to aggregate the accelerator output data 113 to obtain the output data 111, outputs the output data 111 to the outside (step S16), and ends the processing.

[0037] [Execution Example of Shadow Application] FIG. 4 is a flowchart showing an execution example of the shadow application 103 by the vehicle control device 100. In the flowchart shown in FIG. 4, the left column is the processing by the CPU 120, and the right column is the processing by the accelerator 130. That is, the example in FIG. 4 shows one cycle of processing of the shadow application 103.

[0038] First, when the processing of the shadow application 103 starts, the CPU 120 receives the input data 101 and formats the input data 101. That is, the CPU 120 performs pre-processing to convert it into the accelerator input data 112 in a format that can be supplied to the accelerator 130 for control calculation (step S31).

[0039] Next, the computation time estimation unit 104 performs computation time estimation processing to calculate the accelerator processing time from the accelerator input data 112 (step S32). For example, if the processing is image recognition processing, the computation time estimation unit 104 performs computation time estimation processing from the image size (resolution and memory capacity) and the number of objects. If the processing is behavior prediction processing, the computation time estimation unit 104 performs computation time estimation processing from the time series data length and the number of objects of the accelerator input data 112. Furthermore, in the case of a language model, the computation time estimation unit 104 performs computation time estimation processing from the number of tokens, etc.

[0040] Specifically, the computation time estimation unit 104 may pre-determine the computation time per unit size (1 token, 1 object, 1 byte) and estimate the output time using linear approximation. Alternatively, the computation time estimation unit 104 may use a machine learning model for estimation. Furthermore, this computation time estimation process may refer to input data from a different sensor or another application than the current input data. Alternatively, the output data from the previous period may be referred to during the computation time estimation process. In addition, the input data such as the number of objects used by the computation time estimation unit 104 when performing computation time estimation based on the number of objects may be obtained from the execution results of the main application 102. Furthermore, the computation time estimation unit 104 may obtain the number of objects from the previous frame (the frame of image data from which the previous object detection process was performed) of the processing result of the shadow application 103 task, and estimate the processing time from the obtained number of objects. Furthermore, when the input data is image data and a language model, the computation time estimation unit 104 may estimate the number of tokens from the input data and use the estimated number of tokens to estimate the processing time. In this case, the processing time may be estimated by using both the number of tokens and the number of objects. Furthermore, the computation time estimation unit 104 may estimate the output token count request described in the input data and use the token count indicated by the output token count request to estimate the processing time. In this case as well, the processing time may be estimated by using both the number of tokens and the number of objects.

[0041] Next, the computation time estimation unit 104 transmits an accelerator usage request to the accelerator arbitration unit 106 (step S33). Here, the computation time estimation unit 104 transmits a usage permission request and an estimated usage time. The estimated usage time is the time estimated in step S32. Upon receiving the accelerator usage request, the accelerator arbitration unit 106 determines whether to grant accelerator usage permission, put the accelerator on standby, or deny accelerator usage (step S34).

[0042] If the system waits for accelerator use in step S34 (Wait in step S34), the process returns to sending the accelerator use request by the calculation time estimation unit 104 in step S33. If the system rejects accelerator use in step S34 (No in step S34), the processing of the shadow application 103 for this cycle is terminated.

[0043] Then, if accelerator use is permitted in step S34 (Yes in step S34), the accelerator arbitration unit 106 notifies the execution permission time determination unit 105 to start using the accelerator 130 (step S35). When the CPU 120 receives the notification of start of use in step S35, it transfers accelerator input data 112 to the accelerator 130 (step S36). The accelerator input data 112 uses data prepared in the processing of the main application 103. That is, the accelerator 130 receives the accelerator input data 112 transferred in step S36 (step S41) and executes the processing of the shadow application 103 using the received data (step S42).

[0044] After the processing in step S42 is completed, the accelerator 130 transfers the accelerator output data 113 to the CPU 120 (step S43). The CPU 120 receives the accelerator output data 113 transferred in step S43 (step S37) and notifies the execution permission time determination unit 105 that the use of the accelerator 130 has ended (step S38). Then, the CPU 120 performs post-processing to consolidate the accelerator output data 113 into output data 111, outputs the output data 111 to the outside (step S39), and terminates the process.

[0045] [Example of processing by the accelerator arbitration unit] Figure 5 is a flowchart showing an example of processing by the accelerator arbitration unit 106. The accelerator arbitration unit 106 performs an accelerator usage request reception waiting process, waiting for an accelerator usage request from the shadow application 103 (step S51). In this accelerator usage request reception waiting process, when the accelerator arbitration unit 106 receives an accelerator usage request, it proceeds to the permission time determination and determines whether the estimated usage time obtained by the calculation time estimation unit 104 is less than the remaining time until the deadline (step S52).

[0046] In step S52, if the estimated usage time is greater than the remaining time until the deadline (No in step S52), the accelerator arbitration unit 106 rejects the accelerator usage request and notifies the shadow application 103 of this (step S55). After this notification of accelerator usage rejection, the accelerator arbitration unit 106 returns to the accelerator usage request reception waiting process in step S51.

[0047] Furthermore, in step S52, if the estimated usage time is less than the remaining time until the deadline (Yes in step S52), the accelerator arbitration unit 106 determines whether or not the main application is running (step S53). Here, the accelerator arbitration unit 106 refers to the main application execution counter 115 and the shadow application execution counter 116, and determines that the application is not running if both counters are 0.

[0048] In step S53, if the accelerator processing of the main application 102 is in progress, that is, if any counter value is 1 or greater (Yes in step S53), the accelerator arbitration unit 106 performs accelerator usage waiting processing (step S54). In this case, the accelerator arbitration unit 106 notifies the shadow application 103 that it is possible to retry because the main application 102 is in progress but there is still time before the deadline.

[0049] In other words, the accelerator arbitration unit 106, after a certain period of time has elapsed, determines that the accelerator processing of the main application 102 may have finished and the accelerator processing of the shadow application 103 may be able to be executed, and therefore puts the accelerator to wait. After the accelerator usage waiting process, the accelerator arbitration unit 106 returns to the accelerator usage request reception waiting process of step S51.

[0050] Furthermore, in step S53, if the accelerator processing of the main application 102 is not currently running, that is, if all counter values ​​are 0 (No in step S53), the accelerator arbitration unit 106 notifies permission to use the accelerator (step S56). After processing this accelerator usage permission, the accelerator arbitration unit 106 returns to the accelerator usage request reception waiting process in step S51.

[0051] [Processing of the Execution Permission Time Determination Unit] Figures 6, 7, 8, and 9 are flowcharts showing the processing of the Execution Permission Time Determination Unit 105. The four processes shown in Figures 6, 7, 8, and 9 are processes that are executed in parallel by the Execution Permission Time Determination Unit 105. First, we will explain the processing shown in Figure 6 when the main application 102 receives a notification to start using the accelerator.

[0052] The execution permission time determination unit 105 receives a notification from the main application 102 to start using the accelerator (step S61). At this time, the execution permission time determination unit 105 increments the main application execution counter 115 by one (step S62). Then, the execution permission time determination unit 105 calculates the deadline and saves it to the memory 110 (step S63), and after saving, waits until it receives the accelerator usage start notification from step S61.

[0053] The deadline in step S62 is calculated as follows, for example: [Deadline] = [Start time of main application execution] + [Main application execution cycle] - [Margin] The margin is the overhead required for the accelerator 130 to interrupt the current process and start the next process. Since the overhead fluctuates depending on the memory usage of the accelerator 130, it may be dynamically estimated taking this into account.

[0054] If there are multiple instances of Main Application 102, calculate as follows and use the smaller of the two deadlines as the final deadline: [Deadline 1] = [Start time of execution of Main Application 1] + [Execution cycle of Main Application 1] - [Margin 1] [Deadline 2] = [Start time of execution of Main Application 2] + [Execution cycle of Main Application 2] - [Margin 2] [Deadline] = min([Deadline 1], [Deadline 2])

[0055] The execution start time of the main application 102 may be received from the main application 102, or it may be obtained from the main application 102's startup handler or the OS.

[0056] Next, referring to Figure 7, the processing when the main application 102 receives an accelerator usage completion notification will be described. The execution permission time determination unit 105 receives the accelerator usage completion notification for the main application 102 (step S64). At this time, the execution permission time determination unit 105 counts down the main application execution counter 115 and decrements the count value of the main application execution counter 115 by one (step S65). After the countdown, the execution permission time determination unit 105 waits until an accelerator usage completion notification is received.

[0057] Next, referring to Figure 8, the processing when a notification for starting accelerator use of the shadow application 103 is received will be described. The execution permission time determination unit 105 receives a notification for starting accelerator use of the shadow application 103 (step S66). At this time, the execution permission time determination unit 105 increments the shadow application execution counter 116 and adds one to the counter value (step S67). After this increment, it waits until the notification for starting accelerator use in step S66 is received.

[0058] Next, referring to Figure 9, the processing when a notification of completion of accelerator usage for the shadow application 103 is received will be described. The execution permission time determination unit 105 receives the notification of completion of accelerator usage for the shadow application 103 (step S68). At this time, the execution permission time determination unit 105 counts down the shadow application execution counter 116 and subtracts one from the count value of the shadow application execution counter 116 (step S69). After the countdown, the execution permission time determination unit 105 waits until a notification of completion of accelerator usage is received.

[0059] [Effects of the First Embodiment] As described above, according to this embodiment, by estimating the execution time based on the input data, which is the data handled when the task is executed, it becomes possible to schedule accelerator tasks in a way that does not affect the processing of the main application.

[0060] To explain this effect in more detail, the processing time can vary significantly depending on the nature of the input data. For example, in behavior prediction AI that uses artificial intelligence (AI) to predict the behavior of detected objects based on detection data from cameras or sensors, or in tracking AI that tracks detected objects, prediction and matching processes are performed for each object. Therefore, the processing time varies depending on the number of objects in the input data.

[0061] In this embodiment, the execution time is estimated each time based on input data such as the number of objects and the number of tokens. If the estimated processing time falls within the execution permission time of the shadow application, the task is executed. In this case, data such as the number of objects can be obtained as the execution result of the main application, making it easy to obtain data for estimating the execution time. Alternatively, the number of objects can be obtained from the previous frame of the processing result of the shadow application task, also making it easy to obtain data for estimating the execution time. Furthermore, if the execution time does not fall within the permission time, the process is skipped. As a result, processes that do not fall within the permission time of the shadow application are not executed, so the main application can execute without being delayed by an extension of the shadow application's processing time. Therefore, the shadow application can be executed appropriately without causing delays in vehicle control by the main application.

[0062] Furthermore, in this embodiment, the middleware schedules the shadow application's accelerator processing so that it does not occur during the time when the main application uses the accelerator. As a result, the main application's accelerator processing is not executed simultaneously with the shadow application's accelerator processing, eliminating processing delays. In other words, in this embodiment, it becomes possible to separate the impact of the main application from that of the shadow application. In addition, in this embodiment, the overhead associated with the accelerator's task switching process can also be taken into consideration, which has the effect of strictly controlling the impact on the main application compared to scheduling methods used in CPUs and the like.

[0063] <Second Embodiment Example> Next, a vehicle control device and vehicle control method according to a second embodiment example of the present invention will be described with reference to Figure 10. In this embodiment example, the configuration and processing of the vehicle control device are the same as those described in the first embodiment example. The only difference between this embodiment example and the first embodiment example is the processing of the accelerator arbitration unit 106 described in Figure 5. In Figure 10, which describes the processing of this embodiment example, the same step numbers are assigned to the same processing as in Figure 5, and redundant explanations are omitted.

[0064] [Example of processing in the accelerator arbitration unit] As explained in Figure 5, if the accelerator processing of the main application 102 is not currently running, the accelerator arbitration unit 106 notifies permission for the shadow application to use the accelerator in step S56. In this embodiment, when the accelerator arbitration unit 106 notifies permission for the shadow application to use the accelerator, it starts a thread to monitor the execution of the shadow application (step S71).

[0065] When the monitoring thread is started in step S71, the accelerator arbitration unit 106 waits until the deadline calculated by the computation time estimation unit 104 is reached (step S72). When the deadline is reached, the accelerator arbitration unit 106 determines whether or not the shadow application 103 is running (step S73). If the shadow application 103 is running in step S73 (Yes in step S73), the accelerator arbitration unit 106 forcibly terminates the shadow application 103 (step S73). After the forced termination, the accelerator arbitration unit 106 terminates the monitoring thread started in step S71 (step S74).

[0066] Furthermore, in step S73, if the shadow application 103 is not running (No. in step S73), the accelerator arbitration unit 106 terminates the monitoring thread started in step S71 (step S74). Note that in step S73, in the event of a forced termination, the shadow application 103 may be terminated immediately, or the process may be temporarily interrupted at the time of the forced termination and the interrupted process may be resumed at a later time. Other configurations and processes in this embodiment are the same as those described in the first embodiment.

[0067] [Effects of the Second Embodiment] According to this embodiment, even if the processing of the shadow application is delayed for any reason, the delay in starting the main application can be suppressed by interrupting it. Therefore, the shadow application can be executed more appropriately without causing a delay in vehicle control by the main application.

[0068] <Third Embodiment Example> Next, a vehicle control device and a vehicle control method according to the third embodiment example of the present invention will be described with reference to FIGS. 11 to 12. Also in this embodiment example, the configuration and processing of the vehicle control device apply the configuration and processing described in the first embodiment example, and the processing of the accelerator mediation unit 106 is different from that in the first embodiment example and the second embodiment example. In FIG. 11, which is a flowchart for explaining the processing of this embodiment example, the same processing as in FIGS. 5 and 10 is given the same step numbers, and duplicate explanations are omitted.

[0069] [Example of Processing in Accelerator Mediation Unit] In this embodiment example, as shown in FIG. 11, at step S51, the accelerator mediation unit 106 performs a permission time determination process after waiting to receive an accelerator usage request from the shadow application 103 (step S52'). In this step S52', the accelerator mediation unit 106 performs the following processing as the permission time determination process.

[0070] That is, when the estimated usage time is less than the time obtained by subtracting the average accelerator execution time of the main application from the remaining time until the deadline (execution permission time I ｓ ) (Yes in step S52'), it proceeds to step S53. Here, using the average accelerator execution time for the determination is just an example, and instead of the average value, the median value or the maximum value may be used. Also, when the estimated usage time is greater than the execution permission time I ｓ (No in step S52'), it proceeds to the accelerator usage rejection in step S55.

[0071] Here, in this embodiment example, the execution permission time I ｓ is obtained by the calculation of the following [Equation 1].

[0072] [Equation 1] I ｓ = t ｄ - [T ｓ / T ｍ ] · (t ｉ，ｗ - t ｏ，ｗ )

[0073] In [Equation 1], tｄ The deadline is T ｓ T is the execution cycle of the shadow application. ｍ is the execution cycle of the main application, t ｉ，ｗ This is the worst-case execution time (maximum execution time) for the accelerator processing of the main application, t ｏ，ｗ This is the worst-case overhead. Also, [T ｓ / T ｍ ] represents a carry-over in the Gaussian ceiling function.

[0074] Deadline ｄ This is a time set arbitrarily by the user, and is calculated by adding the set time, which is determined from the execution frequency of the shadow application, to the start time of the main application. For example, if you want to process the output of shadow application 103 at a frequency of once every three times, it will be the start time of the main application three cycles after the start time of the main application in the cycle in which the request was received. Steps S53, S54, S55, and S56 are the same as the process explained in Figure 5.

[0075] In this embodiment, as in the second embodiment, when permission to use the accelerator for the shadow application is notified in step S56, the accelerator arbitration unit 106 starts a thread to monitor the execution of the shadow application 103 (step S81). The contents of this monitoring thread also differ in this embodiment from those of the second embodiment.

[0076] The following describes the process after the monitoring thread is started in step S81, based on Figure 11. First, the accelerator arbitration unit 106 performs a wait process for the start of the next cycle of the main application 102 (step S82). Here, the accelerator arbitration unit 106 waits until the time obtained by subtracting a margin from the start time of the next cycle of the main application 102.

[0077] When the start time of the next cycle of the main application 102 approaches, the accelerator arbitration unit 106 determines whether or not the shadow application 103 is running (step S83). If the shadow application 103 is not running in step S83 (No. in step S83), the accelerator arbitration unit 106 terminates the monitoring thread (step S84).

[0078] If step S83 determines that the shadow application 103 is running (Yes in step S83), the accelerator arbitration unit 106 issues a command to temporarily suspend the accelerator processing of the shadow application 103 and saves the context (step S85). After that, the accelerator arbitration unit 106 performs a restart waiting process for the shadow application 103 (step S86). Here, the accelerator arbitration unit 106 waits until the main application execution counter 115 (Figure 2) becomes 0.

[0079] In step S86, the accelerator arbitration unit 106 waits until the main application execution counter 115 becomes 0, and then proceeds to the re-execution process of the shadow application 103 (step S87). Here, the accelerator arbitration unit 106 reads the accelerator processing context of the shadow application 103 and issues a command to resume processing. Then, the accelerator arbitration unit 106 proceeds to the waiting process for the start of the next cycle of the main application 102 in step S82.

[0080] Figure 12 shows an example of a time chart for the processing of the main application 102 and the shadow application 103, as explained in Figure 11. The upper part of Figure 12 shows the theoretical value calculated by the accelerator arbitration unit 106, and the lower part of Figure 12 shows an example of the actual execution state. ｓ " is the theoretical execution permission time obtained by the calculation in equation [Equation 1]. Figure 12 shows the deadline t ｄ , execution period T of shadow application 103 ｓ , execution cycle T of the main application 102 ｍThe worst execution time t for accelerator processing of the main application 102. ｉ，ｗ , worst-case overhead value t ｏ，ｗ This indicates.

[0081] The main task shown in Figure 12 is the execution task for the main application 102, and the shadow processing is the execution task for the shadow application 103. In the example in Figure 12, the task for the shadow application 103 is executed in three separate steps. As shown in Figure 12, the execution timing of each divided task is controlled by determining the worst-case execution time based on theoretical values, so that the task for the shadow application 103 can be executed without affecting the execution of the task for the main application 102 in each cycle.

[0082] [Effects of the Third Embodiment] According to this embodiment, even for high-load shadow applications 103 that cannot be completed within the execution permission time for a single task, the task processing can be divided and re-executed. Furthermore, according to this embodiment, compared to scheduling using the accelerator's scheduling function, scheduling that takes into account the overhead of context switching is possible, and the overhead impact on the main application 102 can be minimized.

[0083] <Fourth Embodiment Example> Next, a vehicle control device and vehicle control method according to a fourth embodiment example of the present invention will be described with reference to Figures 13 to 15. In this embodiment example as well, the configuration and processing of the vehicle control device are the same as those described in the first embodiment example. The only difference between this embodiment example and the first to third embodiments example is the processing of the shadow application 103. In this embodiment example, the shadow application 103 is an application consisting of multiple blocks, and is divided into multiple parts at the request of the shadow application 103. In Figure 13, which explains the processing of this embodiment example, the same step numbers are assigned to the same processing as in Figure 4, and redundant explanations are omitted.

[0084] [Shadow Application Splitting Process] Figure 13 is a flowchart showing an example of a case where a shadow application is split into multiple blocks (multiple tasks) for execution. In Figure 13, the pre-processing in step S31 to the post-processing in step S39 on the CPU side, and the processing in steps S41 to S43 on the accelerator side are the same as the processing in Figure 4. In this embodiment, after the post-processing is performed by the CPU side in step S39, the shadow application 103 determines whether or not there are subsequent processes that will be split into multiple blocks (step S91).

[0085] If there are no subsequent processes that are divided into multiple blocks in step S91 (No. in step S91), the shadow application 103 executes the remaining post-processing (step S92) and terminates the process. If there are subsequent processes that are divided into multiple blocks in step S91 (Yes in step S91), the shadow application 103 returns to the pre-processing in step S31. The shadow application 103 then executes the processes from step S31 onwards for the next block. By performing the process in this manner, it becomes possible to divide the shadow application into multiple parts and execute them. The process of obtaining the estimated processing time in the computation time estimation unit 104 is performed to estimate for each block (task), and the accelerator arbitration unit 106 grants permission to use the accelerator for each block (task).

[0086] [Example of an application consisting of multiple blocks] Figure 14 shows an example of an application consisting of multiple blocks. In this example, for example, the shadow application 103 is configured by running the application to include a feature extraction processing unit 201, an object detection processing unit 203, and a caption generation unit 205. The feature extraction processing unit 201 performs feature extraction processing from input data such as image data to obtain a feature map 202.

[0087] The feature map 202 is supplied to the object detection processing unit 203, where feature data for each object is generated. For example, when the feature map 202 contains the features of n objects (where n is an arbitrary integer), the object detection processing unit 203 individually obtains feature data 204a for the first object, feature data 204b for the second object, feature data 204c for the third object, ..., and feature data 204n for the nth object. Here, the feature data 204a for the first object is processed by the first caption generation unit 205a, and the feature data 204b for the second object is processed by the second caption generation unit 205b.

[0088] When the shadow application 103 is an application consisting of such individual blocks, the shadow application can be properly executed by executing each process in an individual cycle.

[0089] Figure 15 shows an example where each block is executed individually during the downtime after the main task has been processed. The horizontal axis in Figure 15 represents time. As shown in Figure 15, after the first cycle of the main task (the task of the main application 102) is executed, the shadow application 103 performs feature extraction processing. Then, after the second cycle of the main task is executed, the shadow application 103 performs object detection processing. Furthermore, after the third cycle of the main task is executed, the shadow application 103 performs the first caption generation processing. In this way, over multiple cycles, the shadow application 103 executes each of its constituent blocks individually, and in the final cycle, it performs the nth caption generation processing.

[0090] Other configurations and processes in this embodiment are the same as those described in the first embodiment.

[0091] [Effects of the Fourth Embodiment] According to this embodiment, even if the processing of the shadow application 103 is so high-load that it cannot be completed within the execution permission time, if the accelerator's processing can be divided, a request will be sent to the accelerator arbitration unit 106 for each processing block. As a result, the processing of the shadow application 103 is divided, making it possible to execute even a high-load shadow application 103. In this case, as in this embodiment, divided execution of processing is possible even if the accelerator 130 does not have functions such as interruption and resumption.

[0092] <Modifications> The embodiments described so far are explained in detail for the purpose of making the present invention easy to understand, and are not necessarily limited to those having all the configurations described.

[0093] Furthermore, the processes described in each embodiment may be executed in combination. For example, the monitoring thread process described in the second and third embodiments may be used in combination with the process divided into multiple blocks described in the fourth embodiment.

[0094] Furthermore, although the computation time estimation unit 104 estimates the computation time from input data such as the number of objects and the number of tokens, other input data may also be referenced during computation time estimation. For example, the temperature of the vehicle control device constituting the accelerator may be measured, and the measured temperature may be used to estimate the computation time.

[0095] Furthermore, in the configuration diagrams shown in Figures 1 and 2, only control lines and information lines deemed necessary for explanation are shown, and not all control lines and information lines are necessarily shown in the actual product. In reality, it can be assumed that almost all components are interconnected.

[0096] 100...Vehicle control device, 101...Input data holding unit, 102...Main application holding unit, 103...Shadow application holding unit, 104...Calculation time estimation unit, 105...Execution permission time determination unit, 106...Accelerator arbitration unit, 110...Memory, 111...Output data holding unit, 112...Accelerator input data holding unit, 113...Accelerator output data holding unit, 114...Deadline holding unit, 115...Main application execution counter, 116...Shadow application execution counter, 120...CPU, 130...Accelerator, 140...Sensor, 150...ECU, 201...Feature extraction processing unit, 202...Feature map, 203...Object detection processing unit, 204a-204n...Feature data, 205a-205n...Caption generation unit

Claims

1. A vehicle control device comprising: a memory that stores a main application that executes vehicle control processing as a task and a shadow application that executes verification vehicle control processing as a task; a calculation time estimation unit that obtains an estimated processing time when the task is processed by an accelerator from the input data of the shadow application; an execution permission time determination unit that determines an execution permission time which is the time during which the task of the shadow application is able to use the accelerator; and an accelerator arbitration unit that grants permission to use the accelerator to the shadow application when the estimated processing time is shorter than the execution permission time, wherein the shadow application executes a task that uses the accelerator when permission to use the accelerator is granted by the accelerator arbitration unit.

2. The vehicle control device according to claim 1, wherein the calculation time estimation unit estimates the number of objects from the input data and obtains an estimated processing time from the number of objects.

3. The vehicle control device according to claim 1, wherein the calculation time estimation unit obtains the number of objects from the result of processing the input data in the main application, and obtains an estimated processing time from the number of objects.

4. The vehicle control device according to claim 1, wherein the calculation time estimation unit obtains the number of objects from the previous frame of the processing result of the shadow application task and obtains an estimated processing time from the obtained number of objects.

5. The vehicle control device according to claim 1, wherein the shadow application performs processing using divided tasks, the computation time estimation unit obtains the estimated processing time for each of the divided tasks, and the accelerator arbitration unit grants permission to use the accelerator for each of the divided tasks.

6. The vehicle control device according to claim 2, wherein the calculation time estimation unit estimates the number of tokens from the input data and estimates the processing time from the number of objects.

7. The vehicle control device according to claim 2, wherein the calculation time estimation unit estimates the output token count request described in the input data and estimates the processing time from the output token count request.

8. The vehicle control device according to claim 1, wherein the accelerator arbitration unit, after granting the permission to use the shadow application, monitors the execution of the shadow application's task, and if the shadow application's task is being executed at a deadline corresponding to the permission execution time, interrupts or terminates the task.

9. The vehicle control device according to claim 8, further comprising the accelerator arbitration unit, which causes the execution of the shadow application task that was interrupted at the deadline to wait, and then causes the shadow application task that was on hold to be executed after a predetermined periodic execution of the main application task.

10. A vehicle control method comprising a main application that executes vehicle control processing as a task and a shadow application that executes verification vehicle control processing as a task, and having an accelerator execute each task, the method comprising: a computation time estimation process that estimates the estimated processing time when the accelerator processes the task from the input data of the shadow application; an execution permission time determination process that determines the execution permission time which is the time during which the shadow application's task can use the accelerator; and an accelerator arbitration process that grants permission to use the accelerator to the shadow application if the estimated processing time is shorter than the execution permission time, wherein the shadow application executes a task that uses the accelerator if permission to use the accelerator is granted by the accelerator arbitration process.

Images