Control device and communication control method

Abstract

This control device for controlling a communication protocol includes: a recognition software unit for executing object recognition processing by using, as input, sensing results of a plurality of sensors for sensing the external environment; a dynamic change object unit for converting a protocol of data; a control software unit for requesting protocol conversion from the dynamic change object unit; and a latency monitoring unit for monitoring latency that is a time delay caused by protocol conversion. When receiving a protocol conversion request from the control software unit, the dynamic change object unit determines whether communication is congested by using the use rate of an arithmetic resource of the control device and the latency. When the communication is not congested and the data can be transmitted within the allowable time delay, the dynamic change object unit determines whether the protocol conversion request is approved in the descending order of the priority of data to be transmitted, and converts the protocol of the data for which the protocol conversion request is approved.

Description

Control device and communication control method

[0001] The present invention relates to a communication control method in an electronic control unit.

[0002] In recent years, autonomous driving software for automotive electronic control units (ECUs) has been virtualized using Docker and other tools to create containers that virtualize and run software layers on a host OS, resulting in a software structure in which multiple containers run on the ECU. In this type of structure, a function called Shadow Mode is required to improve the automotive software. Shadow Mode is a mechanism for verifying the operation of an application on the ECU. After verifying the operation, the application is written to a software stack in a separate container within the containerized ECU, thereby evolving the automotive software. However, Shadow Mode uses ROS2, and the main system is used on multiple platforms, such as the Adaptive Platform, and communication methods may differ between different platforms. For example, when using the ROS2 tool group (Rviz, CARLA, etc.), the ROS2 standard DDS communication is adopted, but when using multiple platforms such as Adaptive Platform and SOMEIP in addition to ROS2, a container created with ROS2 and the Adaptive Platform cannot communicate as is, and the data must be converted to the destination protocol before being sent.

[0003] The following prior art exists as background art in this technical field: Patent Literature 1 (JP 2013-183186 A) describes a relay device having a request receiving unit, a request forwarding unit, and a proxy sending unit, wherein the request receiving unit receives a response request from a requesting device that requests a response from a response device, the request forwarding unit sends the response request received by the request receiving unit to the response device, and the proxy sending unit sends a predetermined response request to the response device when a predetermined time has elapsed since the request forwarding unit sent the response request.

[0004] JP 2013-183186 A

[0005] In the prior art described above, an in-vehicle data relay device switches communication protocol settings using a communication protocol switching command. However, this method has difficulty dynamically responding to constantly changing conditions, such as the situation around the vehicle (relative distance to an object) and communication load. For example, if an object on the left side is close by, the left side has a higher urgency, so the sensor information on the left side must be given a higher priority. While it is necessary to constantly respond to such conditions, the relay device described in Patent Document 1 has difficulty dynamically responding to constantly changing conditions.

[0006] A representative example of the invention disclosed in the present application is as follows: That is, a control device that controls a communication protocol includes: a recognition software unit that executes object recognition processing using sensing results from a plurality of sensors that sense the external world as input; a dynamic change target unit that converts the protocol of data; a control software unit that requests protocol conversion from the dynamic change target unit; and a latency monitoring unit that monitors latency, which is a delay time caused by the protocol conversion, wherein, upon receiving a protocol conversion request from the control software unit, the dynamic change target unit determines whether communication is congested using the utilization rate of the control device's computing resources and the latency, and if communication is not congested and data can be transmitted within the allowable delay time, determines whether to accept the protocol conversion request in descending order of priority of the data to be transmitted, and converts the protocol of the data for which the protocol conversion request is accepted.

[0007] According to one aspect of the present invention, it is possible to reduce latency due to protocol conversion and suppress the load on the electronic control unit. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments.

[0008] 1 is a configuration diagram of a vehicle control system of a first embodiment. FIG. 2 is a functional configuration diagram of the vehicle control system of the first embodiment. FIG. 3 is a flowchart of data reception processing of the first embodiment. FIG. 4 is a flowchart of a program that realizes data transmission processing of the first embodiment. FIG. 5 is a diagram showing a storage area / priority definition table for transmitted and received data stored in a data storage unit of the first embodiment. FIG. 6 is a diagram showing priority changes depending on the distance to an object in the first embodiment. FIG. 7 is a diagram showing priority changes depending on the distance to an object in the first embodiment. FIG. 8 is a diagram showing priority changes depending on the distance to an object in the first embodiment. FIG. 9 is a configuration diagram of a vehicle control system of a second embodiment. FIG. 10 is a functional configuration diagram of a vehicle control system of the second embodiment. FIG. 11 is a flowchart of data reception processing of the second embodiment. FIG. 12 is a diagram showing a fault state of a sensor stored in a sensor state storage unit of the second embodiment. FIG. 13 is a diagram showing a fault state of a sensor stored in a sensor state storage unit of the second embodiment. FIG. 14 is a diagram showing a fault sensor complementation process of the second embodiment.

[0009] Some embodiments will be described with reference to the drawings. Note that the embodiments described below do not limit the scope of the claimed invention, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention.

[0010] In the following description, processing may be described with a "program" as the operating entity. A program executes processing by a processor (e.g., a CPU (Central Processing Unit)) using appropriate storage resources (e.g., memory) and / or a communication interface device (e.g., a port), but the entire processing may also be executed by the processor. Processing described with a program as the operating entity may also be processing performed by a device including a processor. It may also include dedicated hardware circuits that perform some or all of the processing executed by the processor. A computer program may be installed on a device from a program source. The program source may be, for example, a program distribution server or a non-transitory recording medium that is readable by a computer.

[0011] First Embodiment First, a first embodiment will be described.

[0012] FIG. 1A is a configuration diagram of a vehicle control system 1000 according to the first embodiment.

[0013] The vehicle control system 1000 of the first embodiment is a system mounted on a vehicle such as an automobile, and includes a sensor 10 and ECUs 11 and 20. The sensor 10, the other ECU 11, and the ECU 20 can communicate with each other via in-vehicle networks 30 and 40. The in-vehicle networks 30 and 40 may be any communication network such as Ethernet (registered trademark) or CAN-FD (CAN with Flexible Data-Rate).

[0014] The sensor 10 is one or more sensors that acquire information about the surrounding environment of the vehicle, such as a radar, a LiDAR, a camera, etc. The sensor 10 outputs the acquired information about the surrounding environment to the ECU 20.

[0015] The ECU 11 is an electronic control unit (ECU) that outputs control signals to one or more actuators that operate the accelerator, brake, steering, etc., to control the vehicle based on control information output from the ECU 20. The ECU 20 includes a CPU 22 and a memory 21. The CPU 22 is a computing resource that executes the processing of each unit according to programs stored in the memory 21. The memory 21 is, for example, a random access memory (RAM) and stores programs executed by the CPU 22 and information used when the programs are executed. The memory 21 stores programs for a dynamic change target unit 111, a usage limit overage monitoring unit 112, a latency monitoring unit 113, a data storage unit 114, a recognition software unit 115, a control software unit 116, a data transmission unit 117, and a data acquisition unit 118. The functions of each program will be described later. Note that, for convenience, the following description may refer to the programs as the operating entities, but in reality, it is the CPU 22 that executes the programs that performs the processing.

[0016] FIG. 1B is a functional configuration diagram of a vehicle control system 1000 according to the first embodiment.

[0017] The data acquisition unit 118 transfers the data received from the sensor 10 and the data received from the other ECU 11 to the recognition software unit 115. The recognition software unit 115 recognizes objects using the data received from the sensor 10 and the data received from the other ECU 11, and outputs information about the recognized objects to the dynamic change target unit 111.

[0018] The dynamic change target unit 111 requests confirmation of the usage limit from the usage limit exceedance monitoring unit 112. When the usage limit exceedance monitoring unit 112 receives a request to confirm the usage limit from the dynamic change target unit 111, it compares the number of unprocessed protocol conversion requests received from the control software unit 116 with a predetermined threshold and notifies the dynamic change target unit 111 of information on whether the usage limit has been exceeded. Because the number of protocol conversion requests that the dynamic change target unit 111 can process simultaneously is set, the usage limit exceedance monitoring unit 112 monitors the protocol conversion processing by the dynamic change target unit 111 and determines whether the usage limit of the dynamic change target unit 111 has been exceeded.

[0019] Furthermore, the dynamic change target unit 111 requests latency information from the latency monitoring unit 113. The latency monitoring unit 113 monitors the latency, which is the delay time from when the control software unit 116 transmits a protocol change request until when the control software unit 116 acquires the protocol-converted data. In other words, the latency monitored by the latency monitoring unit 113 represents a communication delay within the ECU 20. Then, upon receiving a request for latency information from the dynamic change target unit 111, the latency monitoring unit 113 notifies the dynamic change target unit 111 of the latency information.

[0020] The dynamic change target unit 111 also determines the priority of protocol conversion for the data based on the acquired information on whether the usage limit has been exceeded and the latency information.The dynamic change target unit 111 then writes the received data and the determined priority to the data storage unit 114.The dynamic change target unit 111 can obtain the data and the priority from the data storage unit 114.

[0021] Furthermore, when the dynamic change target unit 111 receives a protocol conversion request from the control software unit 116, the dynamic change target unit 111 outputs to the control software unit 116 data obtained by protocol-converting the object recognition result, or a protocol conversion rejection notification.

[0022] The control software unit 116 outputs the protocol-converted data received from the dynamic change target unit 111 to the data transmission unit 117. The data transmission unit 117 outputs the protocol-converted data to the other ECU 11.

[0023] FIG. 2 is a flowchart of the data reception process according to the first embodiment.

[0024] The data reception process is performed by the CPU 22 executing a program and the data acquisition unit 118, for example, periodically when information is received from the sensor 10.

[0025] First, the data acquisition unit 118 determines whether data has been received from the sensor 10 (step S110). If the data acquisition unit 118 has received data (step S110: Yes), the dynamic change target unit 111 acquires distance information from the object in the received data (step S120) and determines the distance between the host vehicle and the object (step S130). If the distance between the host vehicle and the object is closer than a predetermined threshold (step S130: Yes), the dynamic change target unit 111 sets the priority of the object's data to a high value (step S140). On the other hand, if the distance between the host vehicle and the object is farther than the predetermined threshold (step S130: No), the dynamic change target unit 111 does not change the priority of the object's data and proceeds to step S150. Then, the dynamic change target unit 111 saves the received data and priority in the data storage unit 114 (S150) and terminates the data reception process. On the other hand, if data has not been received (step S110: No), the data reception process terminates.

[0026] FIG. 3 is a flowchart of a program that realizes the data transmission process of the first embodiment.

[0027] The data transmission process is executed when the dynamic change target unit 111 receives a protocol conversion request from the control software unit 116 .

[0028] The dynamic change target unit 111 checks the contents of the protocol conversion request received from the control software unit 116 (step S210), obtains current latency information and throughput information from the latency monitoring unit 113, and obtains priority information for the data for which protocol conversion is requested from the data storage unit 114 (step S220).

[0029] The dynamic change target unit 111 then determines whether the usage limit has been exceeded based on the usage limit exceeding information received from the usage limit exceeding monitoring unit 112 (step S225). If the number of unprocessed protocol conversion requests is greater than a predetermined threshold and the usage limit has been exceeded (step S225: Yes), the dynamic change target unit 111 transmits a protocol conversion request rejection notification including the reason for rejection to the protocol conversion request source (step S280), and ends the process.

[0030] On the other hand, if the usage limit has not been exceeded (step S225: No), it is determined whether communication is congested using the utilization rate and throughput of the CPU 22 (step S230). If it is determined that communication is congested because at least one of the utilization rate of the CPU 22 exceeds a predetermined threshold and the throughput exceeds a predetermined threshold (step S230: Yes), it is determined whether the data size of the data for which protocol conversion is requested can be transferred within the permissible delay time by referring to the allowable delay time and latency determined for each type of data (step S240).

[0031] The calculation of the transmittable data size in step S240, i.e., the calculation of the transmittable data size from the latency information and throughput information, will now be described. For example, consider a case where the control software unit 116, which requires timestamps and object distance information, requests sensor information at 10 ms intervals and has a latency of 5 ms. In addition to the timestamp and object distance information, the sensor information includes information on numerous objects, such as speed information and position information, and is transmitted as a structure containing this data. Therefore, when converting the protocol and transmitting data, all data must be converted, which may prevent the control software unit 116 from meeting the data-requiring interval. Therefore, if the throughput information is 10 Mbps and the control software processing time is 4 ms, approximately 10 MB of data can be transmitted. In this manner, the transmittable data size is calculated.

[0032] On the other hand, if it is determined from the utilization rate and throughput of the CPU 22 that communication is not congested (step S230: No), the process proceeds to step S250.

[0033] Next, the dynamic change target unit 111 determines whether to accept the protocol conversion request (step S250). For example, if the data size of the data for which protocol conversion is requested can be transferred within the allowable delay time, the dynamic change target unit 111 determines to accept the protocol conversion request (step S250: Yes), executes the protocol conversion process (step S260), transmits the protocol-converted data (step S270), and terminates the data transmission process. If the data size of the data for which protocol conversion is requested cannot be transferred within the allowable delay time, the dynamic change target unit 111 determines to reject the protocol conversion request (step S250: No), transmits the reason for the rejection to the protocol conversion request source (S280), and terminates the data transmission process.

[0034] FIG. 4 is a diagram showing a storage area / priority definition table for transmitted / received data stored in the data storage unit 114 of the memory 21 according to the first embodiment.

[0035] The left column of the definition table in the data storage unit 114 indicates the data storage area where data is saved, and the right column indicates the current priority of the data. The data acquisition unit 118 can reset the priority stored in the data storage unit 114 depending on, for example, the situation around the vehicle. Specifically, it is advisable to set a high priority for data relating to objects that are close to the vehicle.

[0036] 5A, 5B, 5C, and 5D are diagrams showing priority changes depending on the distance to an object in the first embodiment.

[0037] As shown in Figures 5A and 5B, the vehicle 410 has a left sensor 411 that observes the left front of the vehicle and a right sensor 412 that observes the right front of the vehicle. When the right sensor 412 detects an object 420, if the relative distance between the object 420 observed by the right sensor 412 and the vehicle 410 is equal to or less than a predetermined threshold, it is determined that there is a risk of contact, and the priority of the information observed by that sensor is set to high. Since the priority of sensor information is set to high from the beginning, sensor information with a high priority that poses a high risk is set to a higher priority than other sensor information. As a result, observation information of the object 420 that may be involved in contact can be immediately transmitted, thereby preventing contact accidents.

[0038] 5C and 5D, as the vehicle 410 advances from the state shown in Fig. 5B, the object 420 moves out of the observation range of the right sensor 412, and the left sensor 411 observes the object 421. In this case, if the relative distance between the object 421 observed by the left sensor 411 and the vehicle itself is equal to or less than a predetermined threshold, it is determined that there is a risk of contact, and the priority of the information observed by that sensor is set to high. Because the right sensor 412 does not observe the object 420, the priority is lowered and set to the initial priority.

[0039] As described above, according to an embodiment of the present invention, the priority can be changed depending on the relative distance from the vehicle position, and the protocol conversion restrictions and information that should be prioritized for protocol conversion are determined depending on the communication conditions, thereby reducing the latency caused by protocol conversion and suppressing the load on the electronic control unit.

[0040] <Embodiment 2> A second embodiment will be described below. The communication control device of the second embodiment differs from the communication control device of the first embodiment described above in that it includes a sensor state storage unit 119. In the second embodiment, differences from the first embodiment described above will be mainly described, and the same configurations and processes as in the first embodiment will be assigned the same reference numerals as in the first embodiment, and descriptions thereof will be omitted.

[0041] FIG. 6A is a configuration diagram of a vehicle control system 1000 according to the second embodiment.

[0042] The vehicle control system 1000 of the second embodiment is a system mounted on a vehicle such as an automobile, and includes a sensor 10 and ECUs 11 and 20. The sensor 10, the other ECU 11, and the ECU 20 can communicate with each other via in-vehicle networks 30 and 40. The sensor 10, the ECU 11, and the in-vehicle networks 30 and 40 are the same as those in the first embodiment described above.

[0043] The memory 21 of the ECU 20 of the second embodiment stores programs for a dynamic change target unit 111, a usage limit over monitor unit 112, a latency monitor unit 113, a data storage unit 114, a recognition software unit 115, a control software unit 116, a data transmission unit 117, a data acquisition unit 118, and a sensor state storage unit 119. The configuration of the ECU 20 other than the dynamic change target unit 111 and the sensor state storage unit 119 is the same as that of the first embodiment described above.

[0044] The dynamic change target unit 111 checks the failure status of the sensor 10. The sensor state storage unit 119 stores the state of the sensor 10 checked by the dynamic change target unit 111.

[0045] FIG. 6B is a functional configuration diagram of the vehicle control system 1000 according to the second embodiment.

[0046] The data acquisition unit 118 transfers the data received from the sensor 10, the status information of the sensor 10, and the data received from the other ECUs 11 to the recognition software unit 115. The recognition software unit 115 recognizes an object using the data received from the sensor 10, the status information of the sensor 10, and the data received from the other ECUs 11, and outputs information on the recognized object to the dynamic change target unit 111.

[0047] The dynamic change target unit 111 requests the usage limit exceedance monitoring unit 112 to confirm the usage limit. Upon receiving a usage limit confirmation request from the dynamic change target unit 111, the usage limit exceedance monitoring unit 112 notifies the dynamic change target unit 111 of information on whether the usage limit has been exceeded. Furthermore, the dynamic change target unit 111 requests latency information from the latency monitoring unit 113. Upon receiving a latency information request from the dynamic change target unit 111, the latency monitoring unit 113 notifies the dynamic change target unit 111 of the latency information.

[0048] The dynamic change target unit 111 also determines the priority of protocol conversion for the data based on the acquired information on whether the usage limit has been exceeded and the latency information.The dynamic change target unit 111 then writes the received data and the determined priority to the data storage unit 114.The dynamic change target unit 111 can obtain the data and the priority from the data storage unit 114.

[0049] Furthermore, the dynamic change target unit 111 writes the state of the sensor 10 to the sensor state storage unit 119. The dynamic change target unit 111 can acquire the state of the sensor 10 from the sensor state storage unit 119.

[0050] Furthermore, when the dynamic change target unit 111 receives a conversion request from the control software unit 116, the dynamic change target unit 111 outputs data obtained by protocol-converting the object recognition result or a protocol conversion rejection notification to the control software unit 116.

[0051] The control software unit 116 outputs the protocol-converted data received from the dynamic change target unit 111 to the data transmission unit 117. The data transmission unit 117 outputs the protocol-converted data to the other ECU 11.

[0052] 7 is a flowchart of the data reception process of the second embodiment. In FIG. 7, the same processes as those in the data reception process of the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

[0053] First, the data acquisition unit 118 determines whether data has been received from the sensor 10 (step S110). If the data acquisition unit 118 has received data (step S110: Yes), the dynamic change target unit 111 checks the failure status of the sensor 10 confirmed by the dynamic change target unit 111 (step S320). If a failure has occurred in the sensor 10 (step S330: Yes), the dynamic change target unit 111 identifies a sensor 10 to substitute for the failed sensor 10 and executes a sensor complementation process to generate sensor data simulating the failed sensor 10 from the output value of the substitute sensor 10 (step S340).

[0054] Next, the dynamic change target unit 111 obtains distance information between the data and the object from the data storage unit 114 (step S120). The dynamic change target unit 111 determines whether the distance between the host vehicle and the object is closer than a predetermined threshold or whether the sensor 10 has performed interpolation processing (step S350). If the distance between the host vehicle and the object is close or if the sensor 10 has performed interpolation processing (step S350: Yes), the dynamic change target unit 111 changes the priority based on the interpolation status of the sensor 10 and the distance information between the host vehicle and the object (S360). The dynamic change target unit 111 then stores the received data and priority information in the data storage unit 114 (S150) and ends the data reception process.

[0055] On the other hand, if the sensor 10 has not performed the complementation process and the distance between the vehicle and the object is greater than a predetermined threshold (step S350: N), the priority is not changed, the received data and priority information are stored in the data storage unit 114 (step S150), and the data reception process is terminated.

[0056] 8A and 8B are diagrams showing the failure state of the sensor 10 stored in the sensor state storage unit 119 of the second embodiment.

[0057] The sensor state storage unit 119 stores the type of sensor 10 (installation location and recognition range) in association with the failure status of the sensor 10. The dynamic change target unit 111 checks for a failure of the sensor 10 based on the information stored in the sensor state storage unit 119. When a sensor 10 is faulty, the dynamic change target unit 111 identifies a sensor 10 that can complement the recognition range of the faulty sensor 10, and increases the priority of the information observed by the sensor 10 that can complement.

[0058] 8A and 9A, vehicle 200 has front camera 201 that observes the area ahead of vehicle 200, front left camera 202 that observes the area ahead of vehicle 200, and front right camera 203 that observes the area ahead of vehicle 200. If front camera 201 malfunctions, the observation range of front camera 201 is complemented by front right camera 203 and front left camera 202, and therefore, the priority of information observed by front right camera 203 and front left camera 202 is set high.

[0059] Also, as shown in Figures 8B and 9B, if the front right camera 203 and the front left camera 202 are malfunctioning, the front camera 201 complements the observation range of the front right camera 203 and the front left camera 202, so the priority of the information observed by the front camera 201 is set high.

[0060] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with other configurations.

[0061] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0062] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, hard disk, or SSD (Solid State Drive), or in a recording medium such as an IC card, SD card, or DVD.

[0063] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected.

Claims

1. A control device that controls communication protocols, comprising: a recognition software unit that executes object recognition processing using sensing results from multiple sensors that sense the outside world as input; a dynamic change target unit that converts the protocol of data; a control software unit that requests protocol conversion from the dynamic change target unit; and a latency monitoring unit that monitors latency, which is the delay time caused by protocol conversion, wherein, upon receiving a protocol conversion request from the control software unit, the dynamic change target unit determines whether communication is congested using the utilization rate of the control device's computing resources and the latency; if communication is not congested and data can be transmitted within the allowable delay time, determines whether to accept the protocol conversion request in descending order of priority of the data to be transmitted; and converts the protocol of the data that accepts the protocol conversion request.

2. A control device according to claim 1, characterized in that the latency monitoring unit monitors communication throughput, and the dynamic change target unit determines that data can be transmitted within the allowable delay time if the data size to be transmitted does not exceed the transmittable data size calculated from the latency and the throughput.

3. A control device according to claim 1, wherein the dynamic change target unit does not accept a protocol conversion request if the number of protocol conversion processes exceeds a limit value.

4. A control device according to claim 1, characterized in that the dynamic change target unit sets a high priority to data calculated using the output of the sensor sensing the object if the relative distance between the object and the sensor is smaller than a predetermined threshold.

5. A control device according to claim 1, characterized in that the dynamic change target unit sets a high priority to data calculated using the output of a sensor that complements a failed sensor.

6. A communication control method in which a control device controls communications, the control device having: a recognition software unit that executes object recognition processing using sensing results from multiple sensors that sense the outside world as input; a dynamic change target unit that converts the protocol of data; a control software unit that requests protocol conversion from the dynamic change target unit; and a latency monitoring unit that monitors latency, which is the delay time caused by protocol conversion; wherein the communication control method is characterized in that, when the dynamic change target unit receives a protocol conversion request from the control software unit, it determines whether communication is congested using the utilization rate of the control device's computing resources and the latency; if communication is not congested and data can be transmitted within the allowable delay time, the dynamic change target unit determines whether to accept the protocol conversion request in descending order of priority of the data to be transmitted; and the dynamic change target unit converts the protocol of the data that accepts the protocol conversion request.

Images