Vehicle control device, vehicle control method, vehicle control computer program, priority setting device, and vehicle control system

Abstract

To provide a vehicle control device which can execute processing with high priority without delay, in the situation of a road section where a vehicle is positioned or vehicle periphery.SOLUTION: A vehicle control device has: storage parts (14, 23) for storing a priority table indicating priority of each of plurality of processes relating to automatic driving or driving support of a vehicle 2, for each situation of each road section or in which the periphery of the vehicle 2 can be taken; a determination part 32 for determining the situation of the road section where the vehicle 2 is positioned or the periphery of the vehicle 2, according to a sensor signal generated by a sensor 11 that is mounted on the vehicle 2 and detects the peripheral situation of the vehicle 2 or the position of the vehicle 2; a priority determination part 33 for determining each of the priority of the plurality of processes, according to the situation of the road section where the vehicle 2 is positioned or the periphery of the vehicle 2, with reference to the priority table; and a control part 34 for sequentially executing each of the plurality of processes from the processing with higher priority using a shared resource 241.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a vehicle control device, a vehicle control method, a vehicle control computer program, a priority setting device, and a vehicle control system.

Background Art

[0002] In the automatic driving control or driving support of a vehicle, the processing to be applied may differ depending on the position of the vehicle. Therefore, a technique has been proposed that enables automatic driving control according to the current position of the vehicle (see Patent Document 1).

[0003] The automatic driving device disclosed in Patent Document 1 refers to map data including a driving automation level indicating the level of automation of automatic driving control, which is associated with each predetermined section of a road. Then, based on this map data and the current position of the host vehicle, the automatic driving device generates, as guidance information, information regarding automatic driving control according to the current position of the host vehicle and the driving automation level ahead thereof.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Multiple processes may be performed for the automatic driving or driving support of a vehicle. In such a case, in order not to interfere with the automatic driving or driving support, it is required to appropriately execute these multiple processes according to the road section where the vehicle is located or the situation around the vehicle.

[0006] Therefore, an object of the present invention is to provide a vehicle control device capable of executing, without delay, a process with a high priority in the road section where the vehicle is located or the situation around the vehicle.

Means for Solving the Problem

[0007] According to one embodiment, a vehicle control device is provided. This vehicle control device Among the individual road segments represented in the map information For each road section and , stores a priority table representing the priority of each of a plurality of processes related to automatic driving or driving support of the vehicle regarding the execution order in a storage unit and the plurality of processes include at least two of a first process for controlling the vehicle for lane change, a second process for collecting information about predetermined ground objects around the vehicle based on an image obtained by a camera mounted on the vehicle, and a third process for determining the state of the driver of the vehicle and a determination unit that determines the road section where the vehicle is located according to the position of both , the vehicle , a priority determination unit that refers to the priority table and determines the priority of each of the plurality of processes according to the road section where the vehicle is located, and a control unit that executes each of the plurality of processes from the one with the higher priority of the process between on a shared resource among . in order assigns the processes Shared resource to It has.

[0008] According to another aspect of the present invention, a vehicle control method is provided. This vehicle control method , the vehicle determines the road section where the vehicle is located according to the position of both Among the individual road segments represented in the map information , refers to a priority table representing the priority of each of a plurality of processes related to automatic driving or driving support of the vehicle for each road section between , determines the priority of each of the plurality of processes according to the road section where the vehicle is located, and Among the individual executes each of the plurality of processes on a shared resource from the one with the higher priority of the process and . regarding the execution order This includes among . The plurality of processes include at least two of a first process for controlling the vehicle for lane change, a second process for collecting information about predetermined ground objects around the vehicle based on an image obtained by a camera mounted on the vehicle, and a third process for determining the state of the driver of the vehicle, For each of the plurality of processes, from the one with the higher priority of the process in order assigns the processes Shared resource to Execute.

[0009] According to still another aspect of the present invention, a vehicle control computer program is provided. This vehicle control computer program , the vehicle determines the road section where the vehicle is located according to the position of both Among the individual road segments represented in the map information , refers to a priority table representing the priority of each of a plurality of processes related to automatic driving or driving support of the vehicle for each road section between , determines the road section where the vehicle is located, Among the individual For each road section and , for each of a plurality of processes related to automatic driving or driving support of the vehicleregarding the execution order Referring to a priority table representing priorities, for the road section where the vehicle is located among determine the priority of each of a plurality of processes according to the situation, The plurality of processes include at least two of a first process for controlling the vehicle for lane change, a second process for collecting information about predetermined ground objects around the vehicle based on an image obtained by a camera mounted on the vehicle, and a third process for determining the state of the driver of the vehicle, for each of the plurality of processes, starting from the one with the higher priority of the process in order assigns the processes Shared resource to sequentially execute, including instructions for causing a processor mounted on the vehicle to execute.

[0010] According to still another aspect of the present invention, a priority setting device is provided. This priority setting device includes a storage unit, and when any one of a plurality of processes related to the autonomous driving or driving support of the vehicle is executed from at least one vehicle, it represents the road section on which the vehicle was traveling or the situation around the vehicle at that time, and also represents the executed process. A reception processing unit that stores the received execution information in the storage unit each time the execution information is received via the communication unit, and based on the plurality of execution information stored in the storage unit, for each road section or for each possible situation around the vehicle, a priority setting unit that sets the priority of each of the plurality of processes so that the process with a higher number of executions among the plurality of processes has a higher priority, and for each road section or for each possible situation around the vehicle, a notification processing unit that notifies the vehicle via the communication unit of a priority table representing the priority of each of the plurality of processes.

[0011] According to still another aspect of the present invention, a vehicle control system is provided that includes at least one vehicle and a priority setting device capable of communicating with each of the at least one vehicle. In this vehicle control system, each of the at least one vehicle Among the individual For each road section and stores a priority table representing the priority of each of a plurality of processes related to the autonomous driving or driving support of the vehicle regarding the execution order in a storage unit and the plurality of processes include at least two of a first process for controlling the vehicle for lane change, a second process for collecting information about predetermined ground objects around the vehicle based on an image obtained by a camera mounted on the vehicle, and a third process for determining the state of the driver of the vehicle a storage unit and , the vehicle depending on the positions of both, the road section where the vehicle is located between a determination unit that determines, and referring to the priority table, the road section where the vehicle is located amongA priority determination unit that determines the priority of each of a plurality of processes according to the situation, and each of the plurality of processes is arranged from the one with the higher priority of the process in order assigns the processes Shared resource to A control unit that executes, among the plurality of processes When the execution of the process Actually of The process reflected in the behavior of the vehicle and , When the process is executed The road section where the vehicle is located between and Generate execution information representing the situation, and the generated execution information is Communication unit is An execution information generation unit that transmits to the priority setting device via the communication unit. Further, the priority setting device includes a storage unit, and each time it receives execution information from any one of at least one vehicle via the communication unit, a reception processing unit that stores the execution information in the storage unit of the priority setting device, Among the individual road segments represented in the map information Stored in the storage unit of the priority setting device for the actual Market information For road segments where the number of executions has reached a predetermined number, the predetermined number of execution information Based on , multiple Among the several processes that have been executed , and is reflected in the actual behavior of the vehicle The higher the number of times a process is executed, the higher its priority. Set the priority of each of the plurality of processes to update the priority table Priority setting unit, The updated A notification processing unit that notifies each of at least one vehicle of the priority table via the communication unit. to at least has.

Effect of the Invention

[0012] The vehicle control device according to the present invention has the effect that it can execute a process with a high priority in the road section where the vehicle is located or the situation around the vehicle without delay.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0014] Hereinafter, with reference to the drawings, a vehicle control device, a vehicle control method and a vehicle control computer program executed in the vehicle control device, and a priority setting device and a vehicle control system including the vehicle control device and the priority setting device will be described. This vehicle control device is mounted on a vehicle, and when any one of a plurality of processes related to automatic driving or driving support of the vehicle is executed, execution information representing the executed process and the road section on which the vehicle was traveling or the situation around the vehicle when the process was executed is generated. Then, this vehicle control device transmits the generated execution information to the priority setting device. On the other hand, the priority setting device sets the priority of each process based on the plurality of received execution information so that the process with a higher number of executions has a higher priority for each road section or for each situation around the vehicle. Then, the priority setting device distributes a priority table representing the priority of each process set for each road section or for each possible situation around the vehicle to each vehicle.

[0015] Furthermore, when automatically controlling the vehicle or assisting the driver in driving, the vehicle control device refers to a priority table that represents the priority of each of a plurality of processes related to the automatic driving or driving assistance of the vehicle for each road section or for each possible situation around the vehicle. Then, this vehicle control device determines the priority of each process according to the road section where the vehicle is located or the situation around the vehicle, and sequentially executes using the shared resource starting from the process with the highest priority.

[0016] The plurality of processes related to the automatic driving or driving assistance of the vehicle include processes related to lane change, processes related to speed control such as auto cruise control (ACC), and processes related to collision prevention. Further, the plurality of processes may further include processes related to the determination of the driver's state and processes related to the collection of information around the vehicle. Note that the plurality of processes related to the automatic driving or driving assistance of the vehicle may include processes other than the above-mentioned processes. Hereinafter, the plurality of processes related to the automatic driving or driving assistance of the vehicle may be simply referred to as a plurality of processes.

[0017] FIG. 1 is a schematic configuration diagram of a vehicle control system in which a vehicle control device and a priority setting device are implemented. In the present embodiment, the vehicle control system 1 includes at least one vehicle 2 and a server 3 which is an example of a priority setting device. Each vehicle 2 is connected to the server 3 via the wireless base station 5 and the communication network 4, for example, by accessing a wireless base station 5 connected via a communication network 4 to which the server 3 is connected and a gateway (not shown). Note that in FIG. 1, only one vehicle 2 is shown for simplicity, but the vehicle control system 1 may have a plurality of vehicles 2. Similarly, in FIG. 1, only one wireless base station 5 is shown, but a plurality of wireless base stations 5 may be connected to the communication network 4.

[0018] FIG. 2 is a schematic configuration diagram of vehicle 2. Vehicle 2 includes a camera 11, a GPS receiver 12, a wireless communication terminal 13, a storage device 14, and an electronic control unit (ECU) 15 which is an example of a vehicle control device. The camera 11, GPS receiver 12, wireless communication terminal 13, and storage device 14 are communicably connected to the ECU 15 via an in-vehicle network compliant with a standard such as Controller Area Network. Further, vehicle 2 may further include a navigation device (not shown) that searches for a planned route of vehicle 2 and navigates vehicle 2 to travel according to the planned route. Furthermore, vehicle 2 may have a distance sensor (not shown) such as LiDAR or radar that measures the distance from vehicle 2 to an object existing around vehicle 2. Additionally, vehicle 2 may have a driver monitoring camera (not shown) provided to photograph the driver.

[0019] The camera 11 is an example of a sensor that detects the situation around vehicle 2, and includes a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or C-MOS, and an imaging optical system that forms an image of an area to be photographed on the two-dimensional detector. The camera 11 is mounted, for example, inside the vehicle compartment of vehicle 2 so as to face the front of vehicle 2. The camera 11 photographs the front area of vehicle 2 at a predetermined photographing cycle (for example, 1 / 30 second to 1 / 10 second) and generates an image in which the front area is shown. The image obtained by the camera 11 is an example of a sensor signal, and may be a color image or a gray image. Note that a plurality of cameras 11 with different photographing directions or focal lengths may be provided in vehicle 2.

[0020] Each time the camera 11 generates an image, the generated image is output to the ECU 15 via the in-vehicle network.

[0021] The GPS receiver 12 receives GPS signals from GPS satellites at a predetermined cycle, and determines the vehicle 2's own position based on the received GPS signals. Note that the predetermined cycle at which the GPS receiver 12 determines the vehicle 2's own position may be different from the imaging cycle of the camera 11. Then, the GPS receiver 12 outputs, at a predetermined cycle, positioning information representing the result of determining the vehicle 2's own position based on the GPS signals to the ECU 15 via the in-vehicle network. Note that the vehicle 2 may have a receiver compliant with a satellite positioning system other than the GPS receiver 12. In this case, the receiver may determine the vehicle 2's own position.

[0022] The wireless communication terminal 13 is an example of a communication unit or a communication device, and is a device that executes wireless communication processing compliant with a predetermined wireless communication standard. For example, by accessing the wireless base station 5, the wireless communication terminal 13 is connected to the server 3 via the wireless base station 5 and the communication network 4. Then, the wireless communication terminal 13 generates an uplink wireless signal including execution information received from the ECU 15 and the like. Then, the wireless communication terminal 13 transmits the uplink wireless signal to the wireless base station 5, thereby transmitting execution information and the like to the server 3. Further, the wireless communication terminal 13 receives a downlink wireless signal from the wireless base station 5 and passes the priority table and the like from the server 3 included in the wireless signal to the ECU 15. Note that the downlink wireless signal may include traffic information distributed from a traffic information server (not shown) or weather information distributed from a weather information server (not shown).

[0023] The storage device 14 has, for example, a hard disk device, a non-volatile semiconductor memory, or an optical recording medium and its access device. Then, the storage device 14 stores a high-precision map. The high-precision map includes information used for automatic driving control of the vehicle, for example, information on the number of lanes for each road included in a predetermined area represented by the high-precision map, information representing road markings such as lane dividing lines or stop lines, and information representing road signs. Further, a priority table may be associated with the high-precision map.

[0024] Furthermore, the storage device 14 may have a processor for executing processes such as the update process of the high-precision map and the process related to the read request of the high-precision map from the ECU 15. And the storage device 14 may, for example, transmit a request for acquiring a high-precision map to the map server together with the current position of the vehicle 2 via the wireless communication terminal 13 each time the vehicle 2 moves by a predetermined distance. Also, the storage device 14 may receive a high-precision map for a predetermined area around the current position of the vehicle 2 from the map server via the wireless communication terminal 13. Further, when receiving a read request of the high-precision map from the ECU 15, the storage device 14 cuts out a relatively narrow range including the current position of the vehicle 2 from the stored high-precision map, which is narrower than the above-mentioned predetermined area, and outputs it to the ECU 15 via the in-vehicle network.

[0025] FIG. 3 is a hardware configuration diagram of the ECU 15 which is an embodiment of the vehicle control device. The ECU 15 automatically controls the vehicle 2 or supports the driving of the driver of the vehicle 2. Further, the ECU 15 generates execution information based on the processes executed in the vehicle 2. For this purpose, the ECU 15 has a communication interface 21, a memory 22, a buffer memory 23, and a processor 24. Note that the communication interface 21, the memory 22, the buffer memory 23, and the processor 24 may be configured as different circuits from each other, or may be integrally configured as one integrated circuit.

[0026] The communication interface 21 has an interface circuit for connecting the ECU 15 to the in-vehicle network. That is, the communication interface 21 is connected to the camera 11 via the in-vehicle network. And each time the communication interface 21 receives an image from the camera 11, it transfers the received image to the processor 24. Also, each time the communication interface 21 receives positioning information from the GPS receiver 12, it transfers the received positioning information to the processor 24. Further, the communication interface 21 transfers the high-precision map read from the storage device 14 to the processor 24. Furthermore, the communication interface 21 transfers to the processor 24 things such as the priority table received from the wireless communication terminal 13. Furthermore, the communication interface 21 outputs the execution information received from the processor 24 to the wireless communication terminal 13.

[0027] The memory 22 is an example of a storage unit and has, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. And the memory 22 stores the algorithm of the vehicle control process executed by the processor 24 of the ECU 15, various data and parameters used in the vehicle control process. For example, the memory 22 stores the high-precision map read from the storage device 14, the priority table received from the server 3, and the parameter set for identifying the identifier used in the vehicle control process. Further, the memory 22 may store traffic information and weather information. Furthermore, the memory 22 stores various data generated during the vehicle control process, such as execution information and information about the detected object, for a certain period of time.

[0028] The buffer memory 23 is another example of a storage unit and has, for example, a volatile semiconductor memory. And the buffer memory 23 temporarily stores the image received from the camera 11 and the positioning information received from the GPS receiver 12. The buffer memory 23 may temporarily store the driver monitor image received from a driver monitor camera (not shown).

[0029] The processor 24 has, for example, one or more CPUs (Central Processing Units) and its peripheral circuits. The processor 24 may further have a numerical operation circuit or a graphics processing circuit (Graphics Processing Unit, GPU). Further, the processor 24 has a shared memory 241. The shared memory 241 is an example of a shared resource and is configured as a memory circuit accessible from each arithmetic circuit or each processing circuit such as the CPU and GPU included in the processor 24, for example.

[0030] While the vehicle 2 is running, each time the processor 24 receives an image from the camera 11, the received image is written into the buffer memory 23. Similarly, each time the processor 24 receives positioning information from the GPS receiver 12, the received positioning information is written into the buffer memory 23. Further, the processor 24 stores the highly accurate map read from the storage device 14, the traffic information and the weather information received via the wireless communication terminal 13 in the memory 22. Further, the processor 24 executes vehicle control processing based on the image and the like stored in the buffer memory 23.

[0031] FIG. 4 is a functional block diagram of the processor 24 of the ECU 15 regarding vehicle control processing. The processor 24 has an execution information generation unit 31, a determination unit 32, a priority determination unit 33, and a processing control unit 34. Each of these units included in the processor 24 is a functional module realized by, for example, a computer program operating on the processor 24. Among these units, the processing executed by the determination unit 32, the priority determination unit 33, and the processing control unit 34 relates to vehicle control processing. On the other hand, the processing executed by the execution information generation unit 31 generates information used in the priority setting processing executed by the server 3.

[0032] When any one of a plurality of processes related to the automatic driving or driving support of the vehicle 2 is executed, the execution information generation unit 31 generates execution information regarding the executed process. For example, when the driver turns on the ACC by operating a switch (not shown) provided in the vehicle interior, the execution information generation unit 31 generates execution information representing the ACC as the executed process. Also, during the application of the automatic driving control or driving support of the vehicle 2, when receiving a notification indicating that a predetermined process has been executed from the process control unit 34, the execution information generation unit 31 generates execution information representing the process.

[0033] When any one of the processes to be the generation target of the execution information is executed, the execution information generation unit 31 refers to the current position of the vehicle 2 represented by the latest positioning information and the high-precision map to identify the road section where the vehicle 2 is located. Also, the execution information generation unit 31 refers to the latest traffic information and the current position of the vehicle 2 to determine, as representing the situation around the vehicle 2, for example, the presence or absence of construction work, the presence or absence of lane restrictions, or the presence or absence of an accident. Further, the execution information generation unit 31 refers to the latest weather information and the current position of the vehicle to identify the weather around the vehicle 2 as representing the situation around the vehicle 2. Then, the execution information generation unit 31 includes in the execution information information representing the executed process, such as the identification number of the executed process, information representing the road section on which the vehicle 2 was traveling when the process was executed, and information representing the situation around the vehicle 2 when the process was executed. Thereby, the execution information is generated.

[0034] Furthermore, the execution information generation unit 31 may specify the situation around the vehicle 2 from the images obtained by the camera 11. For example, the execution information generation unit 31 may detect other vehicles (hereinafter referred to as surrounding vehicles for convenience) traveling around the vehicle 2 from each of a series of time-series images obtained by the camera 11, and based on the detected surrounding vehicles, determine whether the area around the vehicle 2 is congested. In this case, the execution information generation unit 31 detects the surrounding vehicles by inputting each of the series of images into a discriminator that has been pre-trained to detect surrounding vehicles. As such a discriminator, for example, the execution information generation unit 31 can use a so-called deep neural network (DNN) having a convolutional neural network (CNN) type architecture. Alternatively, as such a discriminator, a DNN having a Self attention network type architecture, or a discriminator based on a machine learning method other than DNN such as a support vector machine may be used. These discriminators are pre-trained according to a predetermined learning method such as the error backpropagation method using a large number of teacher images representing the surrounding vehicles to be detected. Then, the execution information generation unit 31 tracks each individual surrounding vehicle by applying a predetermined tracking method such as KLT tracking to the area representing the surrounding vehicle detected from each image. Furthermore, the execution information generation unit 31 estimates the speed of each individual surrounding vehicle being tracked in the most recent predetermined period, and if the average speed of each individual surrounding vehicle is equal to or lower than a predetermined speed threshold, it may be determined that the area around the vehicle 2 is congested. Note that the execution information generation unit 31 can estimate the speed of the surrounding vehicle of interest based on the speed of the vehicle 2 at the time of generating each image and the distance between the vehicle 2 and the surrounding vehicle of interest at the time of generating each image. Here, it is presumed that the position of the lower end of the area representing the surrounding vehicle of interest on the image represents the azimuth from the camera 11 to the position where the surrounding vehicle touches the road surface. Also, the installation height of the camera 11 is known. Therefore, the execution information generation unit 31 can estimate the distance between the vehicle 2 and the surrounding vehicle of interest based on the azimuth from the camera 11 corresponding to the position of the lower end of the area representing the surrounding vehicle of interest on the image and the installation height of the camera 11.Also, when the vehicle 2 is equipped with a distance sensor, the execution information generation unit 31 may estimate the distance measured by the distance sensor in the direction corresponding to the area where the surrounding vehicle of interest is represented in the image as the distance between the vehicle 2 and the surrounding vehicle of interest. Further, the execution information generation unit 31 may acquire the measured value of the speed of the vehicle 2 at the time of generating each image from a vehicle speed sensor (not shown) mounted on the vehicle 2.

[0035] In addition, when the execution information generation unit 31 can detect a structure such as a sign indicating that construction is being carried out by inputting the image obtained by the camera 11 into the above-described identifier, the execution information generation unit 31 may determine that construction is being carried out around the vehicle 2. Similarly, when the execution information generation unit 31 can detect a structure indicating that lane regulation is being carried out by inputting the image obtained by the camera 11 into the above-described identifier, the execution information generation unit 31 may determine that lane regulation is being carried out around the vehicle 2.

[0036] In this way, the execution information generation unit 31 may include, in the execution information, information representing the situation around the vehicle 2 specified based on the image obtained by the camera 11.

[0037] The execution information generation unit 31 transmits the generated execution information to the server 3 via the communication interface 21 and the wireless communication terminal 13.

[0038] The determination unit 32 determines the road section where the vehicle 2 is located or the situation around the vehicle 2 according to the image generated by the camera 11 or the current position of the vehicle 2 represented by the latest positioning information at a predetermined cycle.

[0039] For example, the determination unit 32 determines the road section where the vehicle 2 is located by referring to the current position of the vehicle 2 represented by the latest positioning information and the high-precision map, in the same manner as the execution information generation unit 31. In addition, the determination unit 32 may determine the presence or absence of construction, the presence or absence of lane regulation, the presence or absence of an accident, or the weather around the vehicle 2 as representing the situation around the vehicle 2 by referring to the latest traffic information or the latest weather information and the current position of the vehicle 2.

[0040] Furthermore, the determination unit 32 may determine the situation around the vehicle 2 based on the image obtained by the camera 11. In this case, the determination unit 32 may determine the situation around the vehicle 2 by performing the same processing as the processing related to the determination of the situation around the vehicle 2 in the execution information generation unit 31 on the image obtained by the camera 11.

[0041] The determination unit 32 notifies the priority determination unit 33 of the road section where the vehicle 2 is located and the situation around the vehicle 2.

[0042] Note that, as will be described later, when the priority determination unit 33 determines the priority of each of a plurality of processes based on either the road section where the vehicle 2 is located or the situation around the vehicle 2, the determination unit 32 may also determine only one of them. In this case, the determination unit 32 may notify the priority determination unit 33 of only one of the road section where the vehicle 2 is located and the situation around the vehicle 2.

[0043] Each time the priority determination unit 33 is notified of the road section where the vehicle 2 is located and the situation around the vehicle 2 from the determination unit 32, the priority determination unit 33 refers to the priority table read from the memory 22. Then, the priority determination unit 33 determines the priority of each of a plurality of processes according to the road section where the vehicle 2 is located or the situation around the vehicle 2.

[0044] The priority table defines the priority of each of a plurality of processes for each combination of individual road sections represented by the high-precision map and the situations that can occur around the vehicle 2. Therefore, the priority determination unit 33 may determine the priority of each process corresponding to the combination of the road section where the vehicle 2 is located and the situation around the vehicle 2 notified from the determination unit 32 by referring to the priority table.

[0045] Note that the priority determination unit 33 may determine the priority of each process based on either the road section where the vehicle 2 is located or the situation around the vehicle 2. In this case, the priority table may define the priority of each of the plurality of processes for each individual road section represented by the high-precision map or for each situation that can occur around the vehicle 2.

[0046] FIG. 5 is a diagram for explaining an overview of priority determination of a plurality of processes related to the automatic driving or driving support of the vehicle 2. In this example, it is assumed that the priority of each process is determined based on the road section where the vehicle 2 is located. Also, in this example, the plurality of processes include a process related to lane change, a process related to speed control, and a process related to information collection around the vehicle.

[0047] In the area 500 shown in FIG. 5, in the road section 501, in the priority table, the priority of the process related to speed control is set highest, the priority of the process related to lane change is set second highest, and the priority of the process related to information collection around the vehicle is set lowest. Therefore, when the vehicle 2 is traveling on the road section 501, the priority is determined in the order of the process related to speed control → the process related to lane change → the process related to information collection around the vehicle.

[0048] Also, in the road section 502, in the priority table, the priority of the process related to lane change is set highest, the priority of the process related to speed control is set second highest, and the priority of the process related to information collection around the vehicle is set lowest. Therefore, when the vehicle 2 is traveling on the road section 502, the priority is determined in the order of the process related to lane change → the process related to speed control → the process related to information collection around the vehicle.

[0049] Furthermore, in road sections other than road section 501 and road section 502, in the priority table, the priority of the process related to information collection around the vehicle is set to be the highest. And the priority of the process related to lane change is set to be the next highest, and the priority of the process related to speed control is set to be the lowest. Therefore, when vehicle 2 is traveling on a road section other than road section 501 and road section 502, the priorities are determined in the order of the process related to information collection around the vehicle → the process related to lane change → the process related to speed control.

[0050] Each time the priority determination unit 33 determines the priority of each process, it notifies the determined priority of each process to the process control unit 34.

[0051] The process control unit 34 is an example of a control unit. By any one of the CPUs included in the processor 24, it manages the execution of a plurality of processes and executes the allocation of the shared memory 241 to each process and the allocation of the arithmetic circuit. And the process control unit 34 sequentially executes each of the plurality of processes using the shared memory 241 in descending order of the priority of that process. Here, the execution of a process means that the process is tried, that is, it is sufficient that a process for determining whether or not each part of vehicle 2 should be operated according to the process is executed, and it does not necessarily have to be reflected in the behavior of each part of vehicle 2.

[0052] For example, it is assumed that the process control unit 34 executes the process related to lane change, the process related to driver state determination, and the process related to information collection around vehicle 2 in order from the highest priority.

[0053] In this case, in order to execute the process related to lane change first, the processing control unit 34 reads a series of time-series images obtained by the camera 11 in the most recent predetermined period from the buffer memory 23 and writes them into the shared memory 241. Then, the processing control unit 34 detects other vehicles (hereinafter referred to as preceding vehicles) traveling in front of the vehicle 2 in the lane in which the vehicle 2 is traveling and other vehicles (hereinafter referred to as parallel traveling vehicles) traveling in the adjacent lane in the series of images by a specific arithmetic circuit of the processor 24 such as a GPU. Further, the processing control unit 34 estimates the speeds of the preceding vehicle and the parallel traveling vehicle by tracking the preceding vehicle and the parallel traveling vehicle detected over a series of images. Further, the processing control unit 34 predicts the future position of the parallel traveling vehicle assuming that the parallel traveling vehicle travels at the estimated speed. Note that the processing control unit 34 may track the preceding vehicle and the parallel traveling vehicle and estimate the speeds of the preceding vehicle and the parallel traveling vehicle by executing the same process as the process related to the tracking of surrounding vehicles in the driving information generation unit 31. Further, by pre-training the identifier used for detecting other vehicles represented in the image to also detect the lane dividing line, the processing control unit 34 can also detect the lane dividing line from each of the series of images. The processing control unit 34 specifies the area representing the own lane and the area representing the adjacent lane in each image based on the detected lane dividing line. Then, the processing control unit 34 may regard other vehicles detected on the area representing the own lane as preceding vehicles and other vehicles detected on the area representing the adjacent lane as parallel traveling vehicles. When the speed of the preceding vehicle is lower than a predetermined speed threshold and there is a space where the vehicle 2 can enter the adjacent lane, the processing control unit 34 proposes to the driver to execute a lane change via the user interface. When approval for the execution of the lane change is obtained via the user interface, the processing control unit 34 controls each part of the vehicle 2 to execute the lane change of the vehicle 2 using any one of the CPUs of the processor 24. The predetermined speed threshold is set to, for example, a speed set for the vehicle 2 or a speed lower by a predetermined offset value from the legal speed of the road section in which the vehicle 2 is traveling. Note that when the speed of the preceding vehicle is equal to or higher than the predetermined speed threshold or there is not enough space for the vehicle 2 to enter the adjacent lane, the processing control unit 34 does not execute the lane change of the vehicle 2.

[0054] When the process related to the lane change ends, the process control unit 34 reads, from the buffer memory 23, a series of time-series driver monitor images obtained by a driver monitor camera (not shown) in the most recent predetermined period in order to execute the process related to the determination of the driver's state. Then, the process control unit 34 writes the series of driver monitor images to the shared memory 241.

[0055] The processing control unit 34 determines the driver's state by detecting the orientation of the driver's face and the like represented in a series of driver monitor images using a specific arithmetic circuit of the processor 24 such as a GPU. For example, the processing control unit 34 inputs each of the series of driver monitor images into an identifier pre-trained for face detection, thereby detecting a face region in which the driver's face is represented from each driver monitor image. As such an identifier, for example, a DNN having a CNN type or DNN type architecture, or an AdaBoost identifier is used. Further, the processing control unit 34 applies a corner detection filter to the face region of each driver monitor image, or applies template matching to detect a plurality of feature points on the driver's face. Then, for each driver monitor image, the processing control unit 34 changes the orientation of the three-dimensional model of the face in various ways and fits each detected feature point to the three-dimensional model to specify the orientation of the three-dimensional model to which each feature point fits best. The processing control unit 34 detects the specified orientation of the three-dimensional model as the orientation of the driver's face. The processing control unit 34 determines whether the driver is looking away from the front of the vehicle 2 based on the orientation of the driver's face detected from each driver monitor image. For example, the processing control unit 34 determines that the driver is looking away if the period during which the detected face orientation deviates from the allowable range of face orientations corresponding to the front of the vehicle 2 continues for a certain time or more. Further, the processing control unit 34 determines that some abnormality has occurred to the driver if the period during which the detected face orientation is included in the range of face orientations corresponding to looking down continues for a certain time or more. Furthermore, the processing control unit 34 applies template matching to the face region of each driver monitor image to detect the region in which the driver's eyes are represented, and obtains the temporal change in the aspect ratio of the region in which the eyes are represented. Then, the processing control unit 34 may estimate the level of the driver's wakefulness based on the temporal change in the aspect ratio of the region in which the eyes are represented.

[0056] The processing control unit 34 executes processes such as warning the driver via a user interface (not shown) or emergency stop control of the vehicle 2 according to the determination results such as looking away and the level of wakefulness.

[0057] When the process related to the state determination of the driver ends, the process control unit 34 reads a series of time-series images obtained by the camera 11 in the most recent predetermined period from the buffer memory 23 and writes them into the shared memory 241 in order to execute the process related to the information collection around the vehicle 2. Then, the process control unit 34 detects predetermined ground objects (for example, road markings such as lane dividing lines or stop lines, curbs, and predetermined signboards such as road signs) existing around the vehicle 2 from the series of images by a specific arithmetic circuit of the processor 24 such as a GPU. For this purpose, the process control unit 34 inputs each of the series of images into an identifier that has been pre-learned to detect predetermined ground objects, thereby detecting the predetermined ground objects from each image. The process control unit 34 can use a DNN having a CNN type or DNN type architecture as such an identifier. Further, the process control unit 34 estimates the positions of the detected individual ground objects according to the method of Structure from Motion (SFM). At this time, the process control unit 34 may use the position and orientation of the vehicle 2 at the time of generating each image, the installation height, orientation, focal length of the camera 11, and the regions represented by the detected individual ground objects on each image, which are estimated from the positioning information and the odometry information, for the estimation of the positions of the individual ground objects. Then, the process control unit 34 generates probe data representing the types and positions of the detected individual ground objects, and transmits the generated probe data to other devices via the wireless communication terminal 13.

[0058] In this way, the processing control unit 34 executes a plurality of processes in order from the highest priority using the shared memory 241. For processes among the plurality of processes that do not use the shared memory 241, the processing control unit 34 may execute them in parallel with other processes regardless of the priority of those processes. And when there is a process actually reflected in the behavior of the vehicle 2, the processing control unit 34 notifies the execution information generation unit 31 that the process has been executed. For example, when the vehicle 2 actually changes lanes by executing a process related to lane change, the processing control unit 34 notifies the execution information generation unit 31 that the process related to lane change has been executed. On the other hand, when the vehicle 2 does not actually change lanes even though a process related to lane change is executed, the processing control unit 34 does not notify the execution information generation unit 31 about the execution of the process related to lane change.

[0059] FIG. 6 is an operation flowchart of vehicle control processing executed by the processor 24. The processor 24 may execute resource management processing according to the following operation flowchart at a predetermined cycle.

[0060] The determination unit 32 of the processor 24 determines the road section where the vehicle 2 is located or the situation around the vehicle 2 according to the current position of the vehicle 2 represented by the image generated by the camera 11 or the latest positioning information (step S101). Next, the priority determination unit 33 of the processor 24 refers to the priority table and determines the priority of each of a plurality of processes related to the automatic driving or driving support of the vehicle 2 according to the road section where the vehicle 2 is located or the situation around the vehicle 2 (step S102). Then, the processing control unit 34 of the processor 24 sequentially executes each of the plurality of processes using the shared memory 241 from the one with the higher priority of that process (step S103). After that, the processor 24 ends the vehicle control processing.

[0061] As described above, this vehicle control device refers to a priority table that represents the priority of each of a plurality of processes related to automatic driving or driving support of the vehicle for each road section or for each possible situation around the vehicle. Then, this vehicle control device determines the priority of each process according to the road section where the vehicle is located or the situation around the vehicle, and sequentially executes using the shared resource starting from the process with the highest priority. Therefore, this vehicle control device can execute without delay the process with the highest priority in the road section where the vehicle is located or the situation around the vehicle.

[0062] Note that the shared resource is not limited to shared memory. For example, when each of a plurality of processes is executed using a specific arithmetic circuit (e.g., GPU) that the processor 24 has, that specific arithmetic circuit also becomes an example of a shared resource.

[0063] Next, the server 3, which is an example of a priority setting device, will be described.

[0064] FIG. 7 is a hardware configuration diagram of the server 3. The server 3 includes a communication interface 41, a storage device 42, a memory 43, and a processor 44. The communication interface 41, the storage device 42, and the memory 43 are connected to the processor 44 via signal lines. The server 3 may further include an input device such as a keyboard and a mouse, and a display device such as a liquid crystal display.

[0065] The communication interface 41 is an example of a communication unit and has an interface circuit for connecting the server 3 to the communication network 4. And the communication interface 41 is configured to be communicable with the vehicle 2 via the communication network 4 and the radio base station 5. That is, the communication interface 41 passes the execution information received from the vehicle 2 via the radio base station 5 and the communication network 4 to the processor 44. Also, the communication interface 41 transmits the priority table received from the processor 44 to the vehicle 2 via the communication network 4 and the radio base station 5.

[0066] The storage device 42 is an example of a storage unit, and has, for example, a hard disk device or an optical recording medium and its access device. The storage device 42 stores various data and information used in the priority setting process. For example, the storage device 42 stores map information used to identify individual road sections, and individual execution information received from the vehicle 2 and identification information of the vehicle 2. Further, the storage device 42 may store a computer program for executing the priority setting process, which is executed on the processor 44.

[0067] The memory 43 is another example of a storage unit, and has, for example, a non-volatile semiconductor memory and a volatile semiconductor memory. The memory 43 temporarily stores various data generated during the execution of the priority setting process.

[0068] The processor 44 has one or more CPUs (Central Processing Units) and its peripheral circuits. The processor 44 may further have other arithmetic circuits such as a logical arithmetic unit or a numerical arithmetic unit. And the processor 44 executes the priority setting process.

[0069] FIG. 8 is a functional block diagram of the processor 44 related to the priority setting process. The processor 44 has a reception processing unit 51, a priority setting unit 52, and a notification processing unit 53. Each of these units included in the processor 44 is a functional module realized by, for example, a computer program operating on the processor 44. Alternatively, each of these units included in the processor 44 may be a dedicated arithmetic circuit provided in the processor 44.

[0070] Each time the reception processing unit 51 receives the execution information from the vehicle 2 via the communication network 4 and the communication interface 41, it stores the received execution information in the storage device 42. At this time, the reception processing unit 51 increments by 1 the count value representing the number of pieces of execution information for the combination of the road section where the vehicle 2 was located and the surrounding situation of the vehicle 2 when the execution information was generated, which is represented in the execution information. When the priority table is created to represent the priorities of a plurality of processes for each road section, the reception processing unit 51 may increment by 1 the count value representing the number of pieces of execution information for the road section where the vehicle 2 was located when the execution information was generated. Also, when the priority table is created to represent the priorities of a plurality of processes for each possible situation around the vehicle 2, the reception processing unit 51 may increment by 1 the count value representing the number of pieces of execution information for the situation around the vehicle 2 when the execution information was generated.

[0071] The priority setting unit 52 sets the priorities of a plurality of processes for the combination of the road section and the surrounding situation of the vehicle where the count value representing the number of received execution information is equal to or greater than a predetermined number. To do so, the priority setting unit 52 reads from the storage device 42 each piece of execution information corresponding to the combination of the road section and the surrounding situation of the vehicle where the count value is equal to or greater than the predetermined number. Then, the priority setting unit 52 refers to the processes executed in the vehicle 2 represented in the read individual execution information and counts the number of times each process has been executed in the vehicle 2. The priority setting unit 52 sets the priority of each process so that the process that has been executed more times in the vehicle 2 has a higher priority. For example, assume that the process related to lane change has been executed 100 times, the process related to speed control has been executed 120 times, and the process related to driver state determination has been executed 80 times. In this case, the priority setting unit 52 sets the highest priority for the process related to speed control, the second highest priority for the process related to lane change, and the lowest priority for the process related to driver state determination.

[0072] In addition, when the priority table is created to represent the priorities of a plurality of processes for each road section, the priority setting unit 52 may set the priority of each of the plurality of processes for a road section where the count value representing the number of received execution information is equal to or greater than a predetermined number. For this purpose, the priority setting unit 52 may read each execution information corresponding to a road section where the count value is equal to or greater than a predetermined number from the storage device 42, and execute the same process as the above process for each of the read individual execution information. Further, when the priority table is created to represent the priorities of a plurality of processes for each possible situation around the vehicle 2, the priority setting unit 52 may set the priority of each process for the situation around the vehicle 2 where the count value representing the number of received execution information is equal to or greater than a predetermined number. For this purpose, the priority setting unit 52 may read each execution information corresponding to the situation around the vehicle 2 where the count value is equal to or greater than a predetermined number from the storage device 42, and execute the same process as the above process for each of the read individual execution information.

[0073] The priority setting unit 52 updates the priority table so as to reflect the set priority of each process, saves the updated priority table in the storage device 42, and passes it to the notification processing unit 53.

[0074] When receiving the updated priority table, the notification processing unit 53 generates distribution information including the received priority table. Then, the notification processing unit 53 refers to the identification information of the vehicle 2, and transmits the generated distribution information to the vehicle 2 via the communication interface 41 and the communication network 4. As a result, the vehicle 2 can use the updated priority table.

[0075] FIG. 9 is an operation flowchart of the priority setting process. When the reception processing unit 51 of the processor 44 receives execution information from the vehicle 2, it saves the received execution information in the storage device 42. Further, the reception processing unit 51 increments by 1 the count value representing the number of received execution information for the combination of the road section corresponding to the execution information and the situation around the vehicle 2 (step S201).

[0076] The priority setting unit 52 of the processor 44 determines whether the count value for any combination of the road section and the possible surrounding situations of the vehicle 2 has reached a predetermined number or more (step S202). If the count value is less than the predetermined number for any combination (step S202 - No), the processor 44 ends the priority setting process. On the other hand, if the count value is equal to or more than the predetermined number for any combination (step S202 - Yes), the priority setting unit 52 reads the respective execution information corresponding to that combination from the storage device 42. Then, based on the read individual execution information, the priority setting unit 52 updates the priority table by setting the priority of each of the plurality of processes so that the process with a higher number of executions has a higher priority (step S203). After that, the notification processing unit 53 of the processor 44 transmits the updated priority table to the vehicle 2 via the communication interface 41 and the communication network 4 (step S204). Then, the processor 44 ends the priority setting process.

[0077] As described above, this priority setting device sets the priority of each of the plurality of processes related to the automatic driving or driving support of the vehicle so that the process with a higher number of executions has a higher priority for each road section or each possible surrounding situation of the vehicle. Therefore, this priority setting device can appropriately set the priority for each process for each road section or each possible surrounding situation of the vehicle.

[0078] Note that the priority of each process for each road section or for each possible situation around the vehicle may be set in advance. For example, the priority of each process may be set based on the vehicle safety or driver convenience for each road section or for each possible situation around the vehicle. In particular, the priority of the process related to vehicle safety may be set to be higher than the priority of other processes regardless of the road section and the possible situation around the vehicle. Then, a priority table may be created according to the set priority of each process for each road section or for each possible situation around the vehicle. In this case, the priority table may be stored in the storage device 14 of the vehicle 2 together with the high-precision map at the time of factory shipment, for example. Also in this case, since the priority setting process may not be executed, the process of the execution information generation unit 31 in the server 3 and the ECU 15 of the vehicle 2 may be omitted.

[0079] As described above, those skilled in the art can make various changes according to the implemented form within the scope of the present invention.

Explanation of Signs

[0080] 1 Vehicle control system 2 Vehicle 11 Camera 12 GPS receiver 13 Wireless communication terminal 14 Storage device 15 ECU 21 Communication interface 22 Memory 23 Buffer memory 24 Processor 241 Shared memory 31 Execution information generation unit 32 Determination unit 33 Priority determination unit 34 Process control unit 3 Server 41 Communication interface 42 Storage device 43 Memory 44 Processor 51 Reception processing unit 52 Priority setting unit 53 Notification processing unit 4 Communication network 5 Radio base station

Claims

1. A storage unit that stores a priority table indicating the priority of the execution order of each of a plurality of processes related to automatic driving or driving support of a vehicle for each individual road section represented in map information, where the plurality of processes include at least two of a first process for controlling the vehicle for lane change, a second process for collecting information about predetermined features around the vehicle based on an image obtained by a camera mounted on the vehicle, and a third process for determining the state of the driver of the vehicle. A determination unit that determines the road section in which the vehicle is located according to the position of the vehicle. A priority determination unit that refers to the priority table and determines the priority of each of the plurality of processes according to the road section in which the vehicle is located. A control unit that allocates shared resources to each of the plurality of processes in order from the process with the higher priority and executes the processes. A vehicle control device having the above components.

2. Determine the road section in which the vehicle is located among the individual road sections represented in the map information according to the position of the vehicle. Refer to a priority table indicating the priority of the execution order of each of a plurality of processes related to automatic driving or driving support of the vehicle for each individual road section represented in the map information, and determine the priority of each of the plurality of processes according to the road section in which the vehicle is located. The plurality of processes include at least two of a first process for controlling the vehicle for lane change, a second process for collecting information about predetermined features around the vehicle based on an image obtained by a camera mounted on the vehicle, and a third process for determining the state of the driver of the vehicle. Allocate shared resources to each of the plurality of processes in order from the process with the higher priority and execute the processes. A vehicle control method including the above steps.

3. Determine the road section in which the vehicle is located among the individual road sections represented in the map information according to the position of the vehicle. For each individual road section represented in the map information, referring to a priority table indicating the priority for the execution order of each of a plurality of processes related to the automatic driving or driving support of the vehicle, determine the priority of each of the plurality of processes according to the road section where the vehicle is located. The plurality of processes includes at least two of a first process for controlling the vehicle for lane change, a second process for collecting information about predetermined features around the vehicle based on an image obtained by a camera mounted on the vehicle, and a third process for determining the state of the driver of the vehicle. For each of the plurality of processes, allocate shared resources to the process in order from the one with the higher priority of the process and execute it. A vehicle control computer program for causing a processor mounted on the vehicle to execute this.

4. At least one vehicle, A priority setting device capable of communicating with each of the at least one vehicle, having Each of the at least one vehicle stores a priority table indicating the priority for the execution order of each of a plurality of processes related to the automatic driving or driving support of the vehicle for each individual road section represented in the map information. The plurality of processes includes at least two of a first process for controlling the vehicle for lane change, a second process for collecting information about predetermined features around the vehicle based on an image obtained by a camera mounted on the vehicle, and a third process for determining the state of the driver of the vehicle, a storage unit; a determination unit that determines the road section where the vehicle is located according to the position of the vehicle; a priority determination unit that refers to the priority table and determines the priority of each of the plurality of processes according to the road section where the vehicle is located; a control unit that allocates shared resources to each of the plurality of processes in order from the one with the higher priority of the process and executes it; Among the plurality of processes, generates execution information representing the process whose execution is reflected in the actual behavior of the vehicle and the road section where the vehicle is located when the process is executed, and transmits the generated execution information to the priority setting device via a communication unit, an execution information generation unit; having The priority setting device a storage unit; a reception processing unit that, each time receiving the execution information from any one of the at least one vehicle via the communication unit, stores the execution information in the storage unit of the priority setting device. Among the individual road sections represented in the map information, for the road sections where the number of the execution information stored in the storage unit of the priority setting device has reached a predetermined number, based on the predetermined number of the execution information, among the plurality of processes, the process that has been executed and the number of times reflected in the actual behavior of the vehicle is larger has a higher priority, and the priority table is updated by setting the priority of each of the plurality of processes; a priority setting unit a notification processing unit that notifies each of the at least one vehicle of the updated priority table via the communication unit A vehicle control system having

Images