Route generation device, route generation method, and computer program for route generation

Abstract

The present invention provides a route generation device capable of generating an appropriate planned driving route based on the conditions surrounding the vehicle. [Solution] The route generation device has a route generation unit 31 that generates a planned route by inputting destination information representing the destination of the vehicle 10 and external sensor information representing the surrounding conditions of the vehicle 10 generated by an external sensor 2 installed on the vehicle 10 into a route generation model 400 that has been pre-learned to generate a planned route for the vehicle 10 to travel. The route generation model 400 has a destination information block 401 into which destination information is input, an external sensor information block 402 into which external sensor information is input, a coupling block 403 having a cross attention mechanism that performs cross attention calculations on the output from the destination information block and the output from the external sensor information block, and an output block 404 that generates a planned route based on the output from the coupling block.

Description

Technical Field

[0004] ,

[0006] , , , , , ,

[0005] , , , , ,

[0003] , , , , ,

[0001] The present invention relates to a route generation device, a route generation method, and a computer program for route generation that generate a driving route planned for a vehicle to travel.

Background Art

[0002] Techniques for setting a route planned for a host vehicle to travel or predicting a route for another vehicle to travel using a learning model based on deep learning have been studied (see Patent Documents 1 and 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is desired to more appropriately generate a route planned for a vehicle to travel (hereinafter simply referred to as a driving planned route) according to the situation around the vehicle.

[0005] Therefore, an object of the present invention is to provide a route generation device capable of generating an appropriate driving planned route according to the situation around the host vehicle.

Means for Solving the Problems

[0007] In one embodiment, destination information includes the position coordinates of the destination of the vehicle, the straight-line distance from the vehicle's current position to the destination, and the direction.

[0008] In one embodiment, the destination information further includes the position coordinates of one or more checkpoints located on the route from the vehicle's current position to the destination, and the upper speed limit at each of the one or more checkpoints.

[0009] Another embodiment provides a route generation method. This route generation method includes generating a planned route by inputting destination information representing the vehicle's destination and external sensor information representing the surrounding conditions of the vehicle, generated by external sensors installed on the vehicle, into a route generation model that has been pre-trained to generate a planned route on which the vehicle is scheduled to travel. The route generation model includes a destination information block into which destination information is input, an external sensor information block into which external sensor information is input, a coupling block having a cross attention mechanism that performs cross attention operations on the output from the destination information block and the output from the external sensor information block, and an output block that generates a planned route based on the output from the coupling block.

[0010] In yet another embodiment, a computer program for route generation is provided. This computer program for route generation includes instructions for a processor installed in the vehicle to generate a planned route by inputting destination information representing the vehicle's destination and external sensor information representing the surrounding conditions of the vehicle, generated by external sensors installed in the vehicle, into a route generation model that has been pre-trained to generate a planned route for the vehicle to travel. The route generation model includes a destination information block into which destination information is input, an external sensor information block into which external sensor information is input, a coupling block having a cross attention mechanism that performs cross attention operations on the output from the destination information block and the output from the external sensor information block, and an output block that generates a planned route based on the output from the coupling block. [Effects of the Invention]

[0011] The route generation device described herein has the effect of generating an appropriate planned route that takes into account the conditions surrounding the vehicle. [Brief explanation of the drawing]

[0012] [Figure 1] This is a schematic diagram of the vehicle control system in which a route generation device is implemented. [Figure 2] This is a hardware configuration diagram of an electronic control unit, which is one embodiment of a route generation device. [Figure 3] This is a functional block diagram of the processor of an electronic control unit related to vehicle control processing, including route generation processing. [Figure 4] This diagram illustrates the overview of the path generation model. [Figure 5] This is an operation flowchart of the vehicle control process, including the route generation process. [Modes for carrying out the invention]

[0013] The following describes the route generation device, the route generation method executed on the route generation device, and the computer program for route generation, with reference to the diagrams. This route generation device generates a planned route for the vehicle by inputting information representing the vehicle's destination (hereinafter referred to as destination information) and external sensor information representing the surrounding conditions of the vehicle into a route generation model having a cross-attention mechanism.

[0014] Figure 1 is a schematic diagram of a vehicle control system on which a route generation device is implemented. Figure 2 is a hardware diagram of an electronic control unit, which is one embodiment of the route generation device. In this embodiment, the vehicle control system 1, which is mounted on and controls a vehicle 10, an example of a vehicle, includes at least one external sensor 2, a GPS receiver 3, a navigation device 4, and an electronic control unit (ECU) 5, which is an example of a route generation device. The individual external sensors 2, GPS receiver 3, navigation device 4, and ECU 5 are connected to each other via an in-vehicle network conforming to a predetermined in-vehicle communication standard. The vehicle control system 1 may further include a wireless communication terminal (not shown) for wireless communication with devices outside the vehicle 10. The vehicle control system 1 may also further include a storage device (not shown) for storing map information.

[0015] Each external sensor 2 is a sensor for detecting the surrounding conditions of the vehicle 10, and includes, for example, an external camera installed to photograph the area around the vehicle 10, or a distance measuring sensor such as radar or LiDAR that measures the distance to objects around the vehicle 10. For example, an external camera, which is an example of an external sensor 2, is mounted facing forward of the vehicle 10 to photograph a predetermined area around the vehicle 10 (for example, the area in front of the vehicle 10) and generate an image representing that predetermined area. The image generated by the external camera is an example of external sensor information. Similarly, a distance measuring sensor, which is another example of an external sensor 2, is mounted on the vehicle 10 to measure the distance to objects in each direction included in a predetermined detection range around the vehicle 10. The distance measuring sensor then generates a distance measuring signal that represents the distance to objects in each direction within the detection range. The distance measuring signal is another example of external sensor information. The vehicle 10 may be equipped with multiple external cameras, each of which is an external sensor 2, with different shooting directions or field of view. Similarly, the vehicle 10 may be provided with multiple distance measuring sensors, each with a different detection range, as individual external sensors 2. Furthermore, each external sensor 2 may be provided with one or more external cameras and one or more distance measuring sensors.

[0016] Each external sensor 2 generates external sensor information at predetermined intervals and outputs the generated external sensor information to the ECU 5 via the in-vehicle network.

[0017] The GPS receiver 3 receives GPS signals from GPS satellites at predetermined intervals and determines the vehicle 10's own position based on the received GPS signals. The GPS receiver 3 then generates positioning information representing the vehicle 10's own position and outputs the generated positioning information to the navigation device 4 and ECU 5 via the in-vehicle network. The vehicle 10 may also have a receiver that determines its own position by receiving positioning signals from satellites of another satellite positioning system instead of a GPS receiver.

[0018] The navigation device 4 searches for a route from the current position of the vehicle 10 to the destination according to an operation by the occupant of the vehicle 10, according to a predetermined route search method such as Dijkstra's algorithm, and generates route information representing the route to the destination. Then, the navigation device 4 outputs the generated route information to the ECU 5 via the in-vehicle network. This route information includes the position coordinates of the destination and information representing each road section on the route.

[0019] The ECU 5 generates a planned travel route according to the destination information and the out-vehicle sensor information, and automatically controls the vehicle 10 so that the vehicle 10 travels along the generated planned travel route.

[0020] As shown in FIG. 2, the ECU 5 includes a communication interface 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 may each be configured as separate circuits, or may be integrally configured as one integrated circuit.

[0021] The communication interface 21 has an interface circuit for connecting the ECU 5 to each part of the vehicle 10 or the in-vehicle network. Then, the communication interface 21 passes the out-vehicle sensor information received from the individual out-vehicle sensors 2 to the processor 23. Also, the communication interface 21 passes the positioning information received from the GPS receiver 3 to the processor 23. Further, the communication interface 21 passes the route information received from the navigation device 4 to the processor 23. Further, the communication interface 21 outputs a control signal for controlling each part of the vehicle 10.

[0022] Memory 22 is an example of a storage unit and includes, for example, volatile semiconductor memory and non-volatile semiconductor memory. Memory 22 stores various data used in vehicle control processing, including route generation processing, executed by processor 23. For example, memory 22 stores external sensor information received from individual external sensors 2, vehicle positioning information received from GPS receiver 3, and route information received from navigation device 4. Furthermore, memory 22 stores various parameters that define the route generation model, for example, weight coefficients used in each layer or sublayer of the route generation model. Furthermore, memory 22 stores parameters related to individual external sensors 2, for example, the focal length, shooting direction, and mounting position of the external camera. Furthermore, memory 22 may store map information read from a storage device. The map information may include information representing the location and speed limit of individual road sections. Furthermore, memory 22 temporarily stores various data generated during vehicle control processing.

[0023] The processor 23 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 23 may further have other arithmetic circuits such as a logic unit, a numerical unit, or a graphics processing unit. The processor 23 stores the external sensor information received from each external sensor 2, the positioning information received from the GPS receiver 3, and the route information received from the navigation device 4 in the memory 22. Furthermore, the processor 23 executes vehicle control processing, including route generation processing.

[0024] Figure 3 is a functional block diagram of the processor 23 relating to vehicle control processing, including route generation processing. The processor 23 has a route generation unit 31 and a control unit 32. Each of these parts of the processor 23 is, for example, a functional module realized by a computer program running on the processor 23. Alternatively, each of these parts of the processor 23 may be a dedicated arithmetic circuit provided on the processor 23.

[0025] The route generation unit 31 generates a planned route from the vehicle 10's current position to the destination by inputting destination information and external sensor information into the route generation model. The planned route is represented as a set of position coordinates of individual points that the vehicle 10 is scheduled to pass through from its current position to the destination. The planned route may also further include at least one of the target speed and the scheduled time for passing each point. The route generation model is pre-trained to generate the planned route using a large amount of training data according to a predetermined learning method such as backpropagation.

[0026] The destination information includes the position coordinates of the destination for vehicle 10, the straight-line distance from vehicle 10's current position to the destination, and the direction. The destination included in the destination information can be the destination for vehicle 10 included in the route information received from the navigation device 4. Alternatively, the destination included in the destination information can be a point located on the route indicated in the route information, at a predetermined distance from vehicle 10's current position. In this case, the route generation unit 31 simply needs to refer to the map information to identify a road section at a predetermined distance from vehicle 10's current position on the route, and then identify the position coordinates of a point on that road section. The current position of vehicle 10 can be the position of vehicle 10 as represented by the latest positioning information. The destination information may further include the position coordinates of one or more checkpoints located on the route from vehicle 10's current position to the destination indicated in the destination information. Each checkpoint is a point that vehicle 10 should pass through, and can be set at any point on the route from vehicle 10's current position to the destination, for example, at an intersection or fork in the road, or at a point where the number of lanes increases or decreases. The route generation unit 31 can also determine the position coordinates of each checkpoint by referring to map information. The route generation unit 31 may express the coordinates of the destination and the position coordinates of each checkpoint as a combination of latitude and longitude, or as a two-dimensional or three-dimensional world coordinate system based on a predetermined position in real space (for example, the current position of the vehicle 10). In addition, the destination information may further include the speed limit at each checkpoint. In this case, the route generation unit 31 can refer to map information to identify the road section containing each checkpoint, and use the speed limit set for the identified road section as the speed limit at that checkpoint.

[0027] The route generation unit 31 can generate destination information as a vector with each of the above-mentioned values ​​as its element. The route generation unit 31 then inputs the destination information, represented as a vector, into the route generation model along with the external sensor information. By using destination information in route generation, it becomes possible to generate a planned route to the destination. Furthermore, by using external sensor information in route generation, it becomes possible to reflect the surrounding conditions of the vehicle 10 in setting the planned route, for example, generating a planned route that avoids obstacles. If the vehicle 10 is equipped with multiple external sensors 2, the route generation unit 31 inputs the external sensor information generated by one of the multiple external sensors 2 (for example, an external camera whose shooting range is the area in front of the vehicle 10, or a distance measuring sensor installed so that the area in front of the vehicle 10 is included in its detection range) into the route generation model. Alternatively, the route generation unit 31 may input the external sensor information generated by two or more external sensors into the route generation model.

[0028] Figure 4 is a diagram illustrating the outline of the route generation model. As shown in Figure 4, the route generation model 400 includes a destination information block 401 into which destination information is input, an external sensor information block 402 into which external sensor information is input, a coupling block 403 having a cross attention mechanism, and an output block 404.

[0029] The destination information block 401 includes an input layer into which destination information is input. The destination information block 401 may also include an encoding layer that encodes the input destination information according to a predetermined encoding rule. Furthermore, the destination information block 401 may have one or more intermediate layers that perform predetermined operations on the output from the preceding layer. Each intermediate layer may include, for example, an attention sublayer that performs an attention operation according to an attention mechanism, a feed-forward sublayer that performs a fully connected operation, and a normalization sublayer placed between those sublayers.

[0030] The external sensor information block 402 can have a configuration similar to that of the encoder in the Vision Transformer. That is, the external sensor information block 402 has an input layer that divides the input external sensor information into multiple patches, embeds each patch into a vector, and adds a position encoding, and an attention sublayer that performs an attention operation on the output from the input layer according to the attention mechanism. The external sensor information block 402 may also have a sublayer that performs a fully connected operation on the output from the attention sublayer, and a normalization sublayer that is placed before or after those sublayers.

[0031] The coupling block 403 receives the output from the destination information block 401 and the output from the external sensor information block 402 as inputs. The coupling block 403 has a cross attention sublayer that performs a cross attention operation according to the cross attention mechanism on the output from the destination information block 401 and the output from the external sensor information block 402. The coupling block 403 may further have a feed forward sublayer that performs a fully coupled operation on the output from the cross attention sublayer, and a normalization sublayer that is placed before or after the feed forward sublayer. In this way, by performing a cross attention operation on the output from the destination information block 401 and the output from the external sensor information block 402, the route generation model can fuse destination information and external sensor information for route generation.

[0032] The output block 404 generates a planned route based on the output from the combined block 403. To do this, the output block 404 has an output layer that performs a softmax calculation on the output from the previous stage. The output layer calculates a confidence score representing the likelihood that the vehicle 10 will pass each point in the direction perpendicular to the direction of travel (hereinafter referred to as the lateral direction) at predetermined intervals along the direction of travel of the vehicle 10 from the current position of the vehicle 10. The planned route is then formed by sequentially connecting the points with the highest confidence score at each predetermined interval from the current position of the vehicle 10 to the destination. If the planned route includes at least one of the target speed and the planned time of passage at each point, the output layer calculates a confidence score for each combination of the lateral point and at least one of the candidate points for the target speed and the planned time of passage at predetermined intervals along the direction of travel of the vehicle 10 from the current position of the vehicle 10. In this case as well, the planned route is formed by sequentially connecting the points included in the combination with the highest confidence score at each predetermined interval from the current position of the vehicle 10 to the destination. Furthermore, for each point included in the planned route, the candidate target speed or planned passing time included in the combination corresponding to that point becomes the target speed or planned passing time at that point.

[0033] Furthermore, the output block 404 may also have one or more intermediate layers that perform predetermined operations on the output from the preceding layer. Each intermediate layer may include, for example, a feed-forward sublayer that performs a fully connected operation. Each intermediate layer may also include an attention sublayer prior to the feed-forward sublayer. Moreover, each intermediate layer may include a normalization sublayer either before or after the feed-forward sublayer.

[0034] The route generation unit 31 notifies the control unit 32 of the generated planned route.

[0035] The control unit 32 controls each part of the vehicle 10 so that the vehicle 10 travels along the planned travel path received from the path generation unit 31. To this end, the control unit 32 compares the position of the vehicle 10 with the planned travel path at predetermined intervals. If the position of the vehicle 10 is on the planned travel path, the control unit 32 determines the steering angle of the vehicle 10 so that the vehicle 10 moves along the planned travel path, and controls the steering of the vehicle 10 to achieve the determined steering angle. If the position of the vehicle 10 is away from the planned travel path, the control unit 32 determines the steering angle of the vehicle 10 so that the vehicle 10 moves closer to the planned travel path, and controls the steering of the vehicle 10 to achieve the determined steering angle.

[0036] Furthermore, if the planned route includes target speeds at individual points along the route, the control unit 32 controls the vehicle's powertrain and brakes so that the vehicle's speed approaches the target speed at each point as it passes through that point. Also, if the planned route includes planned passing times at individual points along the route, the control unit 32 calculates the target speed at each point based on the distance from the vehicle's current position to each point and the planned passing time. The control unit 32 then controls the vehicle's powertrain and brakes so that the vehicle's speed approaches the target speed at each point as it passes through that point.

[0037] Figure 5 is an operation flowchart of the vehicle control process, including the route generation process, executed by the processor 23. The processor 23 should execute the vehicle control process according to the following operation flowchart.

[0038] The route generation unit 31 generates a planned route for the vehicle 10 by inputting destination information and external sensor information into the route generation model (step S101). The control unit 32 controls the vehicle 10 so that it travels along the generated planned route (step S102). Then the processor 23 terminates the vehicle control process.

[0039] As explained above, this route generation device generates a planned route for the vehicle by inputting destination information and external sensor information into a route generation model having a cross-attention mechanism. Therefore, this route generation device can reflect the surrounding conditions of the vehicle in the generated planned route, and as a result, it can generate an appropriate planned route according to the conditions around the vehicle. Furthermore, this route generation device generates a planned route without explicitly detecting obstacles on the road surface. Therefore, even when there are obstacles on the road surface that should be detected, such as fallen objects like rocks or grooves in the road surface, but are difficult to define as such, this route generation device can generate a planned route that avoids such obstacles.

[0040] The computer program that realizes the functions of the processor 23 of the ECU 5 according to the above embodiment or modification may be provided in the form of being recorded on a computer-readable portable recording medium such as semiconductor memory, magnetic recording medium, or optical recording medium.

[0041] As described above, those skilled in the art can make various modifications within the scope of the present invention to suit the implemented form. [Explanation of Symbols]

[0042] 1. Vehicle control system 10 vehicles 2. External sensors 3 GPS receivers 4. Navigation system 5. Electronic control units (ECUs, route generators) 21 Communication Interface 22 memory 23 processors 31 Path generation unit 32 Control Unit

Claims

1. A route generation unit generates a planned route by inputting destination information representing the vehicle's destination and external sensor information representing the surrounding conditions of the vehicle, generated by external sensors installed on the vehicle, into a route generation model that has been pre-learned to generate a planned route for the vehicle to travel. It has, The route generation model comprises a destination information block into which the destination information is input, an external sensor information block into which the external sensor information is input, a coupling block having a cross attention mechanism that performs a cross attention operation on the output from the destination information block and the output from the external sensor information block, and an output block that generates the planned driving route based on the output from the coupling block. Route generation device.

2. The route generation device according to claim 1, wherein the destination information includes the position coordinates of the destination of the vehicle, the straight-line distance from the current position of the vehicle to the destination, and the direction.

3. The route generation device according to claim 2, wherein the destination information further includes the position coordinates of one or more checkpoints located on the route from the current position of the vehicle to the destination, and the upper speed limit at each of the one or more checkpoints.

4. The planned route is generated by inputting destination information representing the vehicle's destination and external sensor information representing the surrounding conditions of the vehicle, generated by external sensors installed on the vehicle, into a route generation model that has been pre-trained to generate the planned route the vehicle is scheduled to travel. This includes, The route generation model comprises a destination information block into which the destination information is input, an external sensor information block into which the external sensor information is input, a coupling block having a cross attention mechanism that performs a cross attention operation on the output from the destination information block and the output from the external sensor information block, and an output block that generates the planned driving route based on the output from the coupling block. Route generation method.

5. The planned route is generated by inputting destination information representing the vehicle's destination and external sensor information representing the surrounding conditions of the vehicle, generated by external sensors installed on the vehicle, into a route generation model that has been pre-trained to generate the planned route the vehicle is scheduled to travel. The processor installed in the vehicle is made to perform this action. The route generation model comprises a destination information block into which the destination information is input, an external sensor information block into which the external sensor information is input, a coupling block having a cross attention mechanism that performs a cross attention operation on the output from the destination information block and the output from the external sensor information block, and an output block that generates the planned driving route based on the output from the coupling block. A computer program for route generation.

Images