Road structure estimation method and road structure estimation system

Abstract

To provide a road structure estimation method and road structure estimation system capable of estimating a structure concerning a direction, in which a moving body wants to advance, while the moving body is moving.SOLUTION: A road structure estimation method and road structure estimation system generate first feature data, which represents a feature of a road structure around a moving body, using road information around the moving body obtained by way of a sensor included in the moving body, and generate second feature data, which represents a feature of a route, using route information on a map which represents a route along which the moving body has advanced. The first feature data and second feature data are integrated to generate integrated data. The integrated data is used to generate a probability map of the road structure.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a road structure estimation method and a road structure estimation device. [Background technology]

[0002] A technology has been proposed in which a deep neural network (DNN) is applied to sensor data such as LiDAR (Light Detection And Ranging) to acquire lane information, generate and update a high definition map (HD map), and estimate a vehicle's own position (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2022-546397 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the technology disclosed in Patent Document 1, the optimization of the self-location and map updates is realized by using the entire road structure that can be estimated when generating and updating the HD Map. However, there is a problem in that it is difficult to estimate the structure related to the direction in which a moving body wants to proceed while the moving body is moving.

[0005] The present invention has been made in view of the above problems, and an object of the present invention is to provide a road structure estimation method and a road structure estimation device that are capable of estimating the structure related to the desired traveling direction of a moving object while the moving object is moving. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, a road structure estimation method and a road structure estimation device according to one aspect of the present invention generate first feature data indicating features of the road structure around a moving body from road information around the moving body obtained via a sensor provided on the moving body, generate second feature data indicating features of the route from route information on a map that shows the route traveled by the moving body, integrate the first feature data and the second feature data to generate integrated data, and generate a probability map of the road structure from the integrated data. Effect of the Invention

[0007] According to the present invention, it is possible to estimate a structure relating to a direction in which a moving object wants to proceed while the moving object is moving. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the configuration of a road structure estimation device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a flowchart showing the processing of the road structure estimation device according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Next, an embodiment of the present invention will be described in detail with reference to the drawings. In the description, the same parts are designated by the same reference numerals and duplicated explanations will be omitted.

[0010] [Configuration of the road structure estimation device] Fig. 1 is a block diagram showing the configuration of a road structure estimation device according to this embodiment. As shown in Fig. 1, the road structure estimation device according to this embodiment includes an acquisition unit 71, a database 73, a controller 100, and an output unit 400. The controller 100 is connected to the acquisition unit 71, the database 73, and the output unit 400 via a wired or wireless communication path. For example, the road structure estimation device may be mounted on a moving body such as a vehicle.

[0011] The acquisition unit 71 acquires road information about the surroundings of the mobile object obtained through a sensor provided in the mobile object. More specifically, the acquisition unit 71 may be connected to a sensor including at least one of a camera, an ultrasonic sensor, and a LiDAR (Laser Imaging Detection and Ranging). The acquisition unit 71 may acquire, as the road information, a captured image of the surroundings of the mobile object captured by a camera.

[0012] The acquisition unit 71 may also acquire distance measurement point data relating to distance measurement points located on objects around the mobile object or on the surface of the road surface using an ultrasonic sensor or LiDAR.

[0013] Ultrasonic sensors mainly measure the distance and direction to objects around a moving body by emitting ultrasonic waves from an emission point around the moving body and detecting the position of the reflection point based on the reflected wave of the emitted ultrasonic waves.

[0014] Ultrasonic sensors measure the distance and direction to an object and recognize the shape of the object by measuring the time it takes for the ultrasonic wave (reflected wave) to bounce back after emitting the ultrasonic wave. Furthermore, ultrasonic sensors can obtain the positional relationship of an object in three dimensions.

[0015] LiDAR mainly measures the distance and direction to objects around a moving body by emitting electromagnetic waves from an emission point around the moving body and detecting the position of the reflection point based on the reflected wave of the emitted electromagnetic wave. In particular, LiDAR is a sensor that emits light (laser light) from an emission point around a specified range around a moving body, detects the position of the reflection point, which is the ranging point, based on the reflected wave, and generates ranging point data related to the ranging point.

[0016] LiDAR measures the distance and direction to an object and recognizes the shape of the object by measuring the time it takes for the light (reflected wave) to bounce back after being emitted. LiDAR can also obtain the positional relationship of an object in three dimensions. It is also possible to map using the intensity of the reflected wave.

[0017] The road information may be information in which a label is attached to each pixel by Semantic Segmentation. Here, the "label" may be information attached to distinguish "road surface," "white lines," "traffic lights," and "objects (cars, people, etc.)" located around a moving object. The objects distinguished by the label are not limited to the examples given here.

[0018] The road information may also be converted into a bird's-eye view seen from above the moving object by image conversion.

[0019] The acquisition unit 71 also acquires route information on a map indicating the route traveled by the moving object. More specifically, the acquisition unit 71 may be connected to a navigation system equipped in the moving object. The acquisition unit 71 may acquire, as the route information, point sequence information indicating a trajectory of past movement of the moving object on the route from the navigation system. The acquisition unit 71 may also acquire, as the route information, intersection information indicating intersections on the route from the navigation system.

[0020] The database 73 stores various learning models used by the controller 100, which will be described later. The learning models may be configured by neural networks.

[0021] When the learning model is a neural network, the neural network includes an input layer (or kernel), an output layer to which output values ​​are output, and at least one hidden layer between the input layer and the output layer, and signals propagate in the order of the input layer, hidden layer, and output layer.

[0022] Each layer, including the input layer, hidden layer, and output layer, is composed of one or more units. The units between layers are connected to each other, and each unit has an activation function (e.g., a sigmoid function, a normalized linear function, a softmax function, etc.). A weighted sum is calculated based on multiple inputs to the unit, and the value of the activation function, whose variable is the sum, becomes the output of the unit. For example, in machine learning, the weights used when calculating the sum in each unit of a neural network are adjusted as parameters related to the learning model.

[0023] The database 73 may store the connections between the units in the neural network and the weights used to calculate the sum for each unit in the neural network. Hereinafter, the "connections" between the units and the "weights" are referred to as "parameters."

[0024] In addition, the database 73 may store teacher road information and teacher route information when generating a neural network by machine learning. The database 73 may store a teacher probability map linked to a set of teacher road information and teacher route information.

[0025] The output unit 400 outputs various information generated by the controller 100. For example, the output unit 400 may be configured to output an output from a learning model to the outside. In particular, the output unit 400 may be configured to output a probability map of a road structure generated by the controller 100. For example, the probability map of a road structure may be output to a route setting device that sets a travel route of a moving body, and may be used for setting the travel route of the moving body.

[0026] The controller 100 (an example of a control unit or a processing unit) is a general-purpose computer equipped with a CPU (Central Processing Unit), a memory, and an input / output unit. A computer program (road structure estimation program) for causing the controller 100 to function as a part of the road structure estimation device is installed in the controller 100. By executing the computer program, the controller 100 functions as a plurality of information processing circuits (110, 120, 130, 140, 150) equipped in the road structure estimation device.

[0027] Here, an example is shown in which the multiple information processing circuits (110, 120, 130, 140, 150) of the road structure estimation device are realized by software. However, it is also possible to configure the information processing circuits (110, 120, 130, 140, 150) by preparing dedicated hardware for executing each information process shown below. Also, the multiple information processing circuits (110, 120, 130, 140, 150) may be configured by individual hardware.

[0028] The controller 100 includes a first feature data generation unit 110, a second feature data generation unit 120, a probability map generation unit 130, and an integrated data generation unit 140 as a plurality of information processing circuits (110, 120, 130, 140, 150). The controller 100 may further include a partial data generation unit 150.

[0029] The first feature data generating unit 110 generates first feature data indicating features of the road structure around the mobile object from the road information. More specifically, the first feature data generating unit 110 may generate the first feature data from the road information via a neural network. A method for configuring the neural network used in the first feature data generating unit 110 will be described later. The first feature data generating unit 110 may only shape the data and output the shaped data as the first feature data to the integrated data generating unit 140 described later.

[0030] For example, the first feature data is vector data expressing features included in the road information. The dimension of the vector when the first feature data is expressed as a vector may be the same as the number of elements of the road information data, or may be smaller than the number of elements of the road information data.

[0031] The second feature data generating unit 120 generates second feature data indicating the features of the route from the route information. More specifically, the second feature data generating unit 120 may generate the second feature data from the route information via a neural network. A method for configuring the neural network used in the second feature data generating unit 120 will be described later. The second feature data generating unit 120 may only shape the data and output the shaped data as the second feature data to the integrated data generating unit 140 described later.

[0032] For example, the second feature data is vector data expressing features included in the route information. The dimension of the vector when the second feature data is expressed as a vector may be the same as the number of elements of the route information data, or may be smaller than the number of elements of the route information data.

[0033] The integrated data generating unit 140 generates integrated data by integrating the first feature data and the second feature data. More specifically, the integrated data generating unit 140 may generate the integrated data from the first feature data and the second feature data via a neural network. A method for configuring the neural network used in the integrated data generating unit 140 will be described later.

[0034] The integrated data generating unit 140 may generate the direct sum of the first feature data expressed as a vector and the second feature data expressed as a vector as the integrated data. In this case, the number of dimensions of the integrated data expressed as a vector is the sum of the number of dimensions of the first feature data and the number of dimensions of the second feature data.

[0035] Alternatively, the integrated data generating unit 140 may perform weighting to integrate the first feature data and the second feature data when generating the integrated data. Here, the "weighting" may be set in advance. The integrated data generating unit 140 may perform a direct sum of the first feature data expressed as a vector and the second feature data expressed as a vector for each dimension to generate the integrated data.

[0036] Also, the weighting may be set so as to improve the accuracy of a probability map of a road structure in the probability map generating unit 130 described later. For example, when the probability map generating unit 130 generates a probability map from integrated data based on the teacher road information and the teacher route information, the weighting may be changed in a direction to reduce the difference between the generated probability map and the teacher probability map, thereby setting the weighting.

[0037] The partial data generating unit 150 extracts a part of the integrated data to generate the partial data. For example, the partial data generating unit 150 may extract a part of the integrated data via a neural network to generate the partial data. A method for configuring the neural network used in the partial data generating unit 150 will be described later.

[0038] The probability map generating unit 130 generates a probability map of the road structure from the integrated data. More specifically, the probability map generating unit 130 may generate the probability map from the integrated data via a neural network. A method for configuring the neural network used in the probability map generating unit 130 will be described later.

[0039] Here, the "probability map" of the road structure is a map that includes information that distinguishes whether the road along which the moving body travels is "straight ahead," "a right turn," or "a left turn." The "probability map" may uniquely identify the type of road along which the moving body travels, or may specify the probability of each type. Alternatively, the "probability map" may show the road along which the moving body is scheduled to travel in a bird's-eye view seen from above the moving body.

[0040] The probability map generating unit 130 may generate a probability map of a road structure based on the partial data generated by the partial data generating unit 150, instead of the integrated data. The probability map generating unit 130 may generate a probability map from the partial data via a neural network.

[0041] In the above description, the first feature data generating unit 110, the second feature data generating unit 120, the probability map generating unit 130, the integrated data generating unit 140, and the partial data generating unit 150 generate various types of data via different neural networks. However, the controller 100 may generate a probability map from road information and route information using a single integrated neural network.

[0042] Next, a method for constructing a neural network will be described. For example, the neural network is generated by machine learning based on teacher road information, teacher route information, and a teacher probability map. The neural network may be generated by the controller 100, or may be generated using a computer resource (for example, a server equipped with a GPU (Graphics Processing Unit)) outside the road structure estimation device. The generated neural network is stored in the database 73.

[0043] When performing machine learning, the neural networks used in each of the first feature data generation unit 110, the second feature data generation unit 120, the probability map generation unit 130, the integrated data generation unit 140, and the partial data generation unit 150 may be generated individually. Alternatively, the neural networks used in each of the first feature data generation unit 110, the second feature data generation unit 120, the probability map generation unit 130, the integrated data generation unit 140, and the partial data generation unit 150 may be generated as a single neural network as a whole by machine learning.

[0044] Specifically, when adjusting the parameters of the neural network used in first feature data generation unit 110, the parameters of the neural networks used in second feature data generation unit 120, probability map generation unit 130, integrated data generation unit 140, and partial data generation unit 150 may be fixed. The same applies when adjusting the parameters of the neural networks used in second feature data generation unit 120, probability map generation unit 130, integrated data generation unit 140, and partial data generation unit 150.

[0045] In addition, when the neural networks used in each of the first feature data generation unit 110, the second feature data generation unit 120, the probability map generation unit 130, the integrated data generation unit 140, and the partial data generation unit 150 are generated as a single neural network as a whole by machine learning, the parameters of the neural networks of each unit may be adjusted simultaneously.

[0046] When generating a neural network through machine learning, training data (e.g., training road information, training route information) is input to the input layer of the neural network. At that time, the parameters related to the neural network are adjusted so that the error between the value output from the output layer of the neural network and the training data (e.g., training probability map) is reduced.

[0047] For example, in order to minimize an error related to the output of a neural network, a gradient descent method, a stochastic gradient descent method, etc. may be used. Here, for gradient calculation in the gradient descent method or the stochastic gradient descent method, an error backpropagation method may be used.

[0048] Other issues that can arise with machine learning using neural networks include generalization performance (the ability to discriminate against unknown data) and overfitting (a phenomenon in which the learning model fits the data used to create it while generalization performance does not improve).

[0049] Therefore, in order to alleviate overfitting, a method such as regularization that restricts the degree of freedom of weights during learning may be used. In addition, a method such as dropout that probabilistically selects units in a neural network and disables other units may be used. Furthermore, in order to improve generalization performance, a method such as data regularization, data standardization, and data augmentation that eliminate bias in the data may be used.

[0050] [Processing example of road structure estimation device] 2 is a flowchart showing the process of the road structure estimation device according to the present embodiment. The process of the information processing device shown in FIG. 2 may be repeatedly executed at a predetermined cycle.

[0051] In step S101, the acquisition unit 71 acquires road information and route information.

[0052] In step S103, the first feature data generating unit 110 generates first feature data indicating the features of the road structure around the mobile object from the road information.

[0053] In step S105, the second characteristic data generating unit 120 generates second characteristic data indicating the characteristics of the route from the route information.

[0054] In step S107, the integrated data generating unit 140 integrates the first feature data and the second feature data to generate integrated data.

[0055] In step S109, the probability map generating unit 130 generates a probability map of the road structure from the integrated data. The probability map generating unit 130 may generate the probability map of the road structure based on the partial data generated by the partial data generating unit 150 instead of the integrated data.

[0056] In step S111, the output unit 400 outputs the generated probability map of the road structure, after which the flowchart in FIG.

[0057] [Effects of the embodiment] As described above in detail, the road structure estimation method and road structure estimation device according to this embodiment generate first feature data indicating the characteristics of the road structure around a moving object from road information around the moving object obtained via a sensor provided on the moving object, and generate second feature data indicating the characteristics of the route from route information on a map that shows the route traveled by the moving object. Then, the first feature data and the second feature data are integrated to generate integrated data, and a probability map of the road structure is generated from the integrated data.

[0058] This makes it possible to estimate the structure of the direction in which the moving object wants to travel while it is moving. In particular, by also using route information on a map that shows the route traveled by the moving object, it is possible to improve the accuracy of generating a probability map of the road structure. As a result, it is possible to accurately estimate the structure of the direction in which the moving object wants to travel.

[0059] Furthermore, the road structure estimation method and the road structure estimation device according to the present embodiment may integrate the first feature data and the second feature data by weighting when generating integrated data, thereby improving the accuracy when generating a probability map of the road structure.

[0060] Furthermore, in the road structure estimation method and road structure estimation device according to the present embodiment, the sensor may be at least one of a camera, an ultrasonic sensor, and a LiDAR. This makes it possible to accurately obtain road information around the moving object. And, it is possible to accurately generate a probability map of the road structure.

[0061] In the road structure estimation method and road structure estimation device according to the present embodiment, the route information may include point sequence information indicating a trajectory of past movements of a moving object on the route, thereby making it possible to identify the route traveled by the moving object. As a result, it is possible to improve the accuracy of generating a probability map of the road structure, and it is possible to accurately estimate the structure related to the desired traveling direction of the moving object.

[0062] Furthermore, in the road structure estimation method and road structure estimation device according to the present embodiment, the route information may include intersection information indicating intersections on the route. This makes it possible to identify parts of the road structure where the moving body may go straight, turn right, or turn left. As a result, it is possible to estimate the structure related to the direction in which the moving body wants to travel while it is moving. Furthermore, by identifying parts of the road structure where the moving body may go straight, turn right, or turn left, it is possible to improve the accuracy in generating a probability map of the road structure.

[0063] Furthermore, the road structure estimation method and the road structure estimation device according to the present embodiment may generate the first feature data from the road information via a neural network. This allows the features included in the road information to be accurately reflected in the first feature data. As a result, the accuracy of generating a probability map of the road structure can be improved.

[0064] Furthermore, the road structure estimation method and the road structure estimation device according to the present embodiment may generate second feature data from the route information via a neural network. This allows the features included in the route information to be accurately reflected in the second feature data. As a result, the accuracy of generating a probability map of the road structure can be improved.

[0065] Furthermore, the road structure estimation method and the road structure estimation device according to the present embodiment may generate integrated data from the first feature data and the second feature data via a neural network. This makes it possible to estimate the structure related to the direction in which the moving body wants to move while it is moving. In particular, it is possible to improve the accuracy in generating a probability map of the road structure.

[0066] Furthermore, the road structure estimation method and the road structure estimation device according to the present embodiment may extract a part of the integrated data through a neural network to generate partial data, thereby making it possible to limit notable features among the features included in the integrated data.

[0067] Furthermore, the road structure estimation method and the road structure estimation device according to the present embodiment may generate a probability map from partial data. This makes it possible to generate a probability map by focusing on noteworthy features among the features included in the integrated data. As a result, it is possible to improve the accuracy of generating a probability map of the road structure.

[0068] Furthermore, the road structure estimation method and the road structure estimation device according to the present embodiment may generate a probability map from the integrated data via a neural network. This makes it possible to estimate the structure related to the direction in which the moving object wants to move while it is moving. In particular, it is possible to improve the accuracy of generating the probability map of the road structure.

[0069] The road structure estimation method and the road structure estimation device according to the present embodiment may generate a probability map from road information and route information via a neural network, thereby making it possible to efficiently generate a probability map of the road structure by utilizing features contained in the road information and features contained in the route information.

[0070] Furthermore, in the road structure estimation method and road structure estimation device according to the present embodiment, the neural network may be generated by machine learning based on the teacher road information, the teacher route information, and the teacher probability map, thereby making it possible to express the relationship established between the teacher road information, the teacher route information, and the teacher probability map as a neural network.

[0071] Each of the functions described in the above embodiments may be implemented by one or more processing circuits, including a programmed processor, an electrical circuit, or a device such as an application specific integrated circuit (ASIC), or a circuit component arranged to perform the described functions.

[0072] Although the contents of the present invention have been described above in accordance with the embodiments, the present invention is not limited to these descriptions, and various modifications and improvements are possible, which will be obvious to those skilled in the art. The descriptions and drawings forming part of this disclosure should not be understood as limiting the present invention. Various alternative embodiments, examples, and operating techniques will be apparent to those skilled in the art from this disclosure.

[0073] Of course, the present invention includes various embodiments not described herein. Therefore, the technical scope of the present invention is defined only by the invention-specifying matters related to the scope of the claims appropriate from the above description. [Explanation of symbols]

[0074] 71 Acquisition Department 73 Database 100 Controller 110 First feature data generation unit 120 Second feature data generation unit 130 Probability map generator 140 Integrated Data Generation Department 150 Partial Data Generation Unit 400 Output section

Claims

1. Road information around the moving object obtained via a sensor provided in the moving object; and Route information on a map showing the route traveled by the moving object A road structure estimation method for controlling a controller to which The controller: generating first characteristic data indicating characteristics of a road structure around the moving object from the road information; generating second characteristic data indicating characteristics of the route from the route information; Integrating the first feature data and the second feature data to generate integrated data; generating a probability map of the road structure from the integrated data; Road structure estimation method.

2. The road structure estimating method according to claim 1 , wherein the controller integrates the first feature data and the second feature data by weighting when generating the integrated data.

3. The road structure estimation method according to claim 1 , wherein the sensor is at least one of a camera, an ultrasonic sensor, or a LiDAR.

4. The road structure estimating method according to claim 1 , wherein the route information includes point sequence information indicating a trajectory of past movements of the moving object on the route.

5. The method of claim 1 , wherein the route information includes intersection information indicating intersections on the route.

6. The road structure estimation method according to claim 1 , wherein the controller generates the first characteristic data from the road information via a neural network.

7. The road structure estimation method according to claim 1 , wherein the controller generates the second characteristic data from the route information via a neural network.

8. The road structure estimation method according to claim 1 , wherein the controller generates the integrated data from the first feature data and the second feature data via a neural network.

9. The road structure estimation method according to claim 1 , wherein the controller extracts a portion of the integrated data through a neural network to generate partial data.

10. The road structure estimation method according to claim 9 , wherein the controller generates the probability map from the partial data.

11. The method of claim 1 , wherein the controller generates the probability map from the integrated data via a neural network.

12. The method of claim 1 , wherein the controller generates the probability map from the road information and the route information via a neural network.

13. The road structure estimation method according to any one of claims 6 to 12, wherein the neural network is generated by machine learning based on teacher road information, teacher route information, and a teacher probability map.

14. Road information around the moving object obtained via a sensor provided in the moving object; and Route information on a map showing the route traveled by the moving object A road structure estimation device comprising a controller to which is input, The controller: generating first characteristic data indicating characteristics of a road structure around the moving object from the road information; generating second characteristic data indicating characteristics of the route from the route information; Integrating the first feature data and the second feature data to generate integrated data; generating a probability map of the road structure from the integrated data; Road structure estimation device.

Images