Location estimation system

Abstract

To provide a position estimation system that consumes low electric power.SOLUTION: A position estimation system has a comparison unit, a learning unit, a data acquisition unit, an inference unit, a data conversion unit, and an evaluation unit. The comparison unit has a function for calculating a first translation movement amount and a first rotation amount on the basis of machine learning data that represents map information. The learning unit has a function for generating a machine learning model through learning in which the machine learning data, the first translation movement amount, and the first rotation amount are used. The data acquisition unit has a function for acquiring acquisition data that represents environment information about the surroundings of the position estimation device. The inference unit has a function for using the machine learning model to infer a second translation movement amount and a second rotation amount on the basis of the acquisition data and the machine learning data. The data conversion unit has a function for converting the machine learning data to evaluation data on the basis of the second translation movement amount and the second rotation amount. The evaluation unit has a function for evaluating the degree of matching between the acquisition data and the evaluation data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] One aspect of the present invention relates to a position estimation system. Another aspect of the present invention relates to a position estimation method. Another aspect of the present invention relates to a position estimation device. Another aspect of the present invention relates to a mobile body having a position estimation device. [Background technology]

[0002] In recent years, autonomous driving technology for automobiles has attracted attention. One example of autonomous driving technology is self-localization technology. Patent document 1 discloses a method in which sensors are installed on an automobile, scans the environment around the automobile using these sensors to acquire scan data in real time, and estimates the self-localization based on the acquired scan data. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Special Publication No. 2018-533721 [Overview of the project] [Problems that the invention aims to solve]

[0004] When acquiring a self-position based on scan data, a massive amount of computation may be required to estimate the self-position. Therefore, if you try to acquire the self-position in real time, you will need to use a high-performance computing device, which can lead to high power consumption.

[0005] Therefore, one aspect of the present invention aims to provide a location estimation system that can estimate location in real time. Another aspect of the present invention aims to provide a location estimation system with reduced power consumption. Another aspect of the present invention aims to provide a low-cost location estimation system. Another aspect of the present invention aims to provide a novel location estimation system. Another aspect of the present invention aims to provide a location estimation method using the above-mentioned location estimation system.

[0006] Furthermore, one aspect of the present invention aims to provide a position estimation device capable of real-time position estimation. Another aspect of the present invention aims to provide a position estimation device with reduced power consumption. Another aspect of the present invention aims to provide a low-cost position estimation device. Another aspect of the present invention aims to provide a novel position estimation device. Another aspect of the present invention aims to provide a position estimation method using the above-mentioned position estimation device.

[0007] Furthermore, the description of these problems does not preclude the existence of other problems. Moreover, one aspect of the present invention does not need to solve all of these problems. Other problems will naturally become apparent from the description in the specification, drawings, and claims, and it is possible to extract other problems from the description in the specification, drawings, and claims. [Means for solving the problem]

[0008] One aspect of the present invention comprises a learning device and a position estimation device, the learning device comprising a comparison unit and a learning unit, the position estimation device comprising a data acquisition unit, an inference unit, a data conversion unit and an evaluation unit, the data acquisition unit comprising a sensor, the comparison unit having the function of selecting two types of machine learning data from three or more types of machine learning data representing map information and calculating a first translation amount and a first rotation amount by comparing the two types of machine learning data, and the learning unit learning using the two types of machine learning data, the first translation amount and the first rotation amount. This position estimation system has the following functions: a function to generate a machine learning model; a data acquisition unit that acquires data using sensors; an inference unit that uses the machine learning model to infer a second translation amount and a second rotation amount based on the acquired data and one type of machine learning data selected from three or more types of machine learning data; a data conversion unit that converts one type of machine learning data into evaluation data based on the second translation amount and the second rotation amount; and an evaluation unit that evaluates the degree of agreement between the acquired data and the evaluation data.

[0009] Alternatively, one aspect of the present invention comprises a learning device and a position estimation device, the learning device comprising a first point cloud-to-image conversion unit, a comparison unit, and a learning unit, the position estimation device comprising a point cloud data acquisition unit, a second point cloud-to-image conversion unit, an inference unit, a data conversion unit, and an evaluation unit, the first point cloud-to-image conversion unit having the function of converting n types (n is an integer of 3 or more) of machine learning point cloud data representing map information into n types of machine learning image data, the comparison unit having the function of selecting two types of machine learning point cloud data from the n types of machine learning point cloud data and calculating a first translation amount and a first rotation amount by comparing the two types of machine learning point cloud data, the learning unit having two types of machine learning image data corresponding to the two types of machine learning point cloud data, and the first translation amount, This position estimation system has the following functions: a function to generate a machine learning model by learning using a first rotation amount; a point cloud data acquisition unit has the function to acquire acquired point cloud data; a second point cloud-to-image conversion unit has the function to convert acquired point cloud data into acquired image data; an inference unit has the function to infer a second translation amount and a second rotation amount based on the acquired image data and one type of machine learning image data selected from n types of machine learning image data using the machine learning model; a data conversion unit has the function to convert one type of machine learning point cloud data corresponding to one type of machine learning image data into evaluation point cloud data based on the second translation amount and the second rotation amount; and an evaluation unit has the function to evaluate the degree of agreement between the acquired point cloud data and the evaluation point cloud data.

[0010] Alternatively, in the above embodiment, the acquired image data and the machine learning image data may be binary data.

[0011] Alternatively, in the above embodiment, the machine learning model may be a convolutional neural network model.

[0012] Alternatively, in the above embodiment, the first translation amount and the first rotation amount may be calculated by scan matching.

[0013] Alternatively, one aspect of the present invention is a position estimation device comprising a data acquisition unit, an inference unit, a data conversion unit, and an evaluation unit, wherein the data acquisition unit has a sensor and a function to acquire data using the sensor, the inference unit has a function to infer a first translation amount and a first rotation amount based on the acquired data and one type of machine learning data selected from three or more types of machine learning data representing map information using a machine learning model, the machine learning model is generated by learning using two types of machine learning data selected from three or more types of machine learning data, and a second translation amount and a second rotation amount calculated by comparing the two types of machine learning data, the data conversion unit has a function to convert one type of machine learning data into evaluation data based on the first translation amount and the first rotation amount, and the evaluation unit has a function to evaluate the degree of agreement between the acquired data and the evaluation data.

[0014] Alternatively, in the above embodiment, the machine learning model may be a convolutional neural network model.

[0015] Alternatively, in the above embodiment, the second translation amount and the second rotation amount may be calculated by scan matching.

[0016] A mobile body having a position estimation device and a battery, according to one aspect of the present invention, is also an aspect of the present invention.

[0017] Alternatively, in the above embodiment, the mobile body may have a function for autonomous driving. [Effects of the Invention]

[0018] According to one aspect of the present invention, a location estimation system capable of real-time location estimation can be provided. Furthermore, according to one aspect of the present invention, a location estimation system with reduced power consumption can be provided. Furthermore, according to one aspect of the present invention, a low-cost location estimation system can be provided. Furthermore, according to one aspect of the present invention, a novel location estimation system can be provided. Furthermore, according to one aspect of the present invention, a location estimation method using the above-mentioned location estimation system can be provided.

[0019] Furthermore, according to one aspect of the present invention, a position estimation device capable of real-time position estimation can be provided. Furthermore, according to one aspect of the present invention, a position estimation device with reduced power consumption can be provided. Furthermore, according to one aspect of the present invention, a low-cost position estimation device can be provided. Furthermore, according to one aspect of the present invention, a novel position estimation device can be provided. Furthermore, according to one aspect of the present invention, a position estimation method using the above-mentioned position estimation device can be provided.

[0020] Furthermore, the effects of one aspect of the present invention are not limited to those listed above. The effects listed above do not preclude the existence of other effects. These other effects are those described below and not mentioned in this section. Those skilled in the art can deduce these effects from the description in the specification, drawings, etc., and can be appropriately extracted from these descriptions. Furthermore, one aspect of the present invention has at least one of the effects listed above and / or other effects. Therefore, one aspect of the present invention may, in some cases, not have the effects listed above. [Brief explanation of the drawing]

[0021] [Figure 1] Figure 1 is a block diagram showing an example configuration of a position estimation system. [Figure 2] Figure 2 shows an example of a mobile body. [Figure 3] Figure 3 shows an example of a machine learning model. [Figure 4]Figure 4 is a flowchart showing an example of a location estimation method. [Figure 5] Figure 5 is a schematic diagram illustrating an example of a position estimation method. [Figure 6] Figures 6A and 6B are schematic diagrams illustrating an example of a position estimation method. [Figure 7] Figure 7 is a flowchart showing an example of a location estimation method. [Figure 8] Figures 8A and 8B are schematic diagrams illustrating an example of a position estimation method. [Modes for carrying out the invention]

[0022] Embodiments will be described in detail with reference to the drawings. However, it will be readily apparent to those skilled in the art that the present invention is not limited to the following description, and that its form and details can be modified in various ways without departing from the spirit and scope of the invention. Therefore, the present invention shall not be construed as being limited to the descriptions of the embodiments shown below.

[0023] For the sake of clarity, the positions, sizes, and ranges of the components shown in the drawings may not necessarily represent their actual positions, sizes, and ranges. Therefore, the disclosed invention is not necessarily limited to the positions, sizes, and ranges disclosed in the drawings.

[0024] Furthermore, the ordinal numbers "first," "second," and "third" used in this specification are added to avoid confusion of constituent elements and do not imply any numerical limitation.

[0025] (Embodiment) In this embodiment, a position estimation system according to one aspect of the present invention, and a position estimation method using the position estimation system, etc., will be explained with reference to the drawings.

[0026] <Example of a position estimation system configuration> Figure 1 is a block diagram showing an example configuration of the position estimation system 10. The position estimation system 10 includes a learning device 20 and a position estimation device 30. Here, it is preferable that the learning device 20 be installed on a device with high computing power, such as a server. The learning device 20 and the position estimation device 30 can exchange data with each other via a network or the like.

[0027] The learning device 20 includes an input unit 21, a point cloud-to-image conversion unit 22, a comparison unit 23, and a learning unit 24. The position estimation device 30 includes a data acquisition unit 31, an inference unit 34, a data conversion unit 35, and an evaluation unit 36. Here, the data acquisition unit 31 includes a point cloud data acquisition unit 32 and a point cloud-to-image conversion unit 33. Although not shown in Figure 1, the learning device 20 and the position estimation device 30 may have, for example, a storage unit. This storage unit can store data and programs used to drive the position estimation system 10, and each component of the position estimation system 10 can read these as needed.

[0028] Figure 1 shows the data exchange between components of the position estimation system 10 using arrows. Note that the data exchange shown in Figure 1 is just one example; for example, data may be exchanged between components not connected by arrows. Furthermore, even between components connected by arrows, data may not be exchanged.

[0029] First, let's explain the learning device 20. The input unit 21 has an interface function and receives point cloud data PD for machine learning. ML The following is input. In one aspect of the present invention, the input unit 21 receives machine learning point cloud data PD of n (where n is an integer of 3 or more). ML The following is input: Point cloud data PD for machine learning. ML This can be point cloud data acquired by a device located outside the position estimation system 10 and stored in a database. Therefore, point cloud data for machine learning can be rephrased as database point cloud data.

[0030] Point cloud data PD for machine learning ML can be obtained by a device having, for example, a laser and a sensor. Specifically, for example, by irradiating laser light and detecting the scattered laser light with a sensor, point cloud data PD for machine learning ML can be obtained. That is, for example, point cloud data PD for machine learning can be obtained using LiDAR (Light Detection And Ranging). ML The obtained point cloud data PD for machine learning ML represents map information and can also include information for specifying a position on the map. That is, point cloud data PD for machine learning ML can be said to be data representing map information including position information. Point cloud data PD for machine learning ML can be supplied to the point cloud-image conversion unit 22, the comparison unit 23, and the data conversion unit 35.

[0031] The point cloud-image conversion unit 22 has a function of converting point cloud data into image data. Specifically, it has a function of converting point cloud data PD for machine learning ML into machine learning image data GD ML For example, the point cloud-image conversion unit 22 has a function of converting point cloud data PD for machine learning ML into binary machine learning image data GD in which coordinates containing points are set to "1" and coordinates not containing points are set to "0". ML As described above, the point cloud data for machine learning can be referred to as database point cloud data. Therefore, the machine learning image data can be referred to as database image data.

[0032] As described above, the point cloud data PD for machine learning ML represents map information, and the machine learning image data GD ML is a conversion of the point cloud data PD for machine learning ML Therefore, the point cloud data PD for machine learning ML and the machine learning image data GD ML can be referred to as map data.

[0033] The comparison unit 23 compares the machine learning point cloud data PD input to the input unit 21. ML From among them, two point cloud data for machine learning PD ML It has the function of calculating the amount of translation and rotation by extracting and comparing data points. For example, point cloud data PD for machine learning. ML When the coordinate system is represented in a two-dimensional coordinate system (xy coordinate system), the comparison unit 23 can calculate the amount of translation, specifically the amount of movement in the x-axis direction Δx1 and the amount of movement in the y-axis direction Δy1. The comparison unit 23 can also calculate the amount of rotation θ1.

[0034] In the following explanation, point cloud data and image data are described assuming they are represented in a two-dimensional coordinate system. However, by increasing the dimensionality of the translation and rotation amounts, the following explanation can also be applied when point cloud data and image data are represented in a three-dimensional coordinate system. For example, when point cloud data and image data are represented in a three-dimensional coordinate system, the translation amount can be represented by a three-dimensional vector. The rotation amount can also be represented by a rotation vector, rotation matrix, Euler angle, or quaternion. Furthermore, when point cloud data and image data are represented in a three-dimensional coordinate system, they can be represented as three-dimensional array data.

[0035] In this specification and other documents, when point cloud data is represented in a two-dimensional coordinate system, and the displacement in the x-axis direction is Δx and the displacement in the y-axis direction is Δy, the translational displacement is denoted as (Δx, Δy).

[0036] The translation amount (Δx1, Δy1) and rotation amount θ1 can be calculated by scan matching, for example, by ICP (Iterative Closest Point) scan matching or NDT (Normal Distribution Transform) scan matching. The translation amount (Δx1, Δy1) and rotation amount θ1 can be calculated by comparing, for example, two machine learning point cloud data GD ML It can be calculated in a way that maximizes the degree of agreement.

[0037] The learning unit 24 has the function of generating a machine learning model (MLM). Examples of machine learning models (MLM) that can be applied include multilayer perceptrons, support vector machines, and neural network models. In particular, it is preferable to apply a convolutional neural network (CNN) as the machine learning model (MLM).

[0038] The learning unit 24 is for machine learning image data GD ML The system has a function to perform learning using the translation amount (Δx1, Δy1) and the rotation amount θ1 to generate a machine learning model MLM. The generation of the machine learning model MLM can be done, for example, by supervised learning. For example, two machine learning point cloud data PD compared by the comparison unit 23 ML GD for two machine learning image data corresponding to the two GDs ML By using this as training data and associating the translation amount (Δx1, Δy1) and rotation amount θ1 as correct labels with this training data, a machine learning model (MLM) can be generated.

[0039] The above is a description of the learning device 20.

[0040] Next, the position estimation device 30 will be described. The data acquisition unit 31 has the function of acquiring data. The data acquisition unit 31 acquires, for example, point cloud data PD. AC , and acquired image data GD AC It has the function to acquire the acquired image data GD. Details will be described later, but the acquired image data GD AC For example, acquired point cloud data PD AC It can be obtained by converting it into image data.

[0041] The data acquired by the data acquisition unit 31 can be supplied to the inference unit 34 and the evaluation unit 36. The data acquisition unit 31 can, for example, acquire point cloud data PD. AC The acquired image data GD is supplied to the evaluation unit 36. AC This can be supplied, for example, to the inference unit 34.

[0042] The point cloud data acquisition unit 32 acquires the point cloud data PD AC It has the function of acquiring point cloud data PD. The point cloud data acquisition unit 32 has, for example, a laser and a sensor, and by irradiating the area around the position estimation device 30 with the laser and detecting the scattered laser light with the sensor, it acquires point cloud data PD. AC This allows for the acquisition of point cloud data PD representing environmental information around the position estimation device 30, for example, using LiDAR. AC The point cloud data acquisition unit 32 can acquire this data.

[0043] The point cloud-to-image conversion unit 33 has the function of converting point cloud data into image data. Specifically, it converts the acquired point cloud data PD AC Acquire image data GD AC It has the function of converting to a point cloud image. The point cloud-to-image conversion unit 33 has the function of converting point cloud data to image data in the same way as the point cloud-to-image conversion unit 22. Specifically, the point cloud-to-image conversion unit 33 converts, for example, acquired point cloud data PD AC This is a binary image data acquisition method where coordinates containing a point are assigned "1" and coordinates not containing a point are assigned "0". AC It has the function to convert to [a specific format].

[0044] The inference unit 34 has the function of performing inference based on the machine learning model MLM. Specifically, it performs inference based on acquired image data GD AC And, the first machine learning image data GD ML When the input to the inference unit 34 is given, it has the function of inferring the amount of translation (Δx2, Δy2) and the amount of rotation θ2 based on the machine learning model MLM.

[0045] The data conversion unit 35 converts the machine learning image data GD input to the inference unit 34. ML PD point cloud data for machine learning that corresponds to this data. ML Based on the amount of translation (Δx2, Δy2) and the amount of rotation θ2, evaluation point cloud data PD E It has the function of converting the machine learning image data GD input to the inference unit 34. Specifically, the data conversion unit 35 has the function of converting the machine learning image data GD input to the inference unit 34. MLPD point cloud data for machine learning that corresponds to this data. ML By translating each point in the data by (Δx2, Δy2) and rotating it by θ2, the point cloud data PD for machine learning is obtained. ML Point cloud data PD for evaluation E It has the function to convert to [a specific format].

[0046] The evaluation unit 36 ​​uses the acquired point cloud data PD. AC And, evaluation point cloud data PD E It has a function to calculate an evaluation value that represents the degree of agreement between and . The evaluation value can be calculated using a method used in scan matching such as ICP scan matching or NDT scan matching. For example, acquired point cloud data PD AC The points included and the corresponding point cloud data PD for evaluation E The distance, or the square of the distance, between each point and the point included in the data is calculated. The sum of the distances, or the sum of the squares of the distances, can be used as the evaluation value. In this case, the smaller the evaluation value, the better the acquired point cloud data PD. AC And, evaluation point cloud data PD E It can be said that the degree of agreement between the two is high.

[0047] Acquired point cloud data PD AC And, point cloud data for evaluation PD E If the degree of agreement is low, the evaluation point cloud data PD E It can be assumed that the position estimation device 30 is located at a position separate from the location represented by PD. On the other hand, the acquired point cloud data PD AC And, evaluation point cloud data PD E If the degree of agreement is high, the evaluation point cloud data PD E It can be assumed that the position estimation device 30 is located close to the location represented by PD. Therefore, the acquired point cloud data PD AC And, evaluation point cloud data PD E By evaluating the degree of agreement between the two, the position of the position estimation device 30 can be estimated.

[0048] The above is an example of the configuration of the position estimation system 10. The position estimation system 10 can calculate the amount of translation (Δx2, Δy2) and the amount of rotation θ2 by inference using a machine learning model (MLM). This reduces the amount of computation performed by the position estimation device 30 compared to when the amount of translation (Δx2, Δy2) and the amount of rotation θ2 are calculated without using a machine learning model. Therefore, it is possible to estimate the position of the position estimation device 30 in real time while reducing the power consumption of the position estimation device 30. In addition, since the CPU (Central Processing Unit) and GPU (Graphics Processing Unit) of the position estimation device 30 do not need to be high performance, the position estimation device 30 can be made inexpensive.

[0049] The position estimation device 30 can be applied to, for example, a moving object. The moving object can be, for example, an automobile. Figure 2 shows an automobile 40 as an example of a moving object. As mentioned above, the point cloud data acquisition unit 32 of the position estimation device 30 can be equipped with a laser and a sensor. Figure 2 shows an example configuration in which the automobile 40 has a laser 37 and a sensor 38.

[0050] Furthermore, the automobile 40 is equipped with a battery 41. The battery 41 can supply the power necessary to drive the position estimation device 30.

[0051] By applying the position estimation device 30 to a moving object, the position of the moving object can be estimated in real time. Therefore, a moving object to which the position estimation device 30 is applied can have the function of performing autonomous driving. As mentioned above, the power consumption of the position estimation device 30 is low. Therefore, even when a moving object is given the function of performing autonomous driving by applying the position estimation device 30, it is possible to suppress a significant increase in the power consumption of the moving object compared to a moving object that does not have the function of performing autonomous driving. Specifically, it is possible to suppress a significant increase in the power consumption of the battery of the moving object.

[0052] As mentioned above, the position estimation system 10 uses machine learning point cloud data PD. ML Image data for machine learning GD ML The data is converted and supplied to the learning unit 24 and the inference unit 34. Furthermore, the acquired point cloud data PD is also converted. AC Acquire image data GD AC The data is converted to image data and supplied to the inference unit 34. In other words, the position estimation system 10 converts point cloud data into image data and uses this image data to perform machine learning. This allows the machine learning model MLM to be, for example, a CNN. Alternatively, machine learning may be performed using the point cloud data without converting it to image data.

[0053] Figure 3 shows a CNN that can be applied to the machine learning model MLM. The machine learning model MLM with a CNN applied has an input layer IL, a hidden layer ML, and an output layer OL. The hidden layer ML has a convolutional layer CL, a pooling layer PL, and a fully connected layer FCL. Figure 3 shows an example where the machine learning model MLM has m convolutional layers CL and pooling layers PL (where m is an integer greater than or equal to 1), and two fully connected layers FCL. Note that the machine learning model MLM may have only one fully connected layer FCL, or three or more.

[0054] In this specification, for example, multiple layers, data, etc. of the same type are distinguished by being described as [1], [2], [m], etc. For example, a convolutional layer CL of layer m is distinguished by being described as convolutional layer CL[1] to convolutional layer CL[m].

[0055] A convolutional layer CL has the function of performing convolution on the data input to it. For example, convolutional layer CL[1] has the function of performing convolution on the data input to the input layer IL. Also, convolutional layer CL[2] has the function of performing convolution on the data output from the pooling layer PL[1]. Furthermore, convolutional layer CL[m] has the function of performing convolution on the data output from the pooling layer PL[m-1].

[0056] Convolution is performed by repeatedly performing sum-of-products operations on the data input to the convolutional layer CL and the weight filter. Through convolution in the convolutional layer CL, feature extraction and other operations are performed on the data input to the machine learning model MLM.

[0057] The convolved data is transformed by an activation function and then output to the pooling layer PL. ReLU (Rectified Linear Units) can be used as the activation function. ReLU outputs "0" if the input value is negative, and outputs the input value itself if the input value is greater than or equal to "0". Other activation functions such as the sigmoid function and the tanh function can also be used.

[0058] The pooling layer PL has the function of performing pooling on the data input from the convolutional layer CL. Pooling is a process that divides the data into multiple regions, extracts predetermined data from each region, and arranges it in a matrix. Pooling allows for a reduction in the amount of data while retaining the features extracted by the convolutional layer CL. It also improves robustness to minute deviations in the input data. Various pooling methods can be used, including maximum pooling, average pooling, and low-pass pooling.

[0059] The fully connected layer (FCL) has the function of combining input data and transforming the combined data using an activation function before outputting it. ReLU, sigmoid function, tanh function, etc., can be used as activation functions.

[0060] Note that the configuration of a machine learning model MLM using a CNN is not limited to the configuration shown in Figure 3. For example, a pooling layer PL may be provided for every multiple convolutional layers CL. In other words, the number of pooling layers PL in a machine learning model MLM may be less than the number of convolutional layers CL. Also, if it is important to preserve as much of the positional information of the extracted features as possible, a pooling layer PL may not be necessary.

[0061] A machine learning model (MLM) that applies a Convolutional Neural Network (CNN) can optimize the filter values ​​of the weight filters, the weight coefficients of the fully connected layer (FCL), and other parameters through training.

[0062] <An example of a position estimation method> The following describes an example of a position estimation method using the position estimation system 10. Specifically, it describes an example of a method for generating a machine learning model MLM by the learning device 20 and a method for estimating a position using the machine learning model MLM by the position estimation device 30. The position of the position estimation device 30 can be estimated, for example, by the method shown below.

[0063] [An example of a method for generating machine learning models] Figure 4 is a flowchart showing an example of a method for generating a machine learning model (MLM). As shown in Figure 4, the machine learning model (MLM) is generated by the method shown in steps S01 to S07.

[0064] To generate a machine learning model (MLM), first, you need machine learning point cloud data (PD). ML [1] or point cloud data for machine learning PD ML Input [n] into the input unit 21 (step S01). As described above, the machine learning point cloud data PD ML This can be point cloud data representing map information including location information, acquired by LiDAR or similar means.

[0065] Next, the point cloud-to-image conversion unit 22 processes the point cloud data PD for machine learning. ML [1] or point cloud data for machine learning PD ML [n] is used for machine learning image data GD ML [1] or Image data for machine learning GD ML Each is converted to [n] (step S02). Figure 5 is a schematic diagram showing an example of the operation in step S02.

[0066] In step S02, the point cloud-to-image conversion unit 22 converts the point cloud data PD for machine learning. ML [1] or point cloud data for machine learning PD MLLet [n] be the binary machine learning image data GD in which the coordinates containing points are "1" and the coordinates not containing points are "0". ML from [1] to the machine learning image data GD ML [n] is converted. In FIG. 5, the machine learning point group data PD ML from [1] to the machine learning point group data PD ML [n] is taken as an example of conversion to the binary machine learning image data GD in which the coordinates containing points are black and the coordinates not containing points are white. ML from [1] to the machine learning image data GD ML [n] is shown.

[0067] Next, the comparison unit 23 sets the values of "i" and "j" (step S03). Thereafter, the machine learning point group data PD ML [i] and the machine learning point group data PD ML [j] are compared, and the translation amount (Δx1 i,j , Δy1 i,j ) and the rotation amount θ1 i,j are calculated (step S04). FIG. 6A is a schematic diagram showing an example of the operation in step S04. Here, i and j are each integers of 1 or more and n or less. Also, i and j are different values from each other. Here, the machine learning point group data PD ML [i] and the machine learning point group data PD[[ID=,29]] ML [j] are preferably point group data representing positions close to each other. Specifically, at least a part of the location represented by the machine learning point group data PD ML [i] is preferably included in the machine learning point group data PD ML [j]. In step S03, the values of "i" and "j" may be set one or more at a time. <,

[0068] As described above, the calculation of the translation amount (Δx1 i,j , Δy1 i,j ) and the rotation amount θ1 i,j can be performed by scan matching, for example, by ICP scan matching or NDT scan matching. The translation amount (Δx1 i,j , Δy1 i,j ) and the rotation amount θ1i,j is, for example, point cloud data PD for machine learning ML [i] and point cloud data PD for machine learning ML [j] can be calculated so that the degree of coincidence is the highest.

[0069] Thereafter, the learning unit 24 uses the image data GD for machine learning ML [i] and the image data GD for machine learning ML [j], and the translation amount (Δx1 i,j , Δy1 i,j ), and the rotation amount θ1 i,j to perform learning (step S05). As a result, the learning unit 24 can generate a machine learning model MLM. FIG. 6B is a schematic diagram showing an example of the operation in step S05.

[0070] In this specification and the like, for example, data obtained by converting the point cloud data PD for machine learning ML [i] into image data is defined as the image data GD for machine learning ML [i], and data obtained by converting the point cloud data PD for machine learning ML [j] into image data is defined as the image data GD for machine learning ML [j]. Then, for example, the point cloud data PD for machine learning ML [i] and the image data GD for machine learning ML [i] are referred to as corresponding data to each other. Also, the point cloud data PD for machine learning ML [j] and the image data GD for machine learning ML [j] are referred to as corresponding data to each other. The same applies when other point cloud data is converted into image data.

[0071] As described above, the above learning can be, for example, supervised learning. For example, the image data GD for machine learning ML [i] and the image data GD for machine learning ML [j] are used as learning data, and the translation amount (Δx1 i,j , Δy1 i,j ) and the rotation amount θ1 i,j are associated with the learning data as correct labels, and by learning, the learning unit 24 can generate a machine learning model MLM.

[0072] Next, a decision is made as to whether or not to terminate the learning process (step S06). The learning process may be terminated when a predetermined number of iterations have been reached. Alternatively, the learning process may be terminated by performing a test using test data, and the machine learning model MLM may be determined to have a translation amount (Δx1 i,j ,Δy1 i,j ) and rotation amount θ1 i,j The training may be terminated when the model can correctly output the result (when the output value of the loss function falls below a threshold). Alternatively, the training may be terminated when the output value of the loss function has saturated to a certain extent. Alternatively, the user may specify when to terminate the training.

[0073] If the learning process is not completed, the actions shown in steps S03 to S06 are repeated. That is, the values ​​of one or both of "i" and "j" are reset to different values, and the learning process is performed again.

[0074] When training is complete, the training unit 24 outputs the trained machine learning model MLM (step S07). The trained machine learning model MLM is supplied to the position estimation device 30. Specifically, the trained machine learning model MLM is supplied to the inference unit 34 of the position estimation device 30.

[0075] The above is one example of a method for generating a machine learning model (MLM).

[0076] [An example of a location estimation method] Figure 7 is a flowchart showing an example of a position estimation method using a machine learning model (MLM). As shown in Figure 7, the position of the position estimation device 30 is estimated by the method shown in steps S11 to S18.

[0077] To perform position estimation, first, the point cloud data acquisition unit 32 acquires point cloud data PD representing the surrounding environment information of the position estimation device 30. AC (Step S11) The point cloud data acquisition unit 32 acquires the point cloud data PD, for example, by LiDAR. AC You can obtain it.

[0078] Next, the point cloud-to-image conversion unit 33 processes the acquired point cloud data PD. AC Acquire image data GD AC Convert to (Step S12). For example, the point cloud-to-image conversion unit 33 converts the acquired point cloud data PD in the same manner as shown in Figure 5. AC Acquire image data GD AC It can be converted to a point cloud-image conversion unit 33. Specifically, the point cloud-image conversion unit 33 converts the acquired point cloud data PD AC This is a binary image data acquisition method where coordinates containing a point are assigned "1" and coordinates not containing a point are assigned "0". AC It can be converted to [this].

[0079] Subsequently, the inference unit 34 sets the value of "k" (step S13), and the acquired image data GD AC and GD image data for machine learning ML [k] is input to the machine learning model MLM built in the inference unit 34. This calculates the translation amount (Δx² k ,Δy2 k ), and rotation amount θ2 k This is inferred (step S14). Figure 8A is a schematic diagram showing an example of the operation in step S14. k is an integer between 1 and n, inclusive.

[0080] Next, the data conversion unit 35 calculates the amount of translation (Δx2 k ,Δy2 k ) and rotation amount θ2 k Using this, point cloud data PD for machine learning ML [k] is the point cloud data PD for evaluation E Convert to [k] (step S15). Figure 8B is a schematic diagram showing an example of the operation in step S15, etc. As described above, the data conversion unit 35 converts the machine learning point cloud data PD ML Each point included in [k] is (Δx² k ,Δy2 k ) is translated by that amount, and θ2 k By rotating only this much, the point cloud data PD for machine learning can be processed. ML [k] is the point cloud data PD for evaluation E It can be converted to [k].

[0081] In this specification, etc., point cloud data PD for machine learning ML [k] and evaluation point cloud data PD E [k] are called corresponding data points.

[0082] Subsequently, the evaluation unit 36 ​​processes the acquired point cloud data PD. AC and evaluation point cloud data PD E An evaluation value representing the degree of agreement of [k] is calculated. This is used to obtain the point cloud data PD. AC and evaluation point cloud data PD E Evaluate the degree of agreement of [k] (step S16). Figure 8B also shows an example of the operation in step S16.

[0083] As mentioned above, the evaluation value can be calculated using methods used in scan matching, such as ICP scan matching or NDT scan matching. Acquired point cloud data PD AC and evaluation point cloud data PD E By evaluating the degree of agreement of [k], the acquired point cloud data PD AC and point cloud data PD for machine learning ML The degree of agreement of [k] can be evaluated. For example, if a point in one of two point cloud datasets is translated and rotated around a single point, and that point cloud dataset matches the other point cloud dataset, then the two point cloud datasets can be considered to be identical.

[0084] Next, it is determined whether the number of set “k” values ​​has reached a specified number (step S17). This specified number can be, for example, n. In this case, all the machine learning point cloud data PD ML In contrast, the acquired point cloud data PD AC The degree of agreement can be evaluated. Furthermore, this specified number may be smaller than n. In this case, for example, all the machine learning point cloud data PD used during training can be evaluated. ML In contrast, the acquired point cloud data PD AC The value of "k" can be set to evaluate the degree of agreement.

[0085] If the number of set "k" values ​​does not reach the specified number, the operations shown in steps S13 to S17 are repeated. In other words, the value of "k" is reset to a different value, and the acquired point cloud data PD is processed. AC and point cloud data PD for machine learning ML Evaluate the degree of agreement of [k].

[0086] If the number of set "k" values ​​reaches a predetermined number, the evaluation unit 36 ​​estimates the position of the position estimation device 30 (step S18). For example, acquired point cloud data PD AC The machine learning point cloud data PD with the highest degree of agreement ML The position represented by is the acquired point cloud data PD AC This can be set to the position of the position estimation device 30 that acquired the position.

[0087] Furthermore, even if the number of set "k" values ​​in step S17 does not reach the specified number, the acquired point cloud data PD AC and evaluation point cloud data PD E If the degree of agreement for [k] exceeds the threshold, you may proceed to step S18. In this case, the acquired point cloud data PD AC The degree of agreement with the evaluation point cloud data PD was above the threshold. E The position represented by [k] can be the position of the position estimation device 30.

[0088] The above is an example of a position estimation method using the position estimation system 10. In the position estimation method using the position estimation system 10, the amount of translation (Δx2, Δy2) and the amount of rotation θ2 can be calculated by inference using the machine learning model MLM. As a result, the amount of computation performed by the position estimation device 30 can be reduced compared to when the amount of translation (Δx2, Δy2) and the amount of rotation θ2 are calculated without using a machine learning model. Therefore, the power consumption of the position estimation device 30 can be reduced while estimating the position of the position estimation device 30 in real time. In addition, since the CPU and GPU of the position estimation device 30 do not need to be high performance, the position estimation device 30 can be made inexpensive. [Explanation of Symbols]

[0089] 10: Position estimation system, 20: Learning device, 21: Input unit, 22: Point cloud-to-image conversion unit, 23: Comparison unit, 24: Learning unit, 30: Position estimation device, 31: Data acquisition unit, 32: Point cloud data acquisition unit, 33: Point cloud-to-image conversion unit, 34: Inference unit, 35: Data conversion unit, 36: Evaluation unit, 37: Laser, 38: Sensor, 40: Automobile, 41: Battery

Claims

1. It has a learning device and a position estimation device, The learning device comprises a comparison unit and a learning unit. The position estimation device comprises a data acquisition unit, an inference unit, a data conversion unit, and an evaluation unit. The data acquisition unit has a sensor, The comparison unit has the function of selecting two types of machine learning data from three or more types of machine learning data, which are point cloud data representing map information including location information, and comparing the two types of machine learning data to calculate a first translation amount and a first rotation amount of the point cloud data. The learning unit has the function of generating a machine learning model by learning using the two types of machine learning data, the first translation amount, and the first rotation amount. The data acquisition unit has the function of acquiring data using the sensor, The inference unit has the function of inferring a second translation amount and a second rotation amount of the point cloud data based on the acquired data and one type of machine learning data selected from the three or more types of machine learning data, using the machine learning model. The data conversion unit has a function to convert the one type of machine learning data into evaluation data based on the second translation amount and the second rotation amount. The evaluation unit is a position estimation system having the function of evaluating the degree of agreement between the acquired data and the evaluation data.

2. In Claim 1, The aforementioned data acquisition unit further includes a laser, A position estimation system that irradiates the area around the position estimation device with the laser and detects the scattered laser light with the sensor to acquire the acquired data.

Images