Position estimation system

Abstract

To provide a position estimation system that consumes low electric power.SOLUTION: A position estimation system has a comparison unit, a learning unit, a data acquisition unit, an inference unit, a data conversion unit, and an evaluation unit. The comparison unit has a function for calculating a first translation movement amount and a first rotation amount on the basis of machine learning data that represents map information. The learning unit has a function for generating a machine learning model through learning in which the machine learning data, the first translation movement amount, and the first rotation amount are used. The data acquisition unit has a function for acquiring acquisition data that represents environment information about the surroundings of the position estimation device. The inference unit has a function for using the machine learning model to infer a second translation movement amount and a second rotation amount on the basis of the acquisition data and the machine learning data. The data conversion unit has a function for converting the machine learning data to evaluation data on the basis of the second translation movement amount and the second rotation amount. The evaluation unit has a function for evaluating the degree of matching between the acquisition data and the evaluation data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] One aspect of the present invention relates to a position estimation system. Another aspect of the present invention relates to a position estimation method. Another aspect of the present invention relates to a position estimation device. Another aspect of the present invention relates to a moving body having a position estimation device.

Background Art

[0002] In recent years, the autonomous driving technology of automobiles has attracted attention. One of the autonomous driving technologies is self-position estimation technology. Patent Document 1 discloses a method of providing a sensor in an automobile, acquiring scan data in real time by scanning the environment around the automobile using the sensor, and estimating the self-position based on the acquired scan data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When acquiring the self-position based on scan data, a huge amount of calculation may be required for estimating the self-position. For this reason, when trying to acquire the self-position in real time, it is necessary to use a high-performance arithmetic device, and the power consumption may increase.

[0005] Therefore, an aspect of the present invention aims to provide a position estimation system capable of performing real-time position estimation. Another aspect of the present invention aims to provide a position estimation system with reduced power consumption. Another aspect of the present invention aims to provide an inexpensive position estimation system. Another aspect of the present invention aims to provide a novel position estimation system. Another aspect of the present invention aims to provide a position estimation method using the above position estimation system.

[0006] In addition, an aspect of the present invention aims to provide a position estimation device capable of performing real-time position estimation. Another aspect of the present invention aims to provide a position estimation device with reduced power consumption. Another aspect of the present invention aims to provide an inexpensive position estimation device. Another aspect of the present invention aims to provide a novel position estimation device. Another aspect of the present invention aims to provide a position estimation method using the above position estimation device.

[0007] Note that the description of these problems does not prevent the existence of other problems. It should be noted that an aspect of the present invention does not need to solve all of these problems. Other problems will become apparent from the descriptions in the specification, drawings, claims, etc., and it is possible to extract these other problems from the descriptions in the specification, drawings, claims, etc.

Means for Solving the Problems

[0008] One aspect of the present invention has a learning device and a position estimation device. The learning device has a comparison unit and a learning unit. The position estimation device has a data acquisition unit, an inference unit, a data conversion unit, and an evaluation unit. The data acquisition unit has a sensor. The comparison unit selects two types of machine learning data from among three or more types of machine learning data representing map information, and calculates a first translation amount and a first rotation amount by comparing the two types of machine learning data with each other. The learning unit has a function of generating a machine learning model by learning using the two types of machine learning data, the first translation amount, and the first rotation amount. The data acquisition unit has a function of acquiring acquisition data using the sensor. The inference unit has a function of inferring a second translation amount and a second rotation amount based on the acquisition data and one type of machine learning data selected from among three or more types of machine learning data using the machine learning model. The data conversion unit has a function of converting one type of machine learning data into evaluation data based on the second translation amount and the second rotation amount. The evaluation unit is a position estimation system having a function of evaluating the degree of coincidence between the acquisition data and the evaluation data.

[0009] Alternatively, one aspect of the present invention includes a learning device and a position estimation device. The learning device includes a first point cloud-image conversion unit, a comparison unit, and a learning unit. The position estimation device includes a point cloud data acquisition unit, a second point cloud-image conversion unit, an inference unit, a data conversion unit, and an evaluation unit. The first point cloud-image conversion unit has a function of converting n types (n is an integer of 3 or more) of machine learning point cloud data representing map information into n types of machine learning image data. The comparison unit has a function of selecting two types of machine learning point cloud data from the n types of machine learning point cloud data and calculating a first translation amount and a first rotation amount by comparing the two types of machine learning point cloud data with each other. The learning unit has a function of generating a machine learning model by learning using two types of machine learning image data corresponding to the two types of machine learning point cloud data, the first translation amount, and the first rotation amount. The point cloud data acquisition unit has a function of acquiring acquired point cloud data. The second point cloud-image conversion unit has a function of converting the acquired point cloud data into acquired image data. The inference unit has a function of inferring a second translation amount and a second rotation amount based on the acquired image data and one type of machine learning image data selected from the n types of machine learning image data using the machine learning model. The data conversion unit has a function of converting one type of machine learning point cloud data corresponding to one type of machine learning image data into evaluation point cloud data based on the second translation amount and the second rotation amount. The evaluation unit has a function of evaluating the degree of coincidence between the acquired point cloud data and the evaluation point cloud data. This is a position estimation system.

[0010] Alternatively, in the above aspect, the acquired image data and the machine learning image data may be binary data.

[0011] Alternatively, in the above aspect, the machine learning model may be a convolutional neural network model.

[0012] Alternatively, in the above aspect, the first translation amount and the first rotation amount may be calculated by scan matching.

[0013] Alternatively, one aspect of the present invention includes a data acquisition unit, an inference unit, a data conversion unit, and an evaluation unit. The data acquisition unit has a sensor and is configured to acquire acquisition data using the sensor. The inference unit is configured to infer a first translation amount and a first rotation amount based on the acquisition data and one type of machine learning data selected from among three or more types of machine learning data representing map information using a machine learning model. The machine learning model is generated by learning using two types of machine learning data selected from among three or more types of machine learning data, and a second translation amount and a second rotation amount calculated by comparing the two types of machine learning data with each other. The data conversion unit is configured to convert one type of machine learning data into evaluation data based on the first translation amount and the first rotation amount. The evaluation unit is a position estimation device configured to evaluate the degree of coincidence between the acquisition data and the evaluation data.

[0014] Alternatively, in the above aspect, the machine learning model may be a convolutional neural network model.

[0015] Alternatively, in the above aspect, the second translation amount and the second rotation amount may be calculated by scan matching.

[0016] A moving body having the position estimation device according to one aspect of the present invention and a battery is also one aspect of the present invention.

[0017] Alternatively, in the above aspect, the moving body may have a function of performing autonomous driving.

Advantages of the Invention

[0018] According to one aspect of the present invention, a position estimation system capable of estimating a position in real time can be provided. Further, according to one aspect of the present invention, a position estimation system with reduced power consumption can be provided. Further, according to one aspect of the present invention, an inexpensive position estimation system can be provided. Further, according to one aspect of the present invention, a novel position estimation system can be provided. Further, according to one aspect of the present invention, a position estimation method using the above position estimation system can be provided.

[0019] Further, according to one aspect of the present invention, a position estimation device capable of estimating a position in real time can be provided. Further, according to one aspect of the present invention, a position estimation device with reduced power consumption can be provided. Further, according to one aspect of the present invention, an inexpensive position estimation device can be provided. Further, according to one aspect of the present invention, a novel position estimation device can be provided. Further, according to one aspect of the present invention, a position estimation method using the above position estimation device can be provided.

[0020] Note that the effects of one aspect of the present invention are not limited to the effects listed above. The effects listed above do not prevent the existence of other effects. Note that other effects are effects not mentioned in this item as described below. Effects not mentioned in this item can be derived by those skilled in the art from the descriptions in the specification, drawings, etc., and can be appropriately extracted from these descriptions. Note that one aspect of the present invention has at least one of the effects listed above and / or other effects. Therefore, one aspect of the present invention may not have the effects listed above in some cases.

Brief Description of the Drawings

[0021]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

[0022] The embodiments will be described in detail with reference to the drawings. However, the present invention is not limited to the following description, and it will be easily understood by those skilled in the art that the form and details thereof can be variously changed without departing from the spirit and scope of the present invention. Therefore, the present invention is not to be construed as being limited to the description of the embodiments shown below.

[0023] Note that the position, size, range, etc. of each component shown in the drawings may not represent the actual position, size, range, etc. for the sake of simplicity of understanding. For this reason, the disclosed invention is not necessarily limited to the position, size, range, etc. disclosed in the drawings.

[0024] Also, the ordinal numbers "first", "second", "third", etc. used in this specification are attached to avoid confusion of components and are not numerically limiting.

[0025] (Embodiment) In this embodiment, a position estimation system according to an aspect of the present invention, a position estimation method using the position estimation system, and the like will be described with reference to the drawings.

[0026] <Configuration Example of Position Estimation System> FIG. 1 is a block diagram showing a configuration example of a position estimation system 10. The position estimation system 10 includes a learning device 20 and a position estimation device 30. Here, it is preferable that the learning device 20 be provided in a device having high computing power, such as a server. The learning device 20 and the position estimation device 30 can exchange data and the like with each other via a network or the like.

[0027] The learning device 20 includes an input unit 21, a point cloud-image conversion unit 22, a comparison unit 23, and a learning unit 24. The position estimation device 30 includes a data acquisition unit 31, an inference unit 34, a data conversion unit 35, and an evaluation unit 36. Here, the data acquisition unit 31 includes a point cloud data acquisition unit 32 and a point cloud-image conversion unit 33. Although not shown in FIG. 1, the learning device 20 and the position estimation device 30 can each have, for example, a storage unit. The storage unit can store data, programs, etc. used for driving the position estimation system 10, and each component of the position estimation system 10 can read these as needed.

[0028] In FIG. 1, the data exchange between the components of the position estimation system 10 is indicated by arrows. Note that the data exchange shown in FIG. 1 is an example, and data and the like can be exchanged, for example, between components not connected by arrows. Also, there may be cases where data is not exchanged between components connected by arrows.

[0029] First, the learning device 20 will be described. The input unit 21 has a function as an interface, and machine learning point cloud data PD ML is input. In one aspect of the present invention, n (where n is an integer of 3 or more) machine learning point cloud data PD ML is input to the input unit 21. The machine learning point cloud data PD ML can be, for example, point cloud data acquired by a device provided outside the position estimation system 10 and stored in a database. Thus, the machine learning point cloud data can be referred to as database point cloud data.

[0030] Point Cloud Data for Machine Learning PD ML can be obtained by, for example, a device having a laser and a sensor. Specifically, for example, by irradiating a laser beam and detecting the scattered laser beam with a sensor, the machine learning point cloud data PD ML In other words, for example, LiDAR (Light Detection and Ranging) can be used to obtain point cloud data PD ML The acquired point cloud data for machine learning PD ML represents map information and can also include information specifying positions on the map. In other words, the point cloud data for machine learning PD ML It can be said that the point cloud data PD for machine learning is data that represents map information including location information. ML can be supplied to the point cloud-to-image converter 22, the comparator 23, and the data converter 35.

[0031] The point cloud-to-image converter 22 has a function of converting point cloud data into image data. Specifically, the point cloud data for machine learning PD ML Image data GD for machine learning ML For example, the point cloud-image converter 22 converts the machine learning point cloud data PD ML The binary image data GD for machine learning is created by assigning "1" to coordinates that contain points and "0" to coordinates that do not contain points. ML As mentioned above, the machine learning point cloud data can be rephrased as database point cloud data. Therefore, the machine learning image data can be rephrased as database image data.

[0032] As mentioned above, the point cloud data for machine learning PD ML represents map information, and image data for machine learning GD ML is point cloud data for machine learning PD ML Therefore, the point cloud data for machine learning PD ML , and Image Data GD for Machine Learning ML can be called map data.

[0033] The comparison unit 23 has a function of calculating a translation amount and a rotation amount by extracting and comparing two pieces of point cloud data PD for machine learning input to the input unit 21. For example, when the point cloud data PD for machine learning is represented by a two-dimensional coordinate system (xy coordinate system), the comparison unit 23 can calculate the translation amount in the x-axis direction Δx1 and the translation amount in the y-axis direction Δy1 as the translation amount. Also, the comparison unit 23 can calculate the rotation amount θ1. ML From among the ML two pieces of point cloud data PD for machine learning ML If it is represented by a two-dimensional coordinate system (xy coordinate system), the comparison unit 23 can calculate the translation amount in the x-axis direction Δx1 and the translation amount in the y-axis direction Δy1 as the translation amount. Also, the comparison unit 23 can calculate the rotation amount θ1.

[0034] In the following, although the point cloud data and the image data will be described as being represented in a two-dimensional coordinate system, by increasing the number of dimensions of the translation amount and the rotation amount, etc., even when the point cloud data and the image data are represented in a three-dimensional coordinate system, the following description can be referred to. For example, when the point cloud data and the image data are represented in a three-dimensional coordinate system, the translation amount can be represented by a three-dimensional vector. Also, the rotation amount can be represented by a rotation vector, a rotation matrix, Euler angles, a quaternion, or the like. Note that when the point cloud data and the image data are represented in a three-dimensional coordinate system, the point cloud data and the image data can be data of a three-dimensional array.

[0035] In this specification, etc., when the point cloud data is represented in a two-dimensional coordinate system and the translation amount in the x-axis direction is Δx and the translation amount in the y-axis direction is Δy, the translation amount is represented as (Δx, Δy).

[0036] The calculation of the translation amount (Δx1, Δy1) and the rotation amount θ1 can be performed by scan matching, for example, by ICP (Iterative Closest Point) scan matching or NDT (Normal Distribution Transform) scan matching. The translation amount (Δx1, Δy1) and the rotation amount θ1 can be calculated so that, for example, the degree of coincidence of the two pieces of point cloud data GD for machine learning to be compared ML becomes the highest.

[0037] The learning unit 24 has a function of generating a machine learning model MLM. As the machine learning model MLM, for example, a multi-layer perceptron, a support vector machine, a neural network model, etc. can be applied. In particular, as the machine learning model MLM, it is preferable to apply a convolutional neural network (CNN).

[0038] The learning unit 24 uses the machine learning image data GD ML and the translation amount (Δx1, Δy1) and the rotation amount θ1 to perform learning and has a function of generating a machine learning model MLM. The generation of the machine learning model MLM can be performed, for example, by supervised learning. For example, the two machine learning point cloud data PD ML compared by the comparison unit 23, and the corresponding two machine learning image data GD ML are used as learning data, and the translation amount (Δx1, Δy1) and the rotation amount θ1 are associated with the learning data as correct labels to perform learning, thereby generating a machine learning model MLM.

[0039] The above is the description of the learning device 20.

[0040] Next, the position estimation device 30 will be described. The data acquisition unit 31 has a function of acquiring data. The data acquisition unit 31 has a function of acquiring, for example, the acquired point cloud data PD AC and the acquired image data GD AC . Although the details will be described later, the acquired image data GD AC can be acquired, for example, by converting the acquired point cloud data PD AC into image data.

[0041] The data acquired by the data acquisition unit 31 can be supplied to the inference unit 34 and the evaluation unit 36. The data acquisition unit 31 can supply, for example, the acquired point cloud data PD AC to the evaluation unit 36 and supply the acquired image data GD AC to the inference unit 34, for example.

[0042] The point cloud data acquisition unit 32 has a function of acquiring the acquired point cloud data PD AC It has, for example, a laser and a sensor. By irradiating the periphery of the position estimation device 30 with the laser and detecting the scattered laser light with the sensor, the acquired point cloud data PD AC can be acquired. That is, for example, by LiDAR, the acquired point cloud data PD representing the environmental information around the position estimation device 30 AC can be acquired by the point cloud data acquisition unit 32.

[0043] The point cloud-image conversion unit 33 has a function of converting point cloud data into image data. Specifically, the acquired point cloud data PD AC is converted into the acquired image data GD AC It has a function of conversion. The point cloud-image conversion unit 33 has a function of converting point cloud data into image data in the same manner as the method performed by the point cloud-image conversion unit 22. Specifically, the point cloud-image conversion unit 33, for example, converts the acquired point cloud data PD AC into binary acquired image data GD with coordinates containing points as "1" and coordinates not containing points as "0" AC It has a function of conversion.

[0044] The inference unit 34 has a function of performing inference based on the machine learning model MLM. Specifically, when the acquired image data GD AC and a piece of machine learning image data GD ML are input to the inference unit 34, it has a function of inferring the translation amount (Δx2, Δy2) and the rotation amount θ2 based on the machine learning model MLM.

[0045] The data conversion unit 35 has a function of converting the machine learning point cloud data PD ML corresponding to the machine learning image data GD input to the inference unit 34 ML into the evaluation point cloud data PD based on the translation amount (Δx2, Δy2) and the rotation amount θ2 E It has a function of conversion. Specifically, the data conversion unit 35, for the machine learning image data GD input to the inference unit 34 MLMachine learning point cloud data PD corresponding thereto ML By translating each point included in ML by (Δx2, Δy2) and rotating it by θ2, machine learning point cloud data PD ML is converted into evaluation point cloud data PD E and has a function of conversion.

[0046] The evaluation unit 36 has a function of calculating an evaluation value representing the degree of coincidence between the acquired point cloud data PD AC and the evaluation point cloud data PD E The evaluation value can be calculated by a method used in scan matching such as ICP scan matching or NDT scan matching. For example, for the points included in the acquired point cloud data PD AC and the points included in the evaluation point cloud data PD E corresponding to the points, the distance or the square of the distance is calculated for each point. The sum of the distances or the sum of the squares of the distances can be used as the evaluation value. In this case, the smaller the evaluation value, the higher the degree of coincidence between the acquired point cloud data PD AC and the evaluation point cloud data PD E can be considered.

[0047] When the degree of coincidence between the acquired point cloud data PD AC and the evaluation point cloud data PD E is low, it can be assumed that the position estimation device 30 exists at a position far from the location represented by the evaluation point cloud data PD E On the other hand, when the degree of coincidence between the acquired point cloud data PD AC and the evaluation point cloud data PD E is high, it can be assumed that the position estimation device 30 exists at a position close to the location represented by the evaluation point cloud data PD E From the above, by evaluating the degree of coincidence between the acquired point cloud data PD AC and the evaluation point cloud data PD E the position of the position estimation device 30 can be estimated.

[0048] The above is a configuration example of the position estimation system 10. The position estimation system 10 can calculate the translation amount (Δx2, Δy2) and the rotation amount θ2 by inference using the machine learning model MLM. Thereby, the computational amount of the position estimation device 30 can be reduced compared to the case of calculating the translation amount (Δx2, Δy2) and the rotation amount θ2 without using the machine learning model. Therefore, while estimating the position of the position estimation device 30 in real time, the power consumption of the position estimation device 30 can be reduced. In addition, since it is not necessary to use high-performance CPUs (Central Processing Units) and GPUs (Graphics Processing Units) that the position estimation device 30 has, the position estimation device 30 can be made inexpensive.

[0049] The position estimation device 30 can be applied to, for example, a moving body. The moving body can be, for example, an automobile. FIG. 2 shows an automobile 40 as an example of a moving body. As described above, the point cloud data acquisition unit 32 of the position estimation device 30 can be provided with a laser and a sensor. FIG. 2 shows a configuration example in which the automobile 40 has a laser 37 and a sensor 38.

[0050] In addition, a battery 41 is provided in the automobile 40. The power required for driving the position estimation device 30 can be supplied by the battery 41.

[0051] By applying the position estimation device 30 to a moving body, the position of the moving body can be estimated in real time. Therefore, the moving body to which the position estimation device 30 is applied can have a function of performing autonomous driving. As described above, the power consumption of the position estimation device 30 is low. Therefore, even when the moving body is provided with the function of performing autonomous driving by applying the position estimation device 30, it is possible to suppress a significant increase in the power consumption of the moving body compared to a moving body that does not have the function of performing autonomous driving. Specifically, it is possible to suppress a significant increase in the power consumption of the battery that the moving body has.

[0052] As described above, in the position estimation system 10, the point cloud data PD for machine learning ML is converted into the image data GD for machine learning ML and supplied to the learning unit 24 and the inference unit 34. Also, the acquired point cloud data PD AC is converted into the acquired image data GD AC and supplied to the inference unit 34. That is, in the position estimation system 10, the point cloud data is converted into image data, and machine learning is performed using the image data. Thereby, the machine learning model MLM can be, for example, a CNN. Note that machine learning may be performed using the point cloud data as it is without converting the point cloud data into image data.

[0053] FIG. 3 is a diagram showing a CNN that can be applied to the machine learning model MLM. The machine learning model MLM to which the CNN is applied has an input layer IL, an intermediate layer ML, and an output layer OL. The intermediate layer ML has a convolutional layer CL, a pooling layer PL, and a fully connected layer FCL. In FIG. 3, an example is shown in which the machine learning model MLM has m convolutional layers CL and m pooling layers PL (m is an integer of 1 or more) and two fully connected layers FCL. Note that the machine learning model MLM may have only one fully connected layer FCL or three or more fully connected layers FCL.

[0054] In this specification and the like, for example, a plurality of layers, data, etc. of the same type are described as [1], [2], [m], etc. for distinction. For example, the m convolutional layers CL are described as convolutional layer CL[1] to convolutional layer CL[m] for distinction.

[0055] The convolutional layer CL has a function of performing convolution on the data input to the convolutional layer CL. For example, the convolutional layer CL[1] has a function of performing convolution on the data input to the input layer IL. Also, the convolutional layer CL[2] has a function of performing convolution on the data output from the pooling layer PL[1]. Also, the convolutional layer CL[m] has a function of performing convolution on the data output from the pooling layer PL[m - 1].

[0056] Convolution is performed by repeatedly calculating the dot product of the data input to the convolutional layer CL and the weight filter. Through the convolution in the convolutional layer CL, feature extraction and the like are performed on the data input to the machine learning model MLM.

[0057] The data on which convolution has been performed is converted by an activation function and then output to the pooling layer PL. As the activation function, ReLU (Rectified Linear Units) or the like can be used. ReLU is a function that outputs "0" when the input value is negative and outputs the input value as it is when the input value is 0 or more. Also, as the activation function, a sigmoid function, a tanh function, or the like can be used.

[0058] The pooling layer PL has a function of performing pooling on the data input from the convolutional layer CL. Pooling is a process of dividing the data into a plurality of regions, extracting predetermined data for each region, and arranging it in a matrix form. Through pooling, the data volume can be reduced while retaining the features extracted by the convolutional layer CL. Also, the robustness against minute displacements of the input data can be enhanced. Note that as pooling, max pooling, average pooling, Lp pooling, or the like can be used.

[0059] The fully connected layer FCL has a function of combining the input data, converting the combined data by an activation function, and outputting it. As the activation function, ReLU, a sigmoid function, a tanh function, or the like can be used.

[0060] Note that the configuration of the machine learning model MLM to which CNN is applied is not limited to the configuration in FIG. 3. For example, a pooling layer PL may be provided for each of a plurality of convolutional layers CL. That is, the number of pooling layers PL included in the machine learning model MLM may be less than the number of convolutional layers CL. Also, when it is desired to retain the position information of the extracted features as much as possible, the pooling layer PL may not be provided.

[0061] By performing learning, the machine learning model MLM to which CNN is applied can optimize filter values of weight filters, weight coefficients of the fully connected layer FCL, and the like.

[0062] <An example of a position estimation method> Hereinafter, an example of a position estimation method using the position estimation system 10 will be described. Specifically, an example of a method for generating the machine learning model MLM by the learning device 20 and a method for estimating a position using the machine learning model MLM by the position estimation device 30 will be described. By the method shown below, for example, the position of the position estimation device 30 can be estimated.

[0063] [An example of a method for generating a machine learning model] FIG. 4 is a flowchart showing an example of a method for generating the machine learning model MLM. As shown in FIG. 4, the machine learning model MLM is generated by the method shown in steps S01 to S07.

[0064] In order to generate the machine learning model MLM, first, the point cloud data PD ML [1] to the point cloud data PD ML [n] are input to the input unit 21 (step S01). As described above, the point cloud data PD ML can be point cloud data representing map information including position information acquired by LiDAR or the like.

[0065] Next, the point cloud-image conversion unit 22 converts the point cloud data PD ML [1] to the point cloud data PD ML [n] into the machine learning image data GD ML [1] to the machine learning image data GD ML [n] respectively (step S02). FIG. 5 is a schematic diagram showing an example of the operation in step S02.

[0066] In step S02, the point cloud-image conversion unit 22 converts the point cloud data PD ML [1] to the point cloud data PD MLLet [n] be the binary machine learning image data GD where the coordinates containing points are "1" and the coordinates not containing points are "0". ML from [1] to the machine learning image data GD ML [n] is converted. In FIG. 5, the machine learning point cloud data PD ML from [1] to the machine learning point cloud data PD ML [n] is converted into the binary machine learning image data GD where the coordinates containing points are black and the coordinates not containing points are white ML from [1] to the machine learning image data GD ML [n]. An example of the conversion is shown.

[0067] Next, the comparison unit 23 sets the values of "i" and "j" (step S03). Thereafter, the machine learning point cloud data PD ML [i] and the machine learning point cloud data PD ML [j] are compared, and the translation amount (Δx1 i,j , Δy1 i,j ) and the rotation amount θ1 i,j are calculated (step S04). FIG. 6A is a schematic diagram showing an example of the operation in step S04. Here, i and j are integers of 1 or more and n or less, respectively. Also, i and j are different from each other. Here, the machine learning point cloud data PD ML [i] and the machine learning point cloud data PD ML [j] are preferably point cloud data representing positions close to each other. Specifically, at least a part of the location represented by the machine learning point cloud data PD ML [i] is preferably included in the machine learning point cloud data PD ML [j]. In step S03, the values of "i" and "j" may be set one or more at a time.

[0068] As described above, the calculation of the translation amount (Δx1 i,j , Δy1 i,j ) and the rotation amount θ1 i,j can be performed by scan matching, for example, by ICP scan matching or NDT scan matching. The translation amount (Δx1 i,j , Δy1 i,j ) and the rotation amount θ1i,j is, for example, point cloud data PD for machine learning ML [i] and point cloud data PD for machine learning ML [j] can be calculated so that the degree of coincidence is the highest.

[0069] After that, the learning unit 24 performs learning using the image data GD for machine learning ML [i] and the image data GD for machine learning ML [j], and the translation amount (Δx1 i,j , Δy1 i,j ), and the rotation amount θ1 i,j (step S05). Thereby, the learning unit 24 can generate a machine learning model MLM. FIG. 6B is a schematic diagram showing an example of the operation in step S05.

[0070] In this specification and the like, for example, data obtained by converting the point cloud data PD ML [i] into image data is defined as the image data GD for machine learning ML [i], and data obtained by converting the point cloud data PD ML [j] into image data is defined as the image data GD for machine learning ML [j]. Then, for example, the point cloud data PD ML [i] and the image data GD for machine learning ML [i] are referred to as corresponding data to each other. Also, the point cloud data PD ML [j] and the image data GD for machine learning ML [j] are referred to as corresponding data to each other. The same applies when other point cloud data is converted into image data.

[0071] As described above, the above learning can be, for example, supervised learning. For example, the image data GD for machine learning ML [i] and the image data GD for machine learning ML [j] are used as learning data, and the translation amount (Δx1 i,j , Δy1 i,j ) and the rotation amount θ1 i,j are associated with the learning data as correct labels, and by learning, the learning unit 24 can generate a machine learning model MLM.

[0072] Next, it is determined whether to end the learning (step S06). The learning may end when a predetermined number of times is reached. Alternatively, the learning may be terminated when a test is performed using the test data and the machine learning model MLM can correctly output the translation amount (Δx1 i,j , Δy1 i,j ) and the rotation amount θ1 i,j (when the output value of the loss function becomes equal to or less than the threshold value). Alternatively, the learning may end when the output value of the loss function saturates to a certain extent. Or, the user may specify the timing to end the learning.

[0073] If the learning is not ended, the operations shown in steps S03 to S06 are performed again. That is, the value of one or both of "i" and "j" is reset to a different value, and learning is performed.

[0074] If the learning is ended, the learning unit 24 outputs the learned machine learning model MLM (step S07). The learned machine learning model MLM is supplied to the position estimation device 30. Specifically, the learned machine learning model MLM is supplied to the inference unit 34 included in the position estimation device 30.

[0075] The above is an example of a method for generating the machine learning model MLM.

[0076] [Example of Position Estimation Method] FIG. 7 is a flowchart showing an example of a position estimation method using the machine learning model MLM. As shown in FIG. 7, the position of the position estimation device 30 is estimated by the method shown in steps S11 to S18.

[0077] To perform position estimation, first, the point cloud data acquisition unit 32 acquires acquisition point cloud data PD AC representing the environmental information around the position estimation device 30 (step S11). As described above, the point cloud data acquisition unit 32 can acquire the acquisition point cloud data PD AC by, for example, LiDAR.

[0078] Next, the point cloud-image conversion unit 33 converts the acquired point cloud data PD AC into the acquired image data GD AC (step S12). For example, the point cloud-image conversion unit 33 can convert the acquired point cloud data PD AC into the acquired image data GD AC by the same method as shown in FIG. 5. Specifically, the point cloud-image conversion unit 33 can convert the acquired point cloud data PD AC into binary acquired image data GD AC where the coordinates containing points are set to "1" and the coordinates not containing points are set to "0".

[0079] Thereafter, the inference unit 34 sets the value of "k" (step S13), and inputs the acquired image data GD AC and the machine learning image data GD ML [k] into the machine learning model MLM constructed in the inference unit 34. As a result, the translation amount (Δx2 k , Δy2 k ), and the rotation amount θ2 k are inferred (step S14). FIG. 8A is a schematic diagram showing an example of the operation in step S14. k is an integer of 1 or more and n or less.

[0080] Next, the data conversion unit 35 uses the translation amount (Δx2 k , Δy2 k ) and the rotation amount θ2 k to convert the machine learning point cloud data PD ML [k] into the evaluation point cloud data PD E [k] (step S15). FIG. 8B is a schematic diagram showing an example of the operation in step S15 and the like. As described above, the data conversion unit 35 can convert the machine learning point cloud data PD ML [k] into the evaluation point cloud data PD k , Δy2 k ) and rotating by θ2 k only. ML [k] into the evaluation point cloud data PD E [k].

[0081] In this specification and the like, the point cloud data PD for machine learning ML [k] and the evaluation point cloud data PD E [k] are referred to as corresponding data to each other.

[0082] After that, the evaluation unit 36 calculates an evaluation value representing the degree of coincidence between the acquired point cloud data PD AC and the evaluation point cloud data PD E [k]. Thereby, the degree of coincidence between the acquired point cloud data PD AC and the evaluation point cloud data PD E [k] is evaluated (step S16). FIG. 8B also shows an example of the operation in step S16.

[0083] As described above, the evaluation value can be calculated by a method used in scan matching such as ICP scan matching or NDT scan matching. By evaluating the degree of coincidence between the acquired point cloud data PD AC and the evaluation point cloud data PD E [k], the degree of coincidence between the acquired point cloud data PD AC and the machine learning point cloud data PD ML [k] can be evaluated. For example, if the points included in one of the two point cloud data are translated and rotated about a point so that the point cloud data coincides with the other point cloud data, the two point cloud data can be regarded as coinciding.

[0084] Next, it is determined whether or not the number of set values of "k" has reached a specified number (step S17). The specified number can be, for example, n. In this case, for all the machine learning point cloud data PD ML , the degree of coincidence with the acquired point cloud data PD AC can be evaluated. Also, the specified number may be smaller than n. In this case, for example, the value of "k" can be set so that the degree of coincidence with the acquired point cloud data PD ML can be evaluated for all the machine learning point cloud data PD AC used during learning.

[0085] If the number of set values of "k" has not reached the specified number, the operations shown in steps S13 to S17 are performed again. That is, the value of "k" is reset to a different value, and the acquired point cloud data PD AC and the point cloud data PD for machine learning ML [k] are evaluated for degree of coincidence.

[0086] When the number of set values of "k" has reached the specified number, the evaluation unit 36 estimates the position of the position estimation device 30 (step S18). For example, the acquired point cloud data PD AC and the point cloud data PD for machine learning with the highest degree of coincidence ML The position represented by can be set as the position of the position estimation device 30 that acquired the acquired point cloud data PD AC .

[0087] In step S17, even if the number of set values of "k" has not reached the specified number, if the degree of coincidence between the acquired point cloud data PD AC and the evaluation point cloud data PD E [k] is equal to or greater than the threshold value, it may proceed to step S18. In this case, the evaluation point cloud data PD AC whose degree of coincidence with the acquired point cloud data PD is equal to or greater than the threshold value E [k] can be set as the position of the position estimation device 30.

[0088] The above is an example of a position estimation method using the position estimation system 10. In the position estimation method using the position estimation system 10, the translation amount (Δx2, Δy2) and the rotation amount θ2 can be calculated by inference using the machine learning model MLM. Thereby, the amount of calculation by the position estimation device 30 can be reduced compared to the case of calculating the translation amount (Δx2, Δy2) and the rotation amount θ2 without using a machine learning model. Therefore, while estimating the position of the position estimation device 30 in real time, the power consumption of the position estimation device 30 can be reduced. In addition, since the CPU, GPU, etc. of the position estimation device 30 do not need to be high-performance, the position estimation device 30 can be made low-cost.

Explanation of Signs

[0089] 10: Position estimation system, 20: Learning device, 21: Input unit, 22: Point cloud-image conversion unit, 23: Comparison unit, 24: Learning unit, 30: Position estimation device, 31: Data acquisition unit, 32: Point cloud data acquisition unit, 33: Point cloud-image conversion unit, 34: Inference unit, 35: Data conversion unit, 36: Evaluation unit, 37: Laser, 38: Sensor, 40: Automobile, 41: Battery

Claims

【Claim 1】 A learning device and a position estimation device, The learning device includes a comparison unit and a learning unit, The position estimation device includes a data acquisition unit, an inference unit, a data conversion unit, and an evaluation unit, The data acquisition unit has a sensor, The comparison unit selects two types of machine learning data from among three or more types of machine learning data representing map information, and calculates a first translation amount and a first rotation amount by comparing the two types of machine learning data with each other. It has the function to do so, The learning unit has a function of generating a machine learning model by learning using the two types of machine learning data, the first translation amount, and the first rotation amount, The data acquisition unit has a function of acquiring acquisition data using the sensor, The inference unit has a function of inferring a second translation amount and a second rotation amount based on the acquisition data and one type of machine learning data selected from among the three or more types of machine learning data using the machine learning model, The data conversion unit has a function of converting the one type of machine learning data into evaluation data based on the second translation amount and the second rotation amount, The evaluation unit is a position estimation system having a function of evaluating the degree of coincidence between the acquisition data and the evaluation data.

Images