Model generation method and inference program

Abstract

A model generation device according to one aspect of the present invention manages execution of machine learning of a first model and a second model. The first model is configured to infer a position of an intermediate point present between two points in a given space. The second model is configured to infer a distance between the two points. The machine learning includes training the first model to obtain a capability of inferring the position of the intermediate point by using the result of inferring the distance by the second model, and training the second model to obtain a capability of inferring the distance by using the result of inferring the intermediate point by the first model and metrics of the space.

Description

Model generation method and inference program

[0001] The present invention relates to a model generation method and an inference program.

[0002] Problems such as path planning, state transition planning, and trajectory generation can be considered as examples of problems for generating a geodesic line between two points. One example of a method for solving the problem of generating a geodesic line is a method for generating an inference model for inferring waypoints using goal-conditional reinforcement learning. According to this method, a geodesic line can be generated by sequentially inferring waypoints from a start point to an end point using the generated inference model. Another example of a method for solving the problem of generating a geodesic line is the method proposed in Non-Patent Document 1. According to the method proposed in Non-Patent Document 1, a geodesic line can be generated by repeatedly inferring subgoals as waypoints using the generated inference model.

[0003] Tom Jurgenson, Edward Groshev, Aviv Tamar, “Sub-Goal Trees -- a Framework for Goal-Directed Trajectory Prediction and Optimization”, [online], [Retrieved September 22, 2020], Internet <URL: https: / / arxiv.org / abs / 1906.05329>

[0004] The present inventors have discovered the following problems with the conventional methods. Specifically, when using a goal-conditional reinforcement learning method, if a reward is given only when the goal is reached, the rewards become sparse, making learning difficult. While a solution to this problem would be to give a reward each time the goal is approached, this method complicates the design of the reward function. This makes it difficult to easily generate an inference model. On the other hand, the method of Non-Patent Document 1 makes it possible to easily generate an inference model. However, in the method of Non-Patent Document 1, a subgoal is any point between two points. Therefore, there is a possibility that a sufficiently dense set of waypoints cannot be obtained. If a sufficiently dense set of waypoints cannot be obtained, various problems can arise. For example, in route planning, it becomes difficult to obtain the shortest route if a sufficiently dense set of waypoints cannot be obtained.

[0005] In one aspect, the present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique for obtaining sufficiently dense waypoints when generating a geodesic line.

[0006] In order to solve the above-mentioned problems, the present invention employs the following configurations. Note that the following configurations of the invention can be combined as appropriate.

[0007] That is, a model generation method according to one aspect of the present invention is an information processing method in which a computer executes steps of directing the execution of machine learning of a first model and a second model and outputting the results of the machine learning. The first model is configured to estimate the position of an intermediate point between two points in a given space based on the positions of the two points. The second model is configured to estimate the distance between the two points based on the positions of the two points. The machine learning includes training the first model to acquire the ability to estimate the position of the intermediate point using a distance estimation result by the second model, and training the second model to acquire the ability to estimate the distance using the intermediate point estimation result of the first model and a metric of the space.

[0008] The intermediate point may be a point that exists between two points and whose ratio of distances from the two points satisfies a predetermined condition (i.e., a ratio that divides the distance between the two points internally is defined). The predetermined ratio (ratio of distances) may vary. According to this configuration, a trained first model that has acquired the ability to estimate an intermediate point that exists between two points in a given space can be generated. By using the generated trained first model to recursively and repeatedly estimate intermediate points located at the predetermined ratio, a geodesic line can be generated while densely obtaining waypoints. Therefore, according to this configuration, a sufficiently dense number of waypoints can be obtained when generating a geodesic line.

[0009] In the model generation method according to the above aspect, the intermediate points may be midpoints. With this configuration, it is possible to obtain sufficiently dense and uniform waypoints.

[0010] In the model generation method according to the above aspect, training the first model may include sampling two points consisting of a first sample point and a second sample point from the space, estimating a first position of a midpoint between the sampled first sample point and the second sample point using the first model, estimating a first distance between the first sample point and the estimated first position using the second model, estimating a second distance between the estimated first position and the second sample point using the second model, and training the first model so that the estimated first distance and the second distance satisfy a condition for the midpoint. This configuration makes it possible to appropriately generate a trained first model.

[0011] In the model generation method according to the above aspect, training the second model includes: (1) sampling two points consisting of a third sample point and a fourth sample point from the space; (2) using the second model to estimate a third distance between the sampled third sample point and the fourth sample point; (3) training the second model so that, if the third sample point and the fourth sample point satisfy a proximity condition, the estimated third distance is equal to a value given by a metric of the space; and (4) if the third sample point and the fourth sample point do not satisfy the proximity condition, (4-1) using the first model to estimate a third distance between the sampled third sample point and the fourth sample point. (4-2) adopting the combination of the third sample point and the intermediate point, and the combination of the intermediate point and the fourth sample point as new combinations of the third sample point and the fourth sample point, respectively, and repeating the process from (2), and (4-3) after repeating from (2) using (4-2), training the second model so that the estimated value of the third distance between the third sample point and the fourth sample point sampled in (1), estimated using the second model, is equal to the sum of values ​​given by a metric of the space. This configuration makes it possible to appropriately generate a trained second model.

[0012] In the model generation method according to the above aspect, the spatial metric may include a cost function. With this configuration, it is possible to generate a geodesic curve taking into account a cost element defined by the cost function.

[0013] Furthermore, the present invention may not be limited to the above-described model generation. One aspect of the present invention may be an inference program that uses at least one of the trained first model and second model generated by the above-described model generation method. Furthermore, the application of the configuration for obtaining dense waypoints by recursively estimating the waypoints may not be limited to the application of the trained first model generated by the above-described aspects.

[0014] For example, an inference program according to one aspect of the present invention may be a program for causing a computer to execute the following steps: using an inference model to infer the position of a midpoint between a first point and a second point given in space, the inference model being configured to infer, from the positions of the two points, the position of a midpoint between the two points and having a predetermined ratio of distances from the two points; generating a geodesic line between the first point and the second point, the geodesic line being composed of a plurality of the inferred midpoints, by repeatedly recursively executing the inference step; and outputting information about the generated geodesic line. With this configuration, a sufficiently dense set of waypoints can be obtained when generating a geodesic line. The inference model may be a trained first model generated by the model generation method according to any of the above aspects.

[0015] In the inference program according to the above aspect, the geodesic curve may be a path. With this configuration, when performing path planning, it is possible to obtain sufficiently dense waypoints, thereby enabling the expectation of appropriate path planning.

[0016] In the inference program according to the above aspect, the path may be a motion path of a robot device. With this configuration, when planning a motion path of the robot device, a sufficiently dense number of waypoints can be obtained, thereby making it possible to expect the acquisition of an appropriate motion path.

[0017] In the inference program according to the above aspect, the route may be a route of a moving object. With this configuration, when planning a route of the moving object, a sufficiently dense number of waypoints can be obtained, thereby making it possible to expect the acquisition of an appropriate route.

[0018] In the inference program according to the above aspect, each point in the space may be configured to represent a state of a device. The geodesic curve may represent a state transition of the device. With this configuration, when inferring a state transition of the device, sufficiently dense waypoints can be obtained, thereby making it possible to expect the acquisition of an appropriate state transition.

[0019] In the inference program according to the above aspect, the device may be a robot device or a mobile object. With this configuration, when inferring the state transition of the robot device or the mobile object, sufficiently dense waypoints can be obtained, thereby making it possible to expect the acquisition of appropriate state transitions.

[0020] In the inference program according to the above aspect, each point in the space may be configured to represent an image. The geodesic curve may represent a transition of the image. With this configuration, when inferring an image transition, sufficiently dense waypoints can be obtained, thereby enabling the acquisition of an appropriate image transition (morphing).

[0021] In the inference program according to the above aspect, each point in the space may be configured to represent a state of an element provided by a simulator. The geodesic curve may represent a trajectory of a state of the element that conforms to physical laws. With this configuration, when inferring a trajectory of a state that conforms to physical laws, sufficiently dense waypoints can be obtained, thereby making it possible to expect the acquisition of appropriate state transitions (e.g., simulation results of physical laws).

[0022] Furthermore, for example, an inference program according to one aspect of the present invention may be a program for causing a computer that stores the trained second model generated by the model generation method of any of the above aspects to execute the steps of: estimating the distance between two given points in space using the trained second model; and outputting information related to the estimated distance. With this configuration, it is possible to estimate the distance between two points.

[0023] Note that the present invention does not need to be limited to the above-described model generation method. As another aspect of the model generation method according to each of the above aspects, one aspect of the present invention may be an information processing device (model generation device) that realizes all or part of the above-described configurations, a program, or a storage medium readable by a machine such as a computer on which such a program is stored. A storage medium readable by a machine such as a computer is a medium that stores information such as a program through electrical, magnetic, optical, mechanical, or chemical action. Similarly, the present invention does not need to be limited to the above-described inference program. As another aspect of the inference program according to each of the above aspects, one aspect of the present invention may be an information processing device (inference device) that realizes all or part of the above-described configurations, an information processing method (inference method), or a storage medium readable by a machine such as a computer on which a program is stored.

[0024] According to the present invention, a technique can be provided for obtaining sufficiently dense waypoints when generating geodesic lines.

[0025] FIG. 1 schematically illustrates an example of a scenario to which the present invention is applied. FIG. 2A schematically illustrates an example of a training process for a first model according to an embodiment. FIG. 2B schematically illustrates an example of a training process for a second model according to an embodiment. FIG. 2C illustrates an example of pseudo program code for a training process for a first model and a second model according to an embodiment. FIG. 3 schematically illustrates an example of a geodesic line generation process according to an embodiment. FIG. 4 schematically illustrates an example of a hardware configuration of a model generation device according to an embodiment. FIG. 5 schematically illustrates an example of a hardware configuration of a (first) inference device according to an embodiment. FIG. 6 schematically illustrates an example of a hardware configuration of a (second) inference device according to an embodiment. FIG. 7 schematically illustrates an example of a software configuration of a model generation device according to an embodiment. FIG. 8 schematically illustrates an example of a software configuration of a (first) inference device according to an embodiment. FIG. 9 schematically illustrates an example of a software configuration of a (second) inference device according to an embodiment. FIG. 10 is a flowchart illustrating an example of a processing procedure of a model generation device according to an embodiment. Fig. 11 is a flowchart showing an example of a processing procedure of a (first) inference device according to an embodiment. Fig. 12 is a flowchart showing an example of a processing procedure of a (second) inference device according to an embodiment. Fig. 13 shows the results of a first experiment. Fig. 14 shows the results of generating geodesic lines in the first experiment. Fig. 15 shows the results of a second experiment.

[0026] An embodiment according to one aspect of the present invention (hereinafter also referred to as "the present embodiment") will be described below with reference to the drawings. However, the present embodiment described below is merely an example of the present disclosure in all respects. Various improvements or modifications may be made without departing from the scope of the present disclosure. In implementing the present disclosure, specific configurations according to the embodiment may be appropriately adopted. Note that while data appearing in the present embodiment is described in natural language, more specifically, it is specified in computer-recognizable pseudo-language, commands, parameters, machine language, etc.

[0027] §1 Application Example Fig. 1 shows a schematic diagram of an example of a situation in which the present invention is applied. The system according to this embodiment comprises a model generation device 1, an inference device 2, and an inference device 3. For convenience of explanation, the inference device 2 may be referred to as the "first inference device," and the inference device 3 may be referred to as the "second inference device."

[0028] The model generation device 1 is one or more computers configured to generate a trained first model 4 and a trained second model 5 through machine learning. In this embodiment, the model generation device 1 directs the execution of machine learning for the first model 4 and the second model 5. The first model 4 is configured to estimate the position of a midpoint between two points in a given space based on the positions of the two points. The midpoint may be a point between the two points and have a predetermined ratio of distances from the two points. The distance ratio (predetermined ratio) may be variable. The second model 5 is configured to estimate the distance between the two points based on the positions of the two points. The machine learning includes training the first model 4 to acquire the ability to estimate the position of the midpoint using the distance estimation result by the second model 5, and training the second model 5 to acquire the ability to estimate distance using the midpoint estimation result by the first model 4 and a spatial metric. This generates the trained first model 4 and the trained second model 5. The model generation device 1 outputs the results of the machine learning.

[0029] The inference device 2 is one or more computers configured to generate a geodesic line by repeatedly estimating intermediate points. In this embodiment, the inference device 2 uses an inference model 6 to estimate the position of an intermediate point between a first point and a second point given in space. The inference model 6 is configured to estimate the position of an intermediate point that exists between two points and has a predetermined ratio of distances from the two points, from the positions of the two points. In one example, the inference model 6 may use the trained first model 4. The inference device 2 recursively executes the step of estimating intermediate points by adding the estimated intermediate points as new points. In this way, the inference device 2 generates a geodesic line between the first point and the second point, which is composed of the estimated intermediate points. The inference device 2 outputs information about the generated geodesic line.

[0030] The inference device 3 is one or more computers configured to infer the distance between two points using a trained second model 5. In this embodiment, the inference device 3 holds the trained second model 5 generated by the model generation device 1. The inference device 3 infers the distance between two given points in space using the trained second model 5. The inference device 3 outputs information related to the inferred distance.

[0031] As described above, in this embodiment, the model generation device 1 can generate a trained first model 4 that has acquired the ability to estimate intermediate points between two points in a given space. Using the generated trained first model 4, a geodesic line can be generated while densely obtaining waypoints by repeatedly recursively estimating intermediate points located at a predetermined ratio. Therefore, according to this embodiment, when generating a geodesic line, sufficiently dense waypoints can be obtained. The inference device 2 employs a method of recursively estimating intermediate points located at a predetermined ratio using the inference model 6 as a geodesic line generation method, thereby obtaining sufficiently dense waypoints. The inference device 3 can estimate the distance between two points by using the trained second model 5. Note that obtaining dense waypoints corresponds to a small distance between waypoints. Whether the distance between waypoints is small or not may be evaluated using, for example, a threshold value.

[0032] (Directing the Execution of Machine Learning) Directing the execution of machine learning may include executing machine learning within the model generation device 1 and giving instructions to an external computer to execute machine learning. That is, in one example, the model generation device 1 may execute machine learning for the first model 4 and the second model 5. In another example, the model generation device 1 may execute machine learning for one of the first model 4 and the second model 5, and an external computer may execute machine learning for the other. In another example, the model generation device 1 may execute part of the machine learning processing for the first model 4 and the second model 5, and an external computer may execute the remaining processing in response to instructions from the model generation device 1. In another example, an external computer may execute machine learning for the first model 4 and the second model 5 in response to instructions from the model generation device 1. The external computer that executes the machine learning for the first model 4 and the external computer that executes the machine learning for the second model 5 may be the same or different. Information on the first model 4 and the second model 5 may be stored in any device (the model generation device 1, the external computer).

[0033] (Outputting Machine Learning Results) Outputting the machine learning results may be configured by outputting any information related to the machine learning results. In one example, outputting the machine learning results may include generating learning result data indicating the machine learning results and storing the generated learning result data in any storage area. In another example, outputting the machine learning results may include outputting a convergence result of training in machine learning. The convergence result may be indicated by, for example, a loss, a number of time steps, a learning curve, etc.

[0034] (Midpoint) A midpoint may be a point where the ratio of distances from two points defined using a metric is predetermined (satisfies a predetermined condition), i.e., a point where a ratio that divides the distance between two points defined using a metric is defined. In one example, the predetermined ratio (ratio of distances) may be 1:1, thereby making the midpoint the midpoint. The midpoint may be a point where the distances from two points defined using a metric are equal. The distance may include a quasi-pseudo-distance. According to one example of the present embodiment, by repeatedly and recursively estimating the midpoint, it is possible to expect the acquisition of sufficiently dense and uniform waypoints. However, the predetermined ratio is not limited to this example and may be selected appropriately depending on the embodiment, such as 1:2, 2:3, etc. Furthermore, the predetermined ratio may be fixed or may vary dynamically. The predetermined ratio may be set within a numerical range.

[0035] (Model Configuration) The first model 4, the second model 5, and the inference model 6 are configured as machine learning models. The machine learning models are configured to have one or more calculation parameters that can be adjusted by machine learning. The one or more calculation parameters are used for the target inference calculation. Machine learning involves adjusting (optimizing) the values ​​of the calculation parameters using training samples. The machine learning models may be configured, for example, by neural networks, regression models, decision tree models, support vector machines, other functional formulas (calculation models), etc. The machine learning method may be selected appropriately depending on the machine learning model employed (for example, backpropagation, etc.).

[0036] In one example, at least one of the first model 4, the second model 5, and the inference model 6 may include a neural network. The structure of the neural network may be determined appropriately depending on the embodiment, and may be specified, for example, by the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) included in each layer, the connection relationships between the nodes in each layer, etc. In one example, the neural network may have a recursive structure. Furthermore, the neural network may include any layer, such as a fully connected layer, a convolutional layer, a pooling layer, a deconvolutional layer, an unpooling layer, a normalization layer, a dropout layer, or a long short-term memory (LSTM). The neural network may include any type of model, such as a diffusion model or a Transformer. The connection weights between each node included in the neural network and the threshold value of each node are examples of calculation parameters.

[0037] In this embodiment, the first model 4 and the inference model 6 are configured to receive input of information on two points and output an estimation result of the midpoint between the two points. The second model 5 is configured to receive input of information on two points and output an estimation result of the distance between the two points. As long as they are configured in this manner, the input / output forms of the first model 4, the second model 5, and the inference model 6 are not particularly limited and may be selected appropriately depending on the embodiment. At least one of the first model 4, the second model 5, and the inference model 6 may be configured to further receive input of information other than the positions of the two points. At least one of the first model 4 and the inference model 6 may be configured to further output any information other than the result of estimating the position of the midpoint. The second model 5 may be configured to further output any information other than the result of estimating the distance between the two points.

[0038] (Training of First Model / Second Model) In this embodiment, the first model 4 is trained using the inference result (distance estimation result) of the second model 5. The second model 5 is trained using the spatial metric and the inference result (midpoint estimation result) of the first model 4. As long as the configuration is as described above, the training method for the first model 4 and the second model 5 is not particularly limited and may be designed appropriately depending on the embodiment. In one example, an actor-critic method may be used as a training method for the first model 4 and the second model 5 by treating the first model 4 as an actor and the second model 5 as a critic. In this case, machine learning of the second model 5 may be performed using the inference result of the first model 4 after it has been updated (on-policy). Alternatively, machine learning of the second model 5 may be performed using the inference result of the first model 4 before it has been updated (off-policy).

[0039] 2A schematically illustrates an example of a training process for the first model 4 according to this embodiment. In this example, training the first model 4 may include the following steps (A) to (E): (A) sampling two points, consisting of a first sample point (s) and a second sample point (g), from the space SP; (B) using the first model 4 to estimate a first position of a midpoint (m) between the sampled first sample point (s) and the second sample point (g); (C) using the second model 5 to estimate a first distance (d(s,m)) between the first sample point (s) and the estimated midpoint (m); (D) using the second model 5 to estimate a second distance (d(m,g)) between the estimated midpoint (m) and the second sample point (g); and (E) training the first model 4 such that the estimated first distance (d(s,m)) and second distance (d(m,g)) satisfy a midpoint condition.

[0040] According to an example of this embodiment, a trained first model 4 can be appropriately generated. Note that the loss function used in training the first model 4 in (E) may be designed appropriately depending on the embodiment. In one example, the loss function may include a term for learning the ability to estimate an intermediate point (m) that satisfies "αd(s,m) = d(m,g)." This allows the first model 4 to acquire the ability to estimate the position of an intermediate point whose distance ratio from two points is α. α may be set according to a predetermined ratio. For example, if the intermediate point is the midpoint, α may be 1. α may be set to a fixed value or a variable value. The value of α may be set within a numerical range. Furthermore, the loss function may include a term for learning the ability to estimate an intermediate point (m) that satisfies "d(s,m) + d(m,g) is minimum." This allows the first model 4 to acquire the ability to estimate the position of an intermediate point on the shortest path.

[0041] 2B schematically illustrates an example of a training process for the second model 5 according to this embodiment. In this example, training the second model 5 may include the following steps (1) to (4). (1) sampling two points consisting of a third sample point (s) and a fourth sample point (g) from the space SP; (2) using a second model 5 to estimate a third distance (d(s, g)) between the sampled third sample point (s) and the fourth sample point (g); (3) training the second model 5 so that, if the third sample point (s) and the fourth sample point (g) satisfy a proximity condition, the estimated third distance (d(s, g)) is equal to the value (c) given by the metric MT of the space SP; and (4) if the third sample point (s) and the fourth sample point (g) do not satisfy the proximity condition, - (4-1) using the first model 4 to estimate a second position of a midpoint (m) between the sampled third sample point (s) and the fourth sample point (g); - (4-2) Repeating the process from (2) above by adopting the combination of the third sample point (s) and the midpoint (m), and the combination of the midpoint (m) and the fourth sample point (g) as new combinations of the third sample point (s) and the fourth sample point (g), respectively; and - (4-3) After repeating the process from (2) according to (4-2), training the second model 5 so that the estimated value of the third distance (d(s, g)) between the (initial) third sample point (s) and the fourth sample point (g) sampled in (1), estimated using the second model 5, is equal to the sum of the values ​​(c) given by the metric MT of the space SP.

[0042] According to an example of this embodiment, a trained second model 5 can be appropriately generated. Note that the loss function in training the second model 5 in (3) may be designed appropriately depending on the embodiment. Furthermore, the first and second sample points sampled in training the first model 4 may be the same as or different from the third and fourth sample points sampled in training the second model 5. Furthermore, similar to the training of the second model 5, in training the first model 4, the combination of the first sample point and the midpoint estimated in (B) and the combination of the midpoint and the second sample point may be adopted as new combinations of the first and second sample points, respectively, and the process of (B) above may be recursively repeated. Then, the processes of (C) to (E) above may be performed on the first sample points, midpoints, and second sample points obtained by the recursive process. In one example, when the first sample point and the second sample point are the same as the third sample point and the fourth sample point, the process (4-1) of the second model 5 and the process (B) of the first model 4 may be executed in common in accordance with the process (4) in the machine learning of the second model 5. Then, the training processes (C) to (E) of the first model 4 may be executed together with the training process (3) of the second model 5.

[0043] Furthermore, the proximity conditions (3) and (4) in the machine learning of the second model 5 may be appropriately designed depending on, for example, the accuracy of generating geodesic curves. In one example, the number of recursions (4) in the machine learning of the second model 5 may be appropriately specified depending on the settings of the machine learning conditions. Accordingly, if the number of times (4) is processed reaches the specified number of recursions, it may be determined that the third sample point and the fourth sample point satisfy the proximity condition. On the other hand, if the number of times (4) is processed does not reach the specified number of recursions, it may be determined that the third sample point and the fourth sample point do not satisfy the proximity condition. The number of recursions may be specified by a user or by computer processing. Furthermore, the number of recursions may be a constant value or a variable value. For example, in the early stages of machine learning, the inference accuracy of both the first model 4 and the second model 5 is low, so even if the number of recursions is increased, it may be difficult to properly train them. Therefore, the number of recursions may be set low in the early stages of machine learning, and the number of recursions may be increased as the machine learning progresses. In another example, whether the proximity condition is satisfied may be determined based on a comparison between the third distance estimated in (2) and a threshold value.

[0044] Furthermore, the processes (2) to (4) in the machine learning of the second model 5 correspond to the following processes (2A) to (4A): (2A) recursively using the first model 4 to estimate a plurality of intermediate points (as waypoints) between the sampled third and fourth sample points, (3A) for each two adjacent points among the third sample point, the estimated intermediate points, and the fourth sample point, obtaining an estimated value of the distance between the third and fourth sample points using the metric MT of the space SP, and summing the obtained estimated values, and (4A) training the second model so that the value of the third distance between the third and fourth sample points estimated by the second model 5 is equal to the estimated value of the distance between the third and fourth sample points calculated using the metric of the space.

[0045] That is, training the second model 5 may include the above processes (1), (2A), (3A), and (4A). Furthermore, training the second model 5 may include performing processes (3A) and (4A) using the obtained midpoints each time a midpoint is recursively estimated in process (2A).

[0046] Furthermore, the training method for the first model 4 and the second model 5 is not limited to the above example, and may be changed as appropriate depending on the embodiment. In another example, training the second model 5 may include the following (3A) and (4A) instead of the above (3) and (4). (3A) If the third sample point and the fourth sample point are evaluated to be sufficiently close, training the second model 5 so that a third distance (d(s, g)) between the third sample point and the fourth sample point is equal to a value given by a spatial metric; (4A) If the third sample point and the fourth sample point are not evaluated to be sufficiently close, - (4A-1) using the first model 4 to estimate a second position of a midpoint between the third sample point and the fourth sample point; - (4A-2) using the second model 5 to estimate a fourth distance (d(s, m)) between the third sample point and the midpoint and a fifth distance (d(m, g)) between the midpoint and the fourth sample point; - (4A-3) training the second model 5 so that the sum of the fourth distance (d(s, m)) and the fifth distance (d(m, g)) is equal to the third distance (d(s, g)). Note that the criterion for whether or not something is sufficiently close may be determined appropriately depending on the accuracy with which the geodesic is generated.

[0047] The method proposed in Non-Patent Document 1 employs a policy gradient method for training a model that predicts subgoals. In this policy gradient method, first, predictions of first-stage subgoals between two points are performed probabilistically multiple times. Second-stage subgoals are further predicted between each of the obtained first-stage subgoals and each of the two points. Using these second-stage subgoals, each first-stage subgoal is evaluated (i.e., model training). In other words, to evaluate the nth-stage subgoal, a process for predicting the n+1th-stage subgoal is performed. This may result in a decrease in the efficiency of the overall training process. In contrast, in the machine learning method disclosed herein, the inference results of the second model 5 are used to train the first model 4. This allows the training process of the first model 4 to be performed without performing recursive processing. Therefore, according to one example of this embodiment, the efficiency of the training process of the first model 4 can be improved.

[0048] Note that the space (space SP) may include any kind of space in which the length of a curve is defined, such as a Euclidean space, a Riemannian manifold, or a Finsler manifold. In one example, a metric that satisfies a trigonometric inequality may be defined in the space. For example, let X be a space. The quasi-metric of X may be defined as d: X × X → [0, ∞] that satisfies the following (Equation 1) and (Equation 2).

[0049] x, y, z are any points in space (X). If "d(x, z) = d(z, y) = d(x, y) / 2", then z is the midpoint of x and y. If "d(x, y) > 0" for any x and y (x ≠ y), then d may be called a quasi-metric.

[0050] The metric (metric MT) defines the distance (length) between two nearby points within a desired accuracy range. The distance may include a quasi-pseudo distance. In one example, the metric may be defined to provide an approximate value of the distance between two points using an approximation method. The space and the metric may be expressed in any manner, such as a function or numerical data. The space may include a real space, and the metric may be defined to provide an actual measurement value or an approximate value of the real space. Points in the space may be associated with any type of data, such as images, sounds, or various sensor data, and the metric may be defined to provide the distance between the data. Points in the space may also be associated with any physical state, and the metric may be defined to reproduce physical laws (e.g., the refractive index of light, kinetic energy, etc.). The present disclosure may be directly applied to the problem of generating geodesic curves, or may be applied to any task that can be considered as a problem of generating geodesic curves, such as path planning, state transition planning, and trajectory generation. The space and the metric may be defined appropriately depending on the task to be solved as a problem of generating geodesic curves.

[0051] As an example of a space, we will show an example of a pseudo-Finsler manifold. A pseudo-Finsler manifold is a differential manifold M with F:TM → [0,∞), where TM = ∪. ｘ∈Ｍ T ｘ M is the tangent bundle of M. T is the tangent space, and Tx is the set of tangent vectors emanating from point x. A quasi-Finsler manifold satisfies the following conditions (Equation 3) and (Equation 4).

[0052]

[0053] Let γ: [0, 1] → M be a differentiable curve on M. The length L of γ can be defined as follows (Equation 5): Then, for two points x, y∈M, the distance d can be defined as follows (Equation 6):

[0054] d is a quasi-metric, i.e., when taking a quasi-Finsler manifold as a given space, the metric of the space may be defined by (Equation 6) above.

[0055] (Example of the training process) Let (X, d) be a quasi-metric space. Assume that the distance is locally known by a function C: X × X → [0, ∞].

[0056] It is assumed that whenever d(x, y) = 0, C(x, y) = 0. The function C is given appropriately to define the distance locally. For example, in a pseudo-Finsler manifold, the coordinate system f:M → R ｎ is given, the distance can be calculated by the following (Equation 7).

[0057] Under this condition, suppose a first model 4 and a second model 5 are trained for space (X).

[0058] In one example of this embodiment, the first model 4 may be treated as an actor and the second model 5 as a critic, and training of the first model 4 and the second model 5 may be performed simultaneously. The first and second sample points may be the same as the third and fourth sample points. Furthermore, the processing of (4-1) for the second model 5 and (B) for the first model 4 may be performed in common, and the training processing of (C) to (E) for the first model 4 may be performed together with the training processing of (3) for the second model 5. The midpoint may be the midpoint.

[0059] FIG. 2C shows an example of pseudo program code for the training process of the first model 4 and the second model 5 when these modes are adopted. In the example of FIG. 2C, the first model 4 is represented by a function π, and the parameters of the first model 4 are represented by θ. The second model 5 is represented by a function V, and the parameters of the second model 5 are represented by φ. The loss (L critic ) may be defined as the following (Equation 8), and the loss of the first model 4 (L actor ) may be defined as follows (Equation 9):

[0060]

[0061] The first term in (Equation 8) corresponds to the training of the second model 5 in (3) and (4-3) above. The second and third terms in (Equation 8) are added to ensure that V(x, x) = 0. On the other hand, the first and second terms in (Equation 9) aim to make π(s, g) the midpoint of s and g, and correspond to the training of the first model 4 in (E) above. Note that d(x, z) ２ +d(z, y) ２ takes a minimum value when z is the midpoint of x and y, as shown in the following (Equation 10): The third and fourth terms in (Equation 9) are added to ensure that V(x,π(x,x)) = V(π(x,x),x) = 0.

[0062]

[0063] The sample data used for training may be collected by the first model 4 with the current parameters. Two points (p ０，０ , p ０，１ ), and a sequence of points may be generated between the initial two points by repeatedly inserting a midpoint between two adjacent points using the first model 4 (i.e., recursively executing the processes of (4-1) and (B) above). Pairs of two adjacent points in each iteration may be collected as sample data to be used for training. The depth of the iteration (number of iterations) D does not necessarily need to be constant, and may be determined by some schedule. The distance (estimated value) of the collected pairs may be simply calculated as the sum of the values ​​of C given by the following (Equation 11):

[0064]

[0065] That is, the Monte Carlo method may be used to train the first model 4 and the second model 5. After collecting sample data, the collected data is divided into mini-batches, and the above-mentioned losses (L actor , L critic ) for each mini-batch. ｅｐｏｃｈｓ The data collection and optimization process may be repeated a sufficient number of times to generate trained first and second models 4, 5.

[0066] (Inference Model) In one example, the inference model 6 used in the inference device 2 may be the trained first model 4 generated by the model generation device 1. However, the method of generating a geodesic line by recursively and repeatedly estimating midpoints at a predetermined ratio may also be applied to models generated by methods other than the model generation method of the present disclosure. The inference model 6 only needs to have the ability to estimate the position of a midpoint from the positions of two points, and the method of generating the inference model 6 is not limited to the method of generating the trained first model 4 in the model generation device 1. In other words, the inference model 6 does not need to be the trained first model 4. In another example, the inference model 6 may be generated by supervised learning using a pair of two points sampled from space as training data and a true value of the midpoint between the two points as a teacher signal. Multiple data sets, each consisting of training data and a teacher signal, may be collected as appropriate.

[0067] 3 is a schematic diagram illustrating an example of a geodesic line generation process according to this embodiment. In this embodiment, the inference device 2 first acquires information on two initial points (P1 and P2), which are the first and second points that serve as the start and end points for generating a geodesic line. The initial first and second points may be given as appropriate. The initial first and second points may be specified by a user, or may be automatically specified by computer processing (e.g., randomly, by satisfying a predetermined condition, etc.).

[0068] The inference device 2 uses the inference model 6 to estimate the position of a midpoint between a first point and a second point and adopts the acquired midpoint as a waypoint. A waypoint is a point on a geodesic line, and a line connecting waypoints is a geodesic line. The inference device 2 adds the acquired midpoint as a new point and uses the inference model 6 to estimate a midpoint between two adjacent points, thereby recursively executing a series of processes to acquire further midpoints. That is, the inference model 6 adopts the combination of the first point and the midpoint and the combination of the midpoint and the second point as new combinations of the first point and the second point, respectively, and recursively executes the process of estimating midpoints. In this way, the inference model 6 generates a geodesic line between the initially given first point and the second point.

[0069] In the example of FIG. 3 , the inference device 2 acquires an intermediate point P3 between points P1 and P2 in the first stage of inference processing. Subsequently, the inference device 2 acquires an intermediate point P4 between points P1 and P3, and an intermediate point P5 between points P3 and P2 in the second stage of inference processing. The inference device 2 can obtain a sufficiently dense set of waypoints by recursively executing the process of generating intermediate points. The number of recursions (recursion count) may be appropriately specified depending on the embodiment. The number of recursions may be specified by a user or automatically specified by computer processing. The number of recursions may be determined based on any criteria. In one example, the inference device 2 may repeat the process of recursively acquiring intermediate points until the maximum distance between two adjacent points in the obtained sequence of points is less than a threshold. The trained second model 5 may be used to estimate the distance.

[0070] Basically, the given space (space SP) in the training stage may be the same as the space in the inference stage where geodesics are generated. However, the space in the training stage and the space in the inference stage do not necessarily have to be the same. The space in the training stage and the space in the inference stage may be different to the extent that the trained model is applicable. For example, the space in the training stage may be a simulated space, while the space in the inference stage may be a real space. For convenience, the space in the training stage may be referred to as a learning space or a training space, and the space in the inference stage may be referred to as an inference space.

[0071] (Task) The generation of a geodesic curve by the reasoning device 2 may be performed to solve a direct task aimed at obtaining a geodesic curve, or may be performed to solve an indirect task that can be regarded as a problem of generating a geodesic curve. Indirect tasks may include, for example, path planning, state transition planning, trajectory generation, etc. The generated geodesic curve may have various meanings depending on the task to be solved. The same applies to the generation of a geodesic curve by the trained first model 4.

[0072] In one example, to perform path planning, the reasoning device 2 may generate geodesic curves. That is, the geodesic curves may be paths. According to one example of this embodiment, a sufficiently dense number of waypoints can be obtained in a path planning scenario, thereby enabling an appropriate path planning to be expected.

[0073] Path planning may be performed for various purposes. In one example, the path may be a motion path of a robotic device. According to one example of the present embodiment, when planning a motion path of a robotic device, a sufficiently dense set of waypoints can be obtained, thereby enabling the acquisition of an appropriate motion path. The type of robotic device is not particularly limited and may be selected appropriately depending on the embodiment. The robotic device may include, for example, an industrial robot used in a production line, an autonomous robot configured to operate autonomously, etc. The industrial robot may include, for example, a vertical articulated robot, a horizontal articulated robot (SCARA robot), a parallel link robot, an Cartesian robot, etc. The autonomous robot may include, for example, a humanoid robot, a guide robot, an agricultural robot, a nursing care robot, a security robot, a transport robot, etc. The content of the autonomous processing may be selected appropriately depending on the embodiment.

[0074] In another example, the route may be a travel route of a moving object. According to one example of this embodiment, when planning a travel route of a moving object, a sufficiently dense number of waypoints can be obtained, and thus, an appropriate travel route can be expected to be obtained. Note that the type of moving object is not particularly limited and may be selected appropriately depending on the embodiment. The moving object may be configured to be mobile as appropriate. The moving object may include, for example, a cleaning robot, the above-mentioned autonomous robot configured to be mobile, a vehicle configured to be capable of automatic driving, an air vehicle capable of automatic flight (such as a drone), etc.

[0075] In one example, to infer the state transition of the device, the inference device 2 may generate a geodesic curve. That is, each point in space may be configured to represent the state of the device. The geodesic curve may represent the state transition of the device. According to one example of this embodiment, when inferring the state transition of the device, it is possible to obtain sufficiently dense waypoints, which can be expected to result in an appropriate state transition.

[0076] The type of device is not particularly limited and may be selected appropriately depending on the embodiment. In one example, the device may be a robotic device or a mobile object. According to one example of the present embodiment, when inferring the state transition of the robotic device or the mobile object, sufficiently dense waypoints can be obtained, and thus, it is possible to expect that an appropriate state transition will be obtained.

[0077] In one example, each point in space may be associated with any type of data, such as an image, sound, or various sensor data, and the inference device 2 may generate a geodesic curve to infer a data transition accordingly. For example, each point in space may be configured to represent an image. The geodesic curve may represent an image transition. According to one example of this embodiment, when inferring an image transition, sufficiently dense waypoints can be obtained, which can be expected to result in an appropriate image transition (morphing).

[0078] In one example, to simulate physical laws, the inference device 2 may generate geodesic curves. That is, each point in space may be configured to represent the state of an element given by the simulator. The element is a character in the simulation. The type and number of elements may be determined appropriately depending on the content of the simulation. The geodesic curve may represent a state trajectory of the element that conforms to the physical laws. The state trajectory may correspond to the results of a simulation of motion. The physical laws to be simulated may be selected appropriately depending on the embodiment. The physical laws may include, for example, the refractive index of light, kinetic energy, etc. According to one example of the present embodiment, when inferring a state trajectory that conforms to the physical laws, sufficiently dense waypoints can be obtained, which can be expected to result in appropriate state transitions (e.g., simulation results of the physical laws).

[0079] (Cost Function) In one example, cost may be taken into account when generating geodesic curves using the trained first model 4 and the inference model 6. In the training stage, the spatial metric may include a cost function. According to one example of the present embodiment, geodesic curves can be generated after taking cost factors into account. The cost may be defined appropriately depending on the task to be solved. In one example, the cost may be defined depending on time, expense, risk, congestion, attractiveness, etc. The cost may be defined depending on factors such as slope, presence or absence of obstacles, device kinematics, asymmetry (different between going and returning), etc. Kinematic factors may include, for example, movement limitations (e.g., vehicle turning radius, etc.), aging, amount of acceleration / deceleration, type of cargo, etc. Furthermore, in one example, when solving the image transition, the cost may be defined depending on the naturalness of the image. Furthermore, in one example, each point in space may be configured to represent a character string (sentence). The geodesic curve may represent the transition of the character string. In this case, the cost may be defined according to the naturalness of the string, and this cost function may be omitted.

[0080] (System Configuration) In one example, as shown in Figure 1, a model generation device 1, an inference device 2, and an inference device 3 may be connected to each other via a network. The type of network may be appropriately selected from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, etc. However, the method of exchanging data between each device is not limited to this example and may be appropriately selected depending on the embodiment. In another example, data may be exchanged using a storage medium.

[0081] 1, the model generation device 1, the inference device 2, and the inference device 3 are each separate computers. However, the configuration of the system according to this embodiment is not limited to this example and may be determined appropriately depending on the embodiment. In another example, a combination of at least one of the model generation device 1, the inference device 2, and the inference device 3 may be configured by a single computer. In yet another example, at least one of the model generation device 1, the inference device 2, and the inference device 3 may be configured by multiple computers.

[0082] §2 Configuration Example [Hardware Configuration] (Model Generation Device) Fig. 4 shows a schematic example of the hardware configuration of the model generation device 1 according to this embodiment. In the example shown in Fig. 4, the model generation device 1 according to this embodiment is a computer to which a control unit 11, a storage unit 12, a communication interface 13, an input device 14, an output device 15, and a drive 16 are electrically connected.

[0083] The control unit 11 includes a hardware processor such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM), and is configured to execute information processing based on programs and various data. The control unit 11 (CPU) is an example of a processor resource. The storage unit 12 may be configured, for example, with a hard disk drive or a solid-state drive. The storage unit 12, the RAM, and the ROM are examples of memory resources. In this embodiment, the storage unit 12 stores various types of information such as a model generation program 81, first learning result data 40, and second learning result data 50.

[0084] The model generation program 81 is a program for causing the model generation device 1 to execute information processing (see FIG. 10 , described below) related to the generation of the trained first model 4 and second model 5. The model generation program 81 includes a series of instructions for the information processing. The first learning result data 40 is configured to indicate information related to the generated trained first model 4. The second learning result data 50 is configured to indicate information related to the generated trained second model 5. In the present embodiment, the first learning result data 40 and the second learning result data 50 may be generated as a result of executing the model generation program 81.

[0085] As long as the learning result data (40, 50) can hold information for executing the computational processing of each trained model (4, 5), the configuration of each learning result data (40, 50) is not particularly limited and may be determined appropriately depending on the embodiment. In one example, each learning result data (40, 50) may be configured to include information indicating the values ​​of computational parameters adjusted by machine learning. In some cases, each learning result data (40, 50) may be configured to include information indicating the configuration of each model (4, 5) (e.g., the structure of a neural network, etc.).

[0086] The communication interface 13 is configured to perform wired or wireless communication via a network. The communication interface 13 may be configured, for example, by a wired LAN (Local Area Network) module, a wireless LAN module, etc. The model generation device 1 may perform data communication with other computers (e.g., the inference device 2, the inference device 3, etc.) via the communication interface 13.

[0087] The input device 14 is a device for inputting, for example, a mouse, a keyboard, etc. The output device 15 is a device for outputting, for example, a display, a speaker, etc. An operator can operate the model generation device 1 by using the input device 14 and the output device 15. The input device 14 and the output device 15 may be connected via an external interface. The input device 14 and the output device 15 may be integrated into one device, for example, a touch panel display, etc.

[0088] The drive 16 is a device for reading various information such as programs stored in a storage medium 91. At least one of the model generation program 81, the first learning result data 40, and the second learning result data 50 may be stored in the storage medium 91 instead of or together with the storage unit 12. The storage medium 91 is configured to accumulate various information (such as stored programs) by electrical, magnetic, optical, mechanical, or chemical action so that a machine such as a computer can read the information. The model generation device 1 may acquire the model generation program 81 from the storage medium 91. The storage medium 91 may be a disk-type storage medium such as a CD or DVD, or a non-disk-type storage medium such as a semiconductor memory (e.g., a flash memory). The type of the drive 16 may be selected appropriately depending on the type of the storage medium 91. The drive 16 may be connected via an external interface. The external interface may be configured appropriately to connect to an external device via a wired or wireless connection, for example, a Universal Serial Bus (USB) port, a dedicated port, or the like.

[0089] Note that, with regard to the specific hardware configuration of the model generation device 1, components may be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or the like. At least one of the communication interface 13, the input device 14, the output device 15, and the drive 16 may be omitted. The model generation device 1 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be identical. Furthermore, the model generation device 1 may be an information processing device designed specifically for the service provided, as well as a general-purpose server device, a general-purpose personal computer (PC), a tablet PC, a terminal device, or the like.

[0090] (First Inference Device) Fig. 5 shows a schematic diagram of an example of the hardware configuration of the inference device 2 according to this embodiment. In the example shown in Fig. 5, the inference device 2 according to this embodiment is a computer to which a control unit 21, a storage unit 22, a communication interface 23, an input device 24, an output device 25, and a drive 26 are electrically connected.

[0091] The control unit 21 to the drive 26 and the storage medium 92 of the inference device 2 may be configured similarly to the control unit 11 to the drive 16 and the storage medium 91 of the model generation device 1. The control unit 21 (CPU) is an example of a processor resource of the inference device 2, and the storage unit 22 (and RAM, ROM) is an example of a memory resource of the inference device 2. In this embodiment, the storage unit 22 stores various information such as an inference program 82 and inference model data 60.

[0092] The inference program 82 is a program for causing the inference device 2 to execute information processing related to the generation of geodesic lines (see FIG. 11 described below). The inference program 82 includes a series of instructions for the information processing. The inference model data 60 is configured to indicate information related to the inference model 6. When a trained first model 4 is adopted as the inference model 6, the inference model data 60 may be the first learning result data 40. At least one of the inference program 82 and the inference model data 60 may be stored in a storage medium 92 instead of or together with the storage unit 22. The inference device 2 may acquire at least one of the inference program 82 and the inference model data 60 from the storage medium 92.

[0093] The inference device 2 may communicate data with another computer (e.g., the model generation device 1) via the communication interface 23. An operator can operate the inference device 2 by using the input device 24 and the output device 25.

[0094] Note that, with regard to the specific hardware configuration of the inference device 2, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 21 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, FPGA, DSP, GPU, ASIC, etc. At least one of the communication interface 23, input device 24, output device 25, and drive 26 may be omitted. The inference device 2 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. The inference device 2 may be an information processing device designed specifically for the service provided, as well as a general-purpose server device, a general-purpose PC, a tablet PC, a terminal device, etc.

[0095] (Second Inference Device) Fig. 6 shows a schematic diagram of an example of the hardware configuration of the inference device 3 according to this embodiment. In the example shown in Fig. 6, the inference device 3 according to this embodiment is a computer in which a control unit 31, a storage unit 32, a communication interface 33, an input device 34, an output device 35, and a drive 36 are electrically connected.

[0096] The control unit 31 to the drive 36 and the storage medium 93 of the inference device 3 may be configured similarly to the control unit 11 to the drive 16 and the storage medium 91 of the model generation device 1. The control unit 31 (CPU) is an example of a processor resource of the inference device 3, and the storage unit 32 (and RAM, ROM) is an example of a memory resource of the inference device 3. In this embodiment, the storage unit 32 stores various information such as an inference program 83 and second learning result data 50.

[0097] The inference program 83 is a program for causing the inference device 3 to execute information processing related to distance estimation (see FIG. 12 , which will be described later). The inference program 83 includes a series of instructions for the information processing. At least one of the inference program 83 and the second learning result data 50 may be stored in a storage medium 93 instead of or together with the storage unit 32. The inference device 3 may acquire at least one of the inference program 83 and the second learning result data 50 from the storage medium 93.

[0098] The inference device 3 may communicate data with another computer (e.g., the model generation device 1) via the communication interface 33. An operator can operate the inference device 3 by using the input device 34 and the output device 35.

[0099] Note that, with regard to the specific hardware configuration of the inference device 3, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 31 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, FPGA, DSP, GPU, ASIC, etc. At least one of the communication interface 33, input device 34, output device 35, and drive 36 may be omitted. The inference device 3 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. The inference device 3 may be an information processing device designed specifically for the service to be provided, as well as a general-purpose server device, a general-purpose PC, a tablet PC, a terminal device, etc.

[0100] 7 schematically shows an example of the software configuration of the model generation device 1 according to this embodiment. The control unit 11 of the model generation device 1 loads a model generation program 81 stored in the storage unit 12 into RAM, and executes instructions included in the model generation program 81 using the CPU. As a result, the model generation device 1 operates as a computer including a learning processing unit 111 and an output unit 112 as software modules.

[0101] The learning processing unit 111 is configured to direct the execution of machine learning of the first model 4 and the second model 5. The machine learning includes training the first model 4 to acquire the ability to estimate the position of a waypoint using a distance estimation result by the second model 5, and training the second model 5 to acquire the ability to estimate distance using a waypoint estimation result of the first model 4 and a spatial metric. The output unit 112 is configured to output the results of the machine learning.

[0102] 8 shows a schematic diagram of an example of the software configuration of the inference device 2 according to this embodiment. The control unit 21 of the inference device 2 loads an inference program 82 stored in the storage unit 22 into RAM, and executes instructions included in the inference program 82 using the CPU. This causes the inference device 2 to operate as a computer equipped with a position inference unit 211 and an output unit 212 as software modules.

[0103] The position inference unit 211 has an inference model 6 (a trained first model 4) by holding inference model data 60 (first learning result data 40). The position inference unit 211 is configured to infer the position of a midpoint between a given first point and a given second point in space using the inference model 6. The position inference unit 211 is configured to generate a geodesic line by recursively repeating the process of adding the inferred midpoint as a new point and inferring a midpoint between two adjacent points. The output unit 212 is configured to output information about the generated geodesic line.

[0104] 9 schematically shows an example of the software configuration of the inference device 3 according to this embodiment. The control unit 31 of the inference device 3 loads the inference program 83 stored in the storage unit 32 into RAM, and causes the CPU to execute instructions included in the inference program 83. As a result, the inference device 3 operates as a computer that includes a distance inference unit 311 and an output unit 312 as software modules.

[0105] The distance inference unit 311 holds the second learning result data 50, thereby providing a trained second model 5. The distance inference unit 311 is configured to infer the distance between two given points in space using the trained second model 5. The output unit 312 is configured to output information related to the inferred distance.

[0106] (Other) In the present embodiment, an example is described in which each software module of the model generating device 1, the inference device 2, and the inference device 3 is implemented by a general-purpose CPU. However, some or all of the above software modules may be implemented by one or more dedicated processors or chipsets. Each of the above modules may be implemented as a hardware module. With regard to the software configurations of the model generating device 1, the inference device 2, and the inference device 3, modules may be omitted, replaced, or added as appropriate depending on the embodiment.

[0107] §3 Operation Example (Model Generation Device) Figure 10 is a flowchart showing an example of the processing procedure of the model generation device 1 according to this embodiment. The following processing procedure is an example of a model generation method executed by a computer. However, the processing procedure of the model generation device 1 is merely an example, and each step may be changed as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0108] In step S101, the control unit 11 operates as the learning processing unit 111 and directs the execution of machine learning of the first model 4 and the second model 5. The machine learning of the first model 4 and the second model 5 may be executed within the model generation device 1 or may be executed on an external computer other than the model generation device 1. In one example, the training of the first model 4 may include the above steps (A) to (E) (FIG. 2A). The training of the second model 5 may include the above steps (1) to (4) (FIG. 2B). In another example, the midpoint may be the midpoint, and the training process of the first model 4 and the second model 5 may be executed according to the pseudocode shown in FIG. 2C. When the training of the first model 4 and the second model 5 is completed, the control unit 11 proceeds to the next step S102.

[0109] In step S102, the control unit 11 operates as the output unit 112 and outputs the results of the machine learning. The output destination and the content of the information to be output may be selected as appropriate depending on the embodiment. In one example, the control unit 11 may generate learning result data (first learning result data 40, second learning result data 50) indicating the results of the machine learning as the output process and store the generated learning result data in a predetermined storage area. The predetermined storage area may be, for example, RAM in the control unit 11, the storage unit 12, an external storage device, a storage medium, or a combination thereof. The storage medium may be, for example, a CD, a DVD, a semiconductor memory, or the like. The external storage device may be, for example, a data server such as a NAS (Network Attached Storage). The external storage device may be, for example, an external storage device. When machine learning of a model is performed on an external computer, the learning result data may be generated by the external computer. In another example, the control unit 11 may output the convergence result of training in the machine learning as the output process. The output destination may be, for example, the RAM in the control unit 11, the output device 15, an external computer, an external storage device, a storage medium, or a combination of these.

[0110] When the output of the machine learning results is completed, the control unit 11 ends the processing procedure of the model generation device 1 according to this operation example. When the inference device 2 uses the trained first model 4 as the inference model 6, the generated first learning result data 40 may be provided to the inference device 2 from the model generation device 1 or an external computer at any timing and by any method. In one example, the first learning result data 40 may be provided to the inference device 2 via a network. In another example, the first learning result data 40 may be provided to the inference device 2 via a storage medium. In yet another example, the first learning result data 40 may be pre-installed in the inference device 2. Similarly, the second learning result data 50 may be provided to the inference device 2 from the model generation device 1 or an external computer at any timing and by any method.

[0111] Furthermore, the control unit 11 may update or newly generate the first learning result data 40 and the second learning result data 50 by periodically or irregularly repeating the processes of steps S101 and S102. During this repetition, at least some of the machine learning conditions may be changed, modified, added, deleted, or the like as appropriate. The control unit 11 may then provide the updated or newly generated first learning result data 40 to the inference device 2, thereby updating the first learning result data 40 held by the inference device 2. The control unit 11 may also update the second learning result data 50 held by the inference device 3, by providing the updated or newly generated second learning result data 50 to the inference device 3.

[0112] (First Inference Device) FIG. 11 is a flowchart showing an example of the processing procedure of the inference device 2 according to this embodiment. The following processing procedure is an example of an inference method executed by a computer. However, the processing procedure of the inference device 2 is merely an example, and each step may be modified as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0113] In step S201, the control unit 21 operates as a position inference unit 211 and uses the inference model 6 to infer the position of the midpoint between a given first point and a given second point in space. The initial first point and second point may be given as appropriate. The control unit 21 inputs information about the given first point and second point into the inference model 6 and executes the calculation processing of the inference model 6. As a result, the control unit 21 obtains the inference result of the midpoint position from the inference model 6.

[0114] In step S202, the control unit 21 operates as the position inference unit 211 and determines whether to repeat the process of step S201. As described above, the number of repetitions may be specified by the user or automatically by computer processing. The number of repetitions may be determined based on any criteria. For example, if it is determined to repeat the process of step S201 because the set number of repetitions has not been reached or the criteria have not been met, the control unit 21 adds the estimated midpoint as a new point, returns to step S201, and re-executes the process of inferring a midpoint between two adjacent points. By recursively repeating the process of step S201, the control unit 21 can generate a geodesic line between the initially given first and second points, which is composed of multiple estimated midpoints. On the other hand, if it is determined not to repeat the process of step S201, the control unit 21 proceeds to the next step S203.

[0115] In step S203, the control unit 21 operates as the output unit 212 and outputs information about the generated geodesic curve. The output destination and the content of the information to be output may each be selected appropriately depending on the embodiment. In one example, the control unit 21 may output the generation result of the geodesic curve as is. In another example, the control unit 21 may execute any information processing on the generated geodesic curve. Then, the control unit 21 may output the result of the information processing as information about the geodesic curve. The output destination may be, for example, RAM in the control unit 21, the output device 25, an external computer, an external storage device, a storage medium, or a combination of these.

[0116] When the output of the information is completed, the control unit 21 ends the processing procedure of the inference device 2 according to this operation example. Note that the control unit 21 may execute the series of information processing steps S201 to S203 at any timing, such as at the request of the user.

[0117] (Second Inference Device) Figure 12 is a flowchart showing an example of the processing procedure of the inference device 3 according to this embodiment. The following processing procedure is an example of an inference method executed by a computer. However, the processing procedure of the inference device 3 is merely an example, and each step may be changed as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0118] In step S301, the control unit 31 operates as the distance inference unit 311 and uses the trained second model 5 to estimate the distance between two given points in space. The two points (first point and second point) may be appropriately given, as in step S201 above. The control unit 31 inputs information about the two given points into the trained second model 5 and executes calculation processing of the trained second model 5. As a result, the control unit 31 obtains the result of estimating the distance between the two points from the trained second model 5. Upon obtaining the distance estimation result, the control unit 31 proceeds to the next step S302.

[0119] In step S302, the control unit 31 operates as the output unit 312 and outputs information related to the estimated distance. The output destination and the content of the information to be output may each be selected appropriately depending on the embodiment. In one example, the control unit 31 may output the distance estimation result as is. In another example, the control unit 31 may perform any information processing based on the distance estimation result. The control unit 31 may then output the result of the information processing as information related to the distance. The output destination may be, for example, RAM in the control unit 31, the output device 35, an external computer, an external storage device, a storage medium, or a combination thereof.

[0120] When the output of the information is completed, the control unit 31 ends the processing procedure of the inference device 3 according to this operation example. Note that the control unit 31 may execute the series of information processing of steps S301 and S302 at any timing, such as at the request of the user.

[0121] [Features] In this embodiment, the model generation device 1 can generate a trained first model 4 that has acquired the ability to estimate intermediate points between two points in a given space by performing the process in step S101. Using the generated trained first model 4, a geodesic line can be generated while densely obtaining waypoints by repeatedly recursively estimating intermediate points located at a predetermined ratio. Therefore, according to this embodiment, a sufficiently dense waypoint can be obtained when generating a geodesic line. Furthermore, by performing the processes in steps S201 and S202, the inference device 2 employs a method of recursively estimating intermediate points located at a predetermined ratio using the inference model 6 as a geodesic line generation method, thereby obtaining a sufficiently dense waypoint. Furthermore, by performing the process in step S301, the inference device 3 can estimate the distance between two points by using the trained second model 5.

[0122] §4 Modifications Although the embodiments of the present disclosure have been described in detail above, the above description is merely illustrative of the present disclosure in all respects. Various improvements or modifications may be made to the above embodiments as appropriate. For example, in the systems according to the above embodiments, the inference device 3 may be omitted. One of the model generation device 1 and the inference device 2 may be omitted. Furthermore, for example, the inference device 2 may further use a trained second model 5 to calculate the distance between each waypoint that constitutes a geodesic line.

[0123] §5 Examples The following experiments were carried out to verify the effectiveness of the above-described embodiment, but the present invention is not limited to the following examples.

[0124] (First Example) The method of the above embodiment and the method of FIG. 2C was adopted as the model generation method of the first example. The above (Equation 9) was adopted for the loss of the first model, and the above (Equation 8) was adopted for the loss of the second model. The number of epochs (N ｅｐｏｃｈｓ ) was set to 10, and the batch size was set to 256. Also, the iteration depth for collecting sample data was gradually increased up to a defined depth D.

[0125] Second Example In a model generation method of the second example, the loss of the first model was replaced from the above (Equation 9) to the following (Equation 12). Other conditions in the second example were set to be the same as those in the first example. From the following (Equation 13), the loss in (Equation 12) means that the first model is trained to estimate the midpoint of 2:1.

[0126]

[0127] (First Comparative Example) In the first comparative example, the method described in Reference 1 (John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov, "Proximal policy optimization algorithms", [online], [searched September 22, 2023], Internet <URL: https: / / arxiv.org / abs / 1707.06347>) was adopted to construct a method for generating a model that sequentially acquires waypoints using general goal-conditional reinforcement learning. The reward function was defined using the pseudo-Finsler metric.

[0128] (Second Comparative Example) In the second comparative example, the reinforcement learning method described in Non-Patent Document 1 was adopted as the model generation method. As a subgoal, the value to be optimized during training was changed so as to estimate the midpoint.

[0129] (First Experiment) For the environment of the first experiment, we prepared the Matsumoto metric space proposed in Reference 2 (Makoto Matsumoto, “A slope of a mountain is a Finsler surface with respect to a time measure”, [online], [searched September 22, 2023], Internet <URL: https: / / www.semanticscholar.org / paper / A-slope-of-a-mountain-is-a-Finsler-surface-with-to-Matsumoto / a7ab70674cbf5dd4d7d89b5aab10c4316e81f895>). The Matsumoto metric is an asymmetric Finsler metric that takes into account the travel time on a slope. The Matsumoto metric F:TM → [0, ∞] was defined as follows (Equation 14) and (Equation 15).

[0130]

[0131] A field was set using a disk M with a radius of 1 centered at the origin and f(p) in the following (Equation 16). For this space, a model was trained using the methods of the first and second examples and the first and second comparative examples. The number of time steps was approximately 2 × 10 ７ was set to.

[0132]

[0133] Furthermore, two points s, g∈M were randomly sampled in this space, and a geodesic curve was generated between the two obtained points (s, g) using the models of each example and each comparative example. It was evaluated whether the local distances between two adjacent points in the sequence of points constituting the geodesic curve were all less than the threshold ε. If all local distances were less than the threshold ε, it was determined that a geodesic curve with sufficiently dense waypoints had been successfully generated. In the first experiment, the number of segments n in the model of the first comparative example was set to 64. Correspondingly, the iteration depth D in the models of the first to second examples and the second comparative example was set to 6 (i.e., 63 waypoints were obtained). The threshold ε was set to 0.1. The success rate of each example and comparative example was measured for 100 pairs of randomly selected two points, and the progress of the learning process was plotted. Furthermore, after the machine learning was completed, a geodesic curve was generated between the two randomly sampled points using the models of the first to second examples and the first to second comparative examples.

[0134] (Second Experiment) For the environment of the second experiment, we prepared the car-like metric space proposed in Reference 3 ("Kinodynamic trajectory optimization and control for car-like robots", [online], [searched September 22, 2023], Internet <URL: https: / / rst.etit.tu-dortmund.de / storages / rst-etit / r / Global / Paper / Roesmann / 2017_Roesmann_IROS.PDF>). The car-like metric is a pseudo-Finsler metric that takes into account the trajectory of a vehicle moving in one direction. M is defined by the following (Equation 17), and the car-like metric is defined as the following (Equations 18) to (Equations 20).

[0135]

[0136] Here, c ｐ is the penalty coefficient, and r ｍｉｎ is the lower limit of the radius of curvature. ｐSet to 100 and r ｍｉｎ was set to 0.5. U was set to a disk with radius 1 centered at the origin. The term proposed in Reference 3 is h ２ A one-way field was set by adding the term. For this space, a model was trained using the methods of the first embodiment and the first to second comparative examples. The number of time steps was approximately 8 × 10 ７ In the same manner as in the first experiment, the first embodiment and each comparative example were evaluated for success in generating a geodesic curve with a sufficiently dense number of waypoints. In the second experiment, the number of segments n in the model of the first comparative example was set to 256. Correspondingly, the iteration depth D in the models of the first embodiment and the second comparative example was set to 8 (i.e., 255 waypoints were obtained). The threshold ε was set to 0.1. The success rate of the first embodiment and each comparative example was measured for 100 pairs of randomly selected two points, and the progression due to learning was plotted.

[0137] (Experimental Results) FIG. 13 shows the plot results of the first experiment. FIG. 14 shows the results of generating geodesic lines in the first experiment. FIG. 15 shows the plot results of the second experiment. As shown in FIG. 13 , each example had a higher success rate of generating geodesic lines with sufficiently dense waypoints than each comparative example. As a result, as shown in FIG. 14 , each example was able to obtain geodesic lines closer to the true value of the shortest path than each comparative example. Furthermore, the result in FIG. 15 also shows that the first example had a higher success rate of generating geodesic lines with sufficiently dense waypoints than each comparative example. These results demonstrate that the model generation method of this embodiment is effective for generating geodesic lines with sufficiently dense waypoints. Furthermore, it was demonstrated that the model generation method of this embodiment is effective for route planning.

[0138] This specification includes the following disclosure: [Supplementary Note 1] A model generation method, in which a computer (1) executes the steps of: directing the execution of machine learning of a first model (4) and a second model (5); and outputting the results of the machine learning, wherein the first model (4) is configured to estimate the position of an intermediate point between two points based on the positions of the two points in a given space; and the second model (5) is configured to estimate the distance between the two points based on the positions of the two points, and the machine learning includes: training the first model (4) to acquire the ability to estimate the position of the intermediate point using a result of the distance estimation by the second model (5), and training the second model (5) to acquire the ability to estimate the distance using a result of the intermediate point estimation by the first model (4) and a metric of the space. [Supplementary Note 2] The model generation method according to Supplementary Note 1, in which the intermediate point is a midpoint. [Supplementary Note 3] The model generation method according to Supplementary Note 1 or 2, wherein training the first model (4) includes: sampling two points from the space, each consisting of a first sample point and a second sample point; using the first model (4) to estimate a first position of a midpoint between the sampled first sample point and the second sample point; using the second model (5) to estimate a first distance between the first sample point and the estimated midpoint; using the second model (5) to estimate a second distance between the estimated midpoint and the second sample point; and training the first model (4) so ​​that the estimated first distance and the second distance satisfy a condition for the midpoint.[Supplementary Note 4] Training the second model (5) includes: (I) sampling two points consisting of a third sample point and a fourth sample point from the space; (II) using the second model (5) to estimate a third distance between the sampled third sample point and the fourth sample point; (III) if the third sample point and the fourth sample point satisfy a proximity condition, training the second model (5) so that the estimated third distance is equal to a value given by a metric of the space; and (IV) if the third sample point and the fourth sample point do not satisfy the proximity condition, (IV-1) using the first model (4) to estimate a second position of a midpoint between the sampled third sample point and the fourth sample point; (IV-2) adopting the combination of the third sample point and the midpoint, and the combination of the midpoint and the fourth sample point as new combinations of the third sample point and the fourth sample point, respectively, and repeating the process from (II); (IV-3) After repeating the process from (II) according to (IV-2), training the second model (5) so that the estimated value of the third distance between the third sample point and the fourth sample point sampled in (I), estimated using the second model (5), is equal to the sum of values ​​given by the spatial metric. [Supplementary Note 5] The model generation method according to any one of Supplementary Notes 1 to 4, wherein the spatial metric includes a cost function.[Supplementary Note 6] An inference program (82) for causing a computer (2) to execute the following steps: using an inference model (6) to infer the position of a midpoint between a first point and a second point given in space, the inference model (6) being configured to infer, from the positions of the two points, the position of a midpoint that exists between the two points and has a predetermined ratio of distances from the two points; generating a geodesic line between the first point and the second point, the geodesic line being composed of a plurality of the inferred midpoints, by recursively executing the inferring step; and outputting information about the generated geodesic line. [Supplementary Note 7] The inference program (82) according to Supplementary Note 6, wherein the geodesic line is a path. [Supplementary Note 8] The inference program (82) according to Supplementary Note 7, wherein the path is a motion path of a robot device. [Supplementary Note 9] The inference program (82) according to Supplementary Note 7, wherein the path is a movement path of a moving object. [Supplementary Note 10] An inference program (82) according to Supplementary Note 6, wherein each point in the space is configured to represent a state of a device, and the geodesic lines represent state transitions of the device. [Supplementary Note 11] An inference program (82) according to Supplementary Note 10, wherein the device is a robotic device or a mobile object. [Supplementary Note 12] An inference program (82) according to Supplementary Note 6, wherein each point in the space is configured to represent an image, and the geodesic lines represent transitions of the image. [Supplementary Note 13] An inference program (82) according to Supplementary Note 6, wherein each point in the space is configured to represent a state of an element provided by a simulator, and the geodesic lines represent a trajectory of a state of the element in accordance with physical laws. [Supplementary Note 14] An inference program (83) for causing a computer (3) holding the trained second model (5) generated by the model generation method according to any one of Supplements 1 to 5 to execute the steps of: estimating a distance between two points given in space using the trained second model (5); and outputting information related to the estimated distance.

[0139] REFERENCE SIGNS LIST 1...model generation device, 11...control unit, 12...storage unit, 13...communication interface, 14...input device, 15...output device, 16...drive, 81...model generation program, 91...storage medium, 111...learning processing unit, 112...output unit, 2...inference device, 21...control unit, 22...storage unit, 23...communication interface, 24...input device, 25...output device, 26...drive, 82...inference program, 92...storage medium, 211...position inference unit, 212...output unit, 3...inference device, 31...control unit, 32...storage unit, 33...communication interface, 34...input device, 35...output device, 36...drive, 83...inference program, 93...storage medium, 311...distance inference unit, 312...output unit, 4...first model, 40...first learning result data, 5...second model, 50...second learning result data, 6...inference model, 60...inference model data

Claims

1. A model generation method, comprising: a computer executing the steps of: directing the execution of machine learning of a first model and a second model; and outputting the results of the machine learning, wherein the first model is configured to estimate the position of an intermediate point between two points from the positions of the two points in a given space; and the second model is configured to estimate the distance between the two points from the positions of the two points, and the machine learning includes: training the first model to acquire the ability to estimate the position of the intermediate point using a result of the distance estimation by the second model; and training the second model to acquire the ability to estimate the distance using the result of the intermediate point estimation by the first model and a metric of the space.

2. The model generation method of claim 1, wherein the intermediate point is a midpoint.

3. The model generation method of claim 1, wherein training the first model includes: sampling two points from the space, the points being a first sample point and a second sample point; estimating a first location of a midpoint between the sampled first sample point and the second sample point using the first model; estimating a first distance between the first sample point and the estimated midpoint using the second model; estimating a second distance between the estimated midpoint and the second sample point using the second model; and training the first model such that the estimated first distance and the second distance satisfy a condition for the midpoint.

4. Training the second model includes: (1) sampling two points consisting of a third sample point and a fourth sample point from the space; (2) using the second model to estimate a third distance between the sampled third sample point and the fourth sample point; (3) training the second model such that, if the third sample point and the fourth sample point satisfy a proximity condition, the estimated third distance is equal to a value given by a metric of the space; and (4) if the third sample point and the fourth sample point do not satisfy the proximity condition, (4-1) using the first model to estimate a second position of a midpoint between the sampled third sample point and the fourth sample point; (4-2) adopting the combination of the third sample point and the midpoint, and the combination of the midpoint and the fourth sample point as new combinations of the third sample point and the fourth sample point, respectively, and repeating the process from (2); (4-3) training the second model so that after repeating the process from (2) through (4-2), an estimated value of the third distance between the third sample point and the fourth sample point sampled in (1), estimated using the second model, is equal to the sum of values ​​given by a metric of the space.

5. The method of claim 1, wherein the spatial metric comprises a cost function.

6. An inference program for causing a computer to execute the steps of: using an inference model to infer a position of a midpoint between a first point and a second point given in space, the inference model being configured to infer, from the positions of the two points, a position of a midpoint that exists between two points and has a predetermined ratio of distances from the two points; generating a geodesic line between the first point and the second point, the geodesic line being composed of a plurality of the inferred midpoints, by repeatedly executing the inference step recursively; and outputting information about the generated geodesic line.

7. The inference program according to claim 6, wherein the geodesic curve is a path.

8. The inference program according to claim 7, wherein the path is a motion path of a robot device.

9. The inference program according to claim 7, wherein the route is a travel route of a moving object.

10. The inference program of claim 6, wherein each point in the space is configured to represent a state of a device, and the geodesic curve represents a state transition of the device.

11. The inference program according to claim 10, wherein the device is a robot device or a moving object.

12. The inference program of claim 6, wherein each point in the space is configured to represent an image, and the geodesic curve represents a transition of the image.

13. The inference program of claim 6, wherein each point in the space is configured to represent a state of an element given by a simulator, and the geodesic line represents a locus of states in the element that obeys physical laws.

14. An inference program for causing a computer that holds the trained second model generated by the model generation method described in any one of claims 1 to 5 to execute the steps of: using the trained second model to estimate the distance between two given points in space; and outputting information related to the estimated distance.

Images