Model generation method and inference program

Abstract

To provide a technology for acquiring a sufficiently dense weigh point when generating a geodesic line.SOLUTION: A model generation device instructs execution of machine learning of a first model and a second model. The first model is constituted so as to infer the position of an intermediate point existing between two points in a given space. The second model is constituted so as to infer the distance between the two points. The machine learning includes: training the first model so as to acquire capacity of inferring the position of the intermediate point by using an inference result of the distance by the two models; and training the second model so as to acquire capacity of inferring the distance by using the inference result of the intermediate point of the first model and a space metric.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a model generation method and an inference program. [Background technology]

[0002] Problems such as route planning, state transition planning, and trajectory generation can be regarded as examples of problems for generating a geodesic line between two points. One example of a method for solving a problem for generating a geodesic line is a method for generating an inference model for inferring a waypoint by goal-conditioned reinforcement learning. According to this method, a geodesic line can be generated by sequentially inferring waypoints from a start point to an end point using the generated inference model. Another example of a method for solving a problem for generating a geodesic line is a method proposed in Non-Patent Document 1. According to the method proposed in Non-Patent Document 1, a geodesic line can be generated by repeatedly inferring subgoals as waypoints using the generated inference model. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Tom Jurgenson, Edward Groshev, Aviv Tamar, “Sub-Goal Trees -- a Framework for Goal-Directed Trajectory Prediction and Optimization”, [online], [Retrieved September 22, 2023], Internet<URL:https: / / arxiv.org / abs / 1906.05329> Summary of the Invention [Problem to be solved by the invention]

[0004] The inventor of the present invention found that the above conventional method has the following problems. That is, when a reinforcement learning method with a goal condition is adopted, if a reward is given only when the goal is reached, the reward becomes sparse, making learning difficult. As a method for solving this, it is possible to adopt a method of giving a reward every time the goal is approached, but this method makes the design of the reward function complicated. Therefore, it is difficult to generate an inference model simply. On the other hand, according to the method of Non-Patent Document 1, it is possible to generate an inference model simply. However, in the method of Non-Patent Document 1, the subgoal is an arbitrary point between two points. Therefore, there is a possibility that a sufficiently dense waypoint cannot be obtained. If a sufficiently dense waypoint cannot be obtained, various problems may occur. For example, in a route planning scene, it becomes difficult to obtain the shortest route if a sufficiently dense waypoint cannot be obtained.

[0005] One aspect of the present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique for obtaining sufficiently dense waypoints when generating a geodesic line. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, the present invention employs the following configurations. Note that the following configurations of the invention can be combined as appropriate.

[0007] That is, a model generation method according to one aspect of the present invention is an information processing method in which a computer executes a step of directing execution of machine learning of a first model and a second model, and a step of outputting the results of the machine learning. The first model is configured to estimate the position of an intermediate point between two points in a given space from the positions of the two points. The second model is configured to estimate the distance between the two points from the positions of the two points. The machine learning is configured to estimate the intermediate point using the distance estimation result by the second model. The method includes training the first model to acquire the ability to estimate the location of an intermediate point, and training the second model to acquire the ability to estimate the distance using the intermediate point estimation results of the first model and a metric of the space.

[0008] The intermediate point may be a point that exists between two points and the ratio of the distance from the two points satisfies a predetermined condition (i.e., the ratio that divides the distance between the two points internally is defined). The predetermined ratio (ratio of distances) may vary. According to this configuration, a trained first model that has acquired the ability to predict an intermediate point that exists between two points in a given space can be generated. By using the generated trained first model to repeatedly predict an intermediate point located at a predetermined ratio recursively, a geodesic line can be generated while densely obtaining waypoints. Therefore, according to this configuration, when generating a geodesic line, sufficiently dense waypoints can be obtained.

[0009] In the model generating method according to the above aspect, the intermediate points may be midpoints. With this configuration, it is possible to obtain sufficiently dense and uniform waypoints.

[0010] In the model generating method according to the above aspect, training the first model may include sampling two points, which are a first sample point and a second sample point, from the space, estimating a first position of a midpoint between the sampled first sample point and the second sample point using the first model, estimating a first distance between the first sample point and the estimated first position using the second model, estimating a second distance between the estimated first position and the second sample point using the second model, and training the first model such that the estimated first distance and the second distance satisfy a condition of the midpoint. With this configuration, a trained first model can be appropriately generated.

[0011] In the model generation method according to the above aspect, training the second model includes: (1) sampling two points consisting of a third sample point and a fourth sample point from the space; (2) using the second model to estimate a third distance between the sampled third sample point and the fourth sample point; (3) training the second model such that, if the third sample point and the fourth sample point satisfy a proximity condition, the estimated third distance is equal to a value given by a metric of the space; and (4) if the third sample point and the fourth sample point do not satisfy the proximity condition, (4-1) training the second model to estimate a third distance between the sampled third sample point and the fourth sample point using the first model. (4-2) adopting the combination of the third sample point and the midpoint, and the combination of the midpoint and the fourth sample point as new combinations of the third sample point and the fourth sample point, respectively, and repeating the process from (2), and (4-3) training the second model such that an estimated value of the third distance between the third sample point and the fourth sample point sampled in (1), estimated using the second model after repeating from (2) by (4-2), is equal to the sum of values ​​given by a metric of the space. With this configuration, a trained second model can be appropriately generated.

[0012] In the model generating method according to the above aspect, the spatial metric may include a cost function. With this configuration, it is possible to generate a geodesic curve taking into account a cost element defined by the cost function.

[0013] Furthermore, the present invention may not be limited to the above-mentioned model generation. One aspect of the present invention may be an inference program that uses at least one of the first model and the second model that have been trained and generated by the above-mentioned model generation method. The application of the configuration for obtaining dense waypoints by predicting the distance between the target object and the target object may not be limited to the application of the trained first model generated in each of the above aspects.

[0014] For example, an inference program according to one aspect of the present invention may be a program for causing a computer to execute a step of estimating a position of a midpoint between a first point and a second point given in space using an inference model, the inference model being configured to estimate a position of a midpoint that exists between two points and has a predetermined ratio of distances from the two points from the positions of the two points, a step of generating a geodesic line between the first point and the second point, the geodesic line being composed of a plurality of the estimated midpoints by repeatedly executing the inference step recursively, and a step of outputting information about the generated geodesic line. According to this configuration, when generating a geodesic line, sufficiently dense waypoints can be obtained. The inference model may be a trained first model generated by the model generation method according to any of the above aspects.

[0015] In the inference program according to the above aspect, the geodesic curve may be a route. According to this configuration, when planning a route, it is possible to obtain sufficiently dense waypoints, and thus it is possible to expect an appropriate route plan.

[0016] In the inference program according to the above aspect, the path may be a motion path of a robot device. With this configuration, when planning a motion path of the robot device, it is possible to obtain sufficiently dense waypoints, and thus it is expected that an appropriate motion path can be obtained.

[0017] In the inference program according to the above aspect, the route may be a route of a moving object. According to this configuration, when planning a route of a moving object, it is possible to obtain sufficiently dense waypoints, and thus it is possible to expect the acquisition of an appropriate route.

[0018] In the inference program according to the above aspect, each point in the space may be configured to indicate a state of the device. The geodesic curve may indicate a state transition of the device. With this configuration, when inferring a state transition of the device, sufficiently dense waypoints can be obtained, and thus it is possible to expect the acquisition of an appropriate state transition.

[0019] In the inference program according to the above aspect, the device may be a robot device or a moving body. With this configuration, when inferring a state transition of the robot device or the moving body, sufficiently dense waypoints can be obtained, and thus it is expected that an appropriate state transition can be obtained.

[0020] In the inference program according to the above aspect, each point in the space may be configured to represent an image. The geodesic curve may represent a transition of the image. According to this configuration, when inferring an image transition, sufficiently dense waypoints can be obtained, and thus, it is expected that an appropriate image transition (morphing) can be obtained.

[0021] In the inference program according to the above aspect, each point in the space may be configured to indicate a state of an element given by a simulator. The geodesic line may indicate a trajectory of a state of the element that conforms to physical laws. With this configuration, when inferring a trajectory of a state that conforms to physical laws, sufficiently dense waypoints can be obtained, and thus it is possible to expect the acquisition of an appropriate state transition (e.g., a simulation result of physical laws).

[0022] In addition, for example, an inference program according to one aspect of the present invention includes a step of: causing a computer that holds the trained second model generated by the model generation method according to any one of the above aspects to execute the training. The computer program may be a program for executing a step of estimating a distance between two given points in space using the second model that has been refined, and a step of outputting information about the estimated distance. With this configuration, the distance between two points can be estimated.

[0023] The embodiment of the present invention may not be limited to the above model generation method. As another aspect of the model generation method according to each of the above aspects, one aspect of the present invention may be an information processing device (model generation device) that realizes all or part of each of the above configurations, a program, or a storage medium that stores such a program and is readable by a machine such as a computer. A storage medium that is readable by a machine such as a computer is a medium that accumulates information such as a program by electrical, magnetic, optical, mechanical, or chemical action. Similarly, the embodiment of the present invention may not be limited to the above inference program. As another aspect of the inference program according to each of the above aspects, one aspect of the present invention may be an information processing device (inference device) that realizes all or part of each of the above configurations, an information processing method (inference method), or a storage medium that stores a program and is readable by a machine such as a computer. Effect of the Invention

[0024] According to the present invention, a technique can be provided for obtaining sufficiently dense waypoints when generating geodesic curves. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 shows a schematic diagram of an example of a situation in which the present invention is applied. [Figure 2A] FIG. 2A illustrates an example of a training process for a first model according to an embodiment. [Figure 2B] FIG. 2B illustrates an example of a training process for a second model according to an embodiment. [Figure 2C] FIG. 2C illustrates an example of a pseudo program code for a training process of a first model and a second model according to an embodiment. [Diagram 3] FIG. 3 illustrates an example of a process for generating a geodesic curve according to an embodiment. [Figure 4] FIG. 4 illustrates an example of a hardware configuration of the model generating device according to the embodiment. [Diagram 5] FIG. 5 illustrates an example of a hardware configuration of a (first) inference device according to an embodiment. [Figure 6] FIG. 6 illustrates an example of a hardware configuration of a (second) inference device according to an embodiment. [Figure 7] FIG. 7 illustrates an example of a software configuration of the model generating device according to the embodiment. [Figure 8] FIG. 8 shows an example of a software configuration of the (first) inference device according to the embodiment. [Figure 9] FIG. 9 shows an example of a software configuration of a (second) inference device according to an embodiment. [Figure 10] FIG. 10 is a flowchart illustrating an example of a processing procedure of the model generating device according to the embodiment. [Figure 11] FIG. 11 is a flowchart showing an example of a processing procedure of the (first) inference device according to the embodiment. [Figure 12] FIG. 12 is a flowchart showing an example of a processing procedure of the (second) inference device according to the embodiment. [Figure 13] FIG. 13 shows the results of the first experiment. [Figure 14] FIG. 14 shows the results of generating geodesic curves in the first experiment. [Figure 15] FIG. 15 shows the results of the second experiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] An embodiment according to one aspect of the present invention (hereinafter, also referred to as "the present embodiment") will be described below with reference to the drawings. However, the present embodiment described below is merely an example of the present disclosure in all respects. Various improvements or modifications may be made without departing from the scope of the present disclosure. In implementing the present disclosure, a specific configuration according to the embodiment may be appropriately adopted. Note that while the data appearing in this embodiment is described in natural language, more specifically, it is specified in pseudo-language, commands, parameters, machine language, etc. that can be recognized by a computer.

[0027] §1 Examples of application 1 shows a schematic diagram of an example of a situation in which the present invention is applied. The system according to this embodiment includes a model generation device 1, an inference device 2, and an inference device 3. For ease of explanation, the inference device 2 may be referred to as a "first inference device" and the inference device 3 may be referred to as a "second inference device."

[0028] The model generating device 1 is one or more computers configured to generate a trained first model 4 and a trained second model 5 by machine learning. In this embodiment, the model generating device 1 directs the execution of machine learning of the first model 4 and the second model 5. The first model 4 is configured to estimate the position of a midpoint existing between two points in a given space from the positions of the two points. The midpoint may be a point existing between two points and having a predetermined ratio of distances from the two points. The ratio of distances (predetermined ratio) may be variable. The second model 5 is configured to estimate the distance between the two points from the positions of the two points. The machine learning includes training the first model 4 to acquire the ability to estimate the position of the midpoint using a distance estimation result by the second model 5, and training the second model 5 to acquire the ability to estimate distance using a midpoint estimation result of the first model 4 and a spatial metric. In this way, the trained first model 4 and the trained second model 5 are generated. The model generating device 1 outputs the result of the machine learning.

[0029] The inference device 2 is one or more computers configured to generate a geodesic line by repeating the estimation of the intermediate point. In this embodiment, the inference device 2 estimates the position of the intermediate point between a first point and a second point given in space using an inference model 6. The inference model 6 is configured to estimate the position of an intermediate point that exists between two points and has a predetermined ratio of distances from the two points, from the positions of the two points. In one example, the inference model 6 may use the trained first model 4. The inference device 2 adds the estimated intermediate point as a new point and recursively executes the step of estimating the intermediate point. In this way, the inference device 2 generates a geodesic line between the first point and the second point, which is composed of the estimated intermediate points. The inference device 2 outputs information about the generated geodesic line.

[0030] The reasoning device 3 is one or more computers configured to infer the distance between two points using a trained second model 5. In this embodiment, the reasoning device 3 holds the trained second model 5 generated by the model generation device 1. The reasoning device 3 infers the distance between two given points in space using the trained second model 5. The reasoning device 3 outputs information about the inferred distance.

[0031] As described above, in this embodiment, the model generating device 1 can generate a trained first model 4 that has acquired the ability to infer an intermediate point that exists between two points in a given space. By using the generated trained first model 4 to repeatedly and recursively infer intermediate points located at a predetermined ratio, a geodesic line can be generated while densely obtaining waypoints. Therefore, according to this embodiment, when generating a geodesic line, it is possible to obtain a sufficiently dense waypoint. In the inference device 2, a method of recursively and repeatedly estimating intermediate points located at a predetermined ratio using an inference model 6 is adopted as a method of generating a geodesic line, so that sufficiently dense waypoints can be obtained. In the inference device 3, a trained second model 5 is used to estimate the distance between two points. Note that obtaining dense waypoints corresponds to a small distance between waypoints. Whether or not the distance between waypoints is small may be evaluated, for example, by a threshold value or the like.

[0032] (Leads the machine learning implementation) Directing the execution of machine learning may include executing machine learning in the model generation device 1 and giving instructions to an external computer to execute machine learning. That is, in one example, the model generation device 1 may execute machine learning of the first model 4 and the second model 5. In another example, the model generation device 1 may execute machine learning of one of the first model 4 and the second model 5, and an external computer may execute machine learning of the other. In another example, the model generation device 1 may execute part of the processing of the machine learning of the first model 4 and the second model 5, and the external computer may execute the remaining processing in response to an instruction from the model generation device 1. In another example, the external computer may execute machine learning of the first model 4 and the second model 5 in response to an instruction from the model generation device 1. The external computer that executes the machine learning of the first model 4 and the external computer that executes the machine learning of the second model 5 may be the same or different. Information on the first model 4 and the second model 5 may be held in any device (model generation device 1, external computer).

[0033] (Outputs the results of machine learning) Outputting the machine learning result may be configured by outputting any information related to the machine learning result. In one example, outputting the machine learning result may include generating learning result data indicating the machine learning result and storing the generated learning result data in any storage area. In another example, outputting the machine learning result may include outputting a convergence result of training in machine learning. The convergence result may be indicated by, for example, a loss, a number of time steps, a learning curve, etc.

[0034] (Midpoint) The midpoint may be a point where the ratio of the distance from two points defined using a metric is predetermined (satisfies a predetermined condition), that is, a ratio that divides the distance between two points defined using a metric is defined. In one example, the predetermined ratio (ratio of distances) may be 1:1, and thus the midpoint may be a midpoint. The midpoint may be a point where the distance from two points defined using a metric is equal. The distance may include a quasi-pseudo distance. According to one example of the present embodiment, by repeatedly recursively estimating the midpoint, it is possible to expect the acquisition of sufficiently dense and uniform waypoints. However, the predetermined ratio is not limited to such an example, and may be appropriately selected according to the embodiment, for example, 1:2, 2:3, etc. Furthermore, the predetermined ratio may be fixed or may vary dynamically. The predetermined ratio may be set within a numerical range.

[0035] (Model configuration) The first model 4, the second model 5, and the inference model 6 are configured by machine learning models. The machine learning models are configured to have one or more calculation parameters that can be adjusted by machine learning. The one or more calculation parameters are used for the calculation of the target inference. Machine learning is to adjust (optimize) the value of the calculation parameter using a training sample. The machine learning models may be configured by, for example, a neural network, a regression model, a decision tree model, a support vector machine, other function formulas (calculation models), etc. The machine learning method may be appropriately selected according to the machine learning model to be adopted (for example, backpropagation method, etc.).

[0036] In one example, at least one of the first model 4, the second model 5, and the inference model 6 may include a neural network. The structure of the neural network may be appropriately determined according to the embodiment, and may be specified by, for example, the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) included in each layer, the connection relationship between the nodes in each layer, and the like. In one example, the neural network may have a recursive structure. In addition, the neural network may include any layer, such as a fully connected layer, a convolution layer, a pooling layer, a deconvolution layer, an unpooling layer, a normalization layer, a dropout layer, or a Long short-term memory (LSTM). The neural network may include any type of model, such as a diffusion model or a Transformer. The weight of the connection between each node included in the neural network and the threshold value of each node are examples of calculation parameters.

[0037] In this embodiment, the first model 4 and the inference model 6 are configured to receive input of information on two points and output an estimation result of a midpoint between the two points. The second model 5 is configured to receive input of information on two points and output an estimation result of the distance between the two points. As long as they are configured in this way, the form of input and output of the first model 4, the second model 5, and the inference model 6 is not particularly limited and may be appropriately selected according to the embodiment. At least one of the first model 4, the second model 5, and the inference model 6 may be configured to further receive input of information other than the positions of the two points. At least one of the first model 4 and the inference model 6 may be configured to further output any information other than the result of estimating the position of the midpoint. The second model 5 may be configured to further output any information other than the result of estimating the distance between the two points.

[0038] (Training the first model / second model) In this embodiment, the first model 4 is trained using the inference result (distance estimation result) of the second model 5. The second model 5 is trained using the spatial metric and the inference result (midpoint estimation result) of the first model 4. As long as it is configured in this way, the training method of the first model 4 and the second model 5 is not particularly limited and may be designed appropriately according to the embodiment. In one example, the first model 4 is treated as an actor and the second model 5 is treated as a critic, and thus, the training method of the first model 4 and the second model 5 is performed by treating the actor as an actor and the second model 5 as a critic. The Tur-Critic method may be used. In this case, the machine learning of the second model 5 may be performed using the inference result of the first model 4 after it has been updated (on-policy). Alternatively, the machine learning of the second model 5 may be performed using the inference result of the first model 4 before it has been updated (off-policy).

[0039] 2A illustrates an example of a training process for the first model 4 according to this embodiment. In this example of this embodiment, training the first model 4 may include the following steps (A) to (E). (A) sampling two points from a space SP, the two points being a first sample point (s) and a second sample point (g); (B) estimating a first location of a midpoint (m) between a first sample point (s) and a second sample point (g) using a first model (4); (C) using a second model 5 to estimate a first distance (d(s,m)) between a first sample point (s) and an estimated midpoint (m); (D) using a second model 5 to estimate a second distance (d(m,g)) between the estimated midpoint (m) and a second sample point (g); and (E) Training a first model 4 such that the estimated first distance (d(s,m)) and second distance (d(m,g)) satisfy a midpoint condition.

[0040] According to an example of this embodiment, the trained first model 4 can be appropriately generated. Note that the loss function in training the first model 4 in (E) is appropriately designed according to the embodiment. In one example, the loss function may include a term for learning the ability to estimate an intermediate point (m) that satisfies "αd(s,m)=d(m,g)". This allows the first model 4 to acquire the ability to estimate the position of an intermediate point whose distance ratio from two points is α. α may be set according to a predetermined ratio. For example, when the intermediate point is a midpoint, α may be 1. α may be set to a fixed value or may be set to a variable value. The value of α may be set in a numerical range. In addition, the loss function may include a term for learning the ability to estimate an intermediate point (m) that satisfies "d(s,m)+d(m,g) is minimum". This allows the first model 4 to acquire the ability to estimate the position of an intermediate point on the shortest route.

[0041] 2B is a schematic diagram showing an example of a training process of the second model 5 according to the present embodiment. In the example of the present embodiment, training the second model 5 may include the following steps (1) to (4). (1) Sampling two points from the space SP, the two points being a third sample point (s) and a fourth sample point (g); (2) estimating a third distance (d(s,g)) between a third sample point (s) and a fourth sample point (g) using a second model 5; (3) training a second model 5 such that, if the third sample point (s) and the fourth sample point (g) satisfy a proximity condition, the estimated third distance (d(s,g)) is equal to a value (c) given by the metric MT of the space SP; and (4) If the third sample point (s) and the fourth sample point (g) do not satisfy the proximity condition, (4-1) estimating a second location of a midpoint (m) between a third sample point (s) and a fourth sample point (g) using a first model (4); (4-2) adopting the combination of the third sample point (s) and the midpoint (m), and the combination of the midpoint (m) and the fourth sample point (g) as new combinations of the third sample point (s) and the fourth sample point (g), respectively, and repeating the process from (2) above; and (4-3) After repeating the process from (2) by (4-2), training the second model 5 so that the estimated value of the third distance (d(s,g)) between the (first) third sample point (s) and the fourth sample point (g) sampled in (1), estimated using the second model 5, is equal to the sum of the values ​​(c) given by the metric MT of the space SP.

[0042] According to an example of the present embodiment, the trained second model 5 can be appropriately generated. The loss function in the training of the second model 5 in (3) may be appropriately designed according to the embodiment. The first and second sample points sampled in the training of the first model 4 may be the same as or different from the third and fourth sample points sampled in the training of the second model 5. Similarly to the training of the second model 5, in the training of the first model 4, the combination of the first sample points and the midpoint estimated in (B) and the combination of the midpoint and the second sample point may be adopted as new combinations of the first and second sample points, respectively, and the process of (B) may be recursively repeated. Then, the processes of (C) to (E) may be performed on the first sample points, the midpoints, and the second sample points obtained by the recursive process. In one example, when the first sample point and the second sample point are the same as the third sample point and the fourth sample point, the process (4-1) of the second model 5 and the process (B) of the first model 4 may be executed in common in response to the process (4) in the machine learning of the second model 5. Then, the training processes (C) to (E) of the first model 4 may be executed together with the training process (3) of the second model 5.

[0043] In addition, the proximity conditions (3) and (4) in the machine learning of the second model 5 may be appropriately designed according to the embodiment, such as the accuracy of generating geodesic lines. In one example, the number of recursions in (4) in the machine learning of the second model 5 may be appropriately specified according to the settings of the machine learning conditions. In response to this, when the number of processes in (4) reaches the specified number of recursions, the third submodel may be It may be determined that the first sample point and the fourth sample point satisfy the proximity condition. On the other hand, if the number of times of processing in (4) has not reached the designated number of recursions, it may be determined that the third sample point and the third sample point do not satisfy the proximity condition. The number of recursions may be specified by a user or may be specified by computer processing. In addition, the number of recursions may be a constant value or a variable value. For example, in the early stage of machine learning, since the inference accuracy of both the first model 4 and the second model 5 is low, it may be difficult to train them appropriately even if the number of recursions is increased. Therefore, in the early stage of machine learning, the number of recursions may be set small, and as the machine learning progresses, the number of recursions may be increased. In another example, whether or not the proximity condition is satisfied may be determined according to a comparison between the third distance estimated in (2) and a threshold value.

[0044] Moreover, the processes (2) to (4) in the machine learning of the second model 5 correspond to the following processes (2A) to (4A). (2A) recursively using the first model 4 to estimate a number of intermediate points (as waypoints) between the sampled third and fourth sample points; (3A) obtaining distance estimates for each two adjacent points of the third sample point, the estimated intermediate points, and the fourth sample point using a metric MT of the space SP, and summing the obtained estimates to calculate an estimate of the distance between the third sample point and the fourth sample point; and (4A) training the second model 5 such that a value of a third distance between the third sample point and the fourth sample point estimated by the second model 5 is equal to an estimated value of the distance between the third sample point and the fourth sample point calculated by a spatial metric;

[0045] That is, training the second model 5 may include the above processes (1), (2A), (3A), and (4A). Furthermore, training the second model 5 may include performing processes (3A) and (4A) using the obtained midpoints each time a midpoint is recursively estimated in the process (2A).

[0046] In addition, the training method of the first model 4 and the second model 5 is not limited to the above example, and may be appropriately changed depending on the embodiment. In another example, training the second model 5 may include the following (3A) and (4A) instead of the above (3) and (4). (3A) training a second model 5 such that, if the third sample point and the fourth sample point are evaluated to be sufficiently close, a third distance (d(s,g)) between the third sample point and the fourth sample point is equal to a value given by a spatial metric; (4A) If the third and fourth sample points are not evaluated as being sufficiently close, (4A-1) estimating a second location of a midpoint between the third sample point and the fourth sample point using the first model; (4A-2) estimating a fourth distance (d(s,m)) between the third sample point and the midpoint and a fifth distance (d(m,g)) between the midpoint and the fourth sample point using a second model 5; (4A-3) Training the second model 5 so that the sum of the fourth distance (d(s,m)) and the fifth distance (d(m,g)) is equal to the third distance (d(s,g)). The criterion for whether or not something is sufficiently close may be determined appropriately depending on the accuracy with which the geodesic curve is generated.

[0047] In the method proposed in Non-Patent Document 1, a policy gradient method is adopted for training a model that predicts subgoals. In this policy gradient method, first, predictions of first-stage subgoals between two points are performed probabilistically multiple times. Then, predictions of second-stage subgoals are further performed between each of the obtained first-stage subgoals and each of the two points. Using these second-stage subgoals, evaluation of each first-stage subgoal (i.e., training of a model) is performed. In other words, in order to evaluate the n-th stage subgoals, a process of predicting the n+1-th stage subgoals is executed. This may result in a decrease in the efficiency of the overall training process. In contrast, in the above machine learning method of the present disclosure, the inference results of the second model 5 are used to train the first model 4. is used. This makes it possible to execute the training process for the first model 4 without performing recursive processing. Therefore, according to this example of the present embodiment, it is possible to improve the efficiency of the training process for the first model 4.

[0048] The space (space SP) may include any kind of space in which the length of a curve is defined, such as a Euclidean space, a Riemannian manifold, or a Finsler manifold. In one example, a metric that satisfies a trigonometric inequality may be defined in the space. For example, let the space be X. The quasi-metric of X may be defined as d:X×X→[0,∞) that satisfies the following (Equation 1) and (Equation 2).

[0049]

number

number

[0050] The metric (metric MT) defines the distance (length) between two points that are close to the desired accuracy. The distance may include a quasi-pseudo distance. In one example, the metric may be defined to provide an approximation of the distance between two points by an approximation method. The space and the metric may be expressed in any manner, such as a function, numerical data, etc. The space may include a real space, and the metric may be defined to provide an actual measurement value or an approximation value of the real space. Each point in the space may be associated with any type of data, such as an image, sound, or various sensor data, and the metric may be defined to provide a distance between the data. Also, each point in the space may be associated with any physical state, and the metric may be defined to reproduce a physical law (e.g., the refractive index of light, kinetic energy, etc.). The present disclosure may be directly applied to the problem of generating a geodesic line, or may be applied to any task that can be regarded as a problem of generating a geodesic line, such as path planning, state transition planning, and trajectory generation. The space and the metric may be appropriately defined depending on the task to be solved as a problem of generating a geodesic line.

[0051] As an example of a space, we will show an example of a pseudo-Finsler manifold. A pseudo-Finsler manifold is a differential manifold M with F:TM→[0,∞). TM=∪ x∈M T x M is the tangent bundle of M. T is the tangent space, and Tx is the set of tangent vectors going out from the point x. A pseudo-Finsler manifold satisfies the following conditions (Equation 3) and (Equation 4).

[0052]

number

number

[0053] Let γ:[0,1]→M be a differentiable curve on M. The length L of γ can be defined as follows (Equation 5). Then, for two points x, y∈M, the distance d is defined as follows (Equation 6) can be defined as follows:

[0054]

number

number

[0055] (Specific examples of training processes) Let (X,d) be a quasi-quasi-metric space. Assume that the metric is locally known by a function C:X×X→[0,∞).

[0056] We assume that whenever d(x,y)=0, C(x,y)=0. The function C is given appropriately to define the distance locally. For example, in a pseudo-Finsler manifold, in the coordinate system f:M→R n Given this, the distance can be calculated using the following (Equation 7).

[0057]

number

[0058] In one example of this embodiment, the first model 4 may be treated as an actor and the second model 5 as a critic, and training of the first model 4 and the second model 5 may be performed simultaneously. The first and second sample points may be the same as the third and fourth sample points. Furthermore, the processing of (4-1) of the second model 5 and (B) of the first model 4 may be performed in common, and the training processing of (3) of the second model 5 and the training processing of (C) to (E) of the first model 4 may be performed together. The midpoint may be a midpoint.

[0059] FIG. 2C shows an example of a pseudo program code for the training process of the first model 4 and the second model 5 when these forms are adopted. In the example of FIG. 2C, the first model 4 is represented by a function π, and the parameters of the first model 4 are represented by θ. The second model 5 is represented by a function V, and the parameters of the second model 5 are represented by φ. The loss (L critic ) is as follows The loss of the first model 4 (L actor ) may be defined as follows (Equation 9):

[0060]

number

number

[0061] The first term in (Equation 8) corresponds to the training of the second model 5 in (3) and (4-3) above. The second and third terms in (Equation 8) are added to ensure that V(x,x) = 0. On the other hand, the first and second terms in (Equation 9) aim to make π(s,g) the midpoint of s and g, and correspond to the training of the first model 4 in (E) above. Note that d(x,z) 2 +d(z,y) 2 takes a minimum value when z is the midpoint of x and y, as shown in Equation 10 below. The third and fourth terms in Equation 9 are added to ensure that V(x,π(x,x))=V(π(x,x),x)=0.

[0062]

number

[0063] The sample data used for training may be collected by a first model 4 with the current parameters. Two points (p 0,0 , p 0,1), and a sequence of points may be generated between the initial two points by repeatedly inserting a midpoint between two adjacent points using a first model 4 (i.e., recursively executing the above processes (4-1) and (B)). Pairs of two adjacent points in each iteration may be collected as sample data to be used for training. The depth of the iteration (number of iterations) D does not necessarily need to be constant, and may be determined by some schedule. The distance (estimated value) of the collected pairs may be simply calculated as the sum of C given by the following (Equation 11).

[0064]

number

[0065] That is, the Monte Carlo method may be used to train the first model 4 and the second model 5. After collecting sample data, the collected data is divided into mini-batches, and each of the losses (L actor , L critic ) The parameters of the first model 4 and the second model 5 may be optimized according to This optimization is done by iterating over N mini-batches. epochs The data collection and optimization process may be repeated a sufficient number of times to generate the trained first model 4 and second model 5.

[0066] (Inference model) In one example, the inference model 6 used in the inference device 2 may be the trained first model 4 generated by the model generation device 1. However, the method of generating a geodesic line by repeatedly recursively estimating midpoints at a predetermined ratio may also be applied to a model generated by a method other than the model generation method of the present disclosure. The inference model 6 only needs to have the ability to estimate the position of a midpoint from the positions of two points, and the method of generating the inference model 6 does not need to be limited to the method of generating the trained first model 4 in the model generation device 1 described above. In other words, the inference model 6 does not need to be the trained first model 4. In another example, the inference model 6 may be generated by supervised learning using a pair of two points sampled from space as training data and a true value of the midpoint between the two points as a teacher signal. Using the training data and the teacher signal, The multiple data sets that make up each may be collected as appropriate.

[0067] 3 shows a schematic diagram of an example of a process for generating a geodesic line according to this embodiment. In this embodiment, the inference device 2 first acquires information on two initial points, a first point and a second point (P1, P2), which are the start point and end point for generating a geodesic line. The initial first point and second point may be given appropriately. The initial first point and second point may be specified by a user, or may be automatically specified by computer processing (e.g., randomly, by satisfying a predetermined condition, etc.).

[0068] The inference device 2 uses the inference model 6 to infer the position of a midpoint between the first point and the second point, and adopts the acquired midpoint as a waypoint. A waypoint is a point on a geodesic line, and a line connecting the waypoints is a geodesic line. The inference device 2 adds the acquired midpoint as a new point, and uses the inference model 6 to infer a midpoint between two adjacent points, thereby recursively repeating a series of processes to further acquire midpoints. That is, the inference model 6 adopts the combination of the first point and the midpoint and the combination of the midpoint and the second point as new combinations of the first point and the second point, respectively, and recursively repeats the process of inferring the midpoint. In this way, the inference model 6 generates a geodesic line between the first point and the second point that were initially given.

[0069] In the example of FIG. 3, the inference device 2 acquires an intermediate point P3 between points P1 and P2 in the first stage of inference processing. Then, the inference device 2 acquires an intermediate point P4 between points P1 and P3, and an intermediate point P5 between points P3 and P2 in the second stage of inference processing. The inference device 2 can obtain sufficiently dense waypoints by repeatedly executing the process of generating the intermediate points recursively. The number of repetitions (recursion count) may be appropriately specified according to the embodiment. The number of repetitions may be specified by a user or may be automatically specified by computer processing. The number of repetitions may be given according to any criterion. In one example, the inference device 2 may repeat the process of recursively acquiring intermediate points until the maximum value of the distance between two adjacent points in the obtained point sequence becomes less than a threshold value. The trained second model 5 may be used for estimating the distance.

[0070] Basically, the given space (space SP) in the training stage may be the same as the space in the inference stage where the geodesic line is generated. However, the space in the training stage and the space in the inference stage may not necessarily be the same. The space in the training stage and the space in the inference stage may be different to the extent that the trained model is applicable. For example, the space in the training stage may be a simulated space, whereas the space in the inference stage may be a real space. For convenience, the space in the training stage may be called a learning space or a training space, and the space in the inference stage may be called an inference space.

[0071] (task) The generation of a geodesic curve by the reasoning device 2 may be performed to solve a direct task aimed at obtaining a geodesic curve, or may be performed to solve an indirect task that can be regarded as a problem of generating a geodesic curve. Indirect tasks may include, for example, path planning, state transition planning, trajectory generation, etc. The generated geodesic curve may have various meanings depending on the task to be solved. The same is true for the generation of a geodesic curve by the trained first model 4.

[0072] In one example, in order to perform route planning, the reasoning device 2 may generate a geodesic curve. That is, the geodesic curve may be a route. According to one example of the present embodiment, in a route planning scene, sufficiently dense waypoints can be obtained, and thus, an appropriate route planning can be expected.

[0073] Path planning may be performed for various purposes. In one example, the path may be a motion path of a robotic device. According to one example of the present embodiment, the motion path of the robotic device is planned. In a scene where the robot device is to be determined, a sufficiently dense number of waypoints can be obtained, and thus it is expected that an appropriate operating path can be obtained. The type of the robot device is not particularly limited and may be appropriately selected depending on the embodiment. The robot device may include, for example, an industrial robot in a production line, an autonomous robot configured to be able to operate autonomously, etc. The industrial robot may include, for example, a vertical articulated robot, a horizontal articulated robot (SCARA robot), a parallel link robot, an orthogonal robot, etc. The autonomous robot may include, for example, a humanoid robot, a guide robot, an agricultural robot, a care robot, a security robot, a transport robot, etc. The content of the autonomous processing may be appropriately selected depending on the embodiment.

[0074] In another example, the route may be a travel route of a moving object. According to one example of this embodiment, when planning a travel route of a moving object, sufficiently dense waypoints can be obtained, and thus, it is expected that an appropriate travel route can be obtained. The type of moving object is not particularly limited, and may be appropriately selected according to the embodiment. The moving object may be appropriately configured to be movable. The moving object may include, for example, a cleaning robot, the above-mentioned autonomous robot configured to be movable, a vehicle configured to be automatically driven, an aircraft capable of automatically flying (such as a drone), and the like.

[0075] In addition, in one example, in order to infer the state transition of the device, the inference device 2 may generate a geodesic curve. That is, each point in the space may be configured to indicate the state of the device. The geodesic curve may indicate the state transition of the device. According to one example of the present embodiment, in a situation where the state transition of the device is inferred, sufficiently dense waypoints can be obtained, and thus, it is possible to expect the acquisition of an appropriate state transition.

[0076] The type of device is not particularly limited and may be appropriately selected depending on the embodiment. In one example, the device may be a robot device or a moving body. According to one example of the present embodiment, when inferring the state transition of the robot device or the moving body, sufficiently dense waypoints can be obtained, and thus, it is expected that an appropriate state transition can be obtained.

[0077] Also, in one example, each point in the space may be associated with any type of data, such as an image, sound, various sensor data, etc., and accordingly, the inference device 2 may generate a geodesic line to infer a transition of the data. For example, each point in the space may be configured to indicate an image. The geodesic line may indicate a transition of the image. According to one example of the present embodiment, in a scene where an image transition is inferred, sufficiently dense waypoints can be obtained, and thus, it is expected that an appropriate image transition (morphing) can be obtained.

[0078] Also, in one example, in order to simulate the physical laws, the inference device 2 may generate geodesic lines. That is, each point in space may be configured to indicate the state of an element given by the simulator. The element is a character in the simulation. The type and number of elements may be appropriately determined according to the contents of the simulation. The geodesic lines may indicate the trajectory of the state in accordance with the physical laws in the element. The trajectory of the state may correspond to the simulation result of the motion. The physical laws to be simulated may be appropriately selected according to the embodiment. The physical laws may include, for example, the refractive index of light, kinetic energy, etc. According to one example of the present embodiment, in a scene where the trajectory of the state in accordance with the physical laws is inferred, it is possible to obtain sufficiently dense waypoints, and thus it is possible to expect the acquisition of an appropriate state transition (for example, the simulation result of the physical laws).

[0079] (Cost function) In one example, a cost may be taken into account when generating a geodesic curve by the trained first model 4 and the inference model 6. In the training stage, the spatial metric may include a cost function. According to one example of the present embodiment, the geodesic curve is generated after taking into account a cost factor. The cost may be defined appropriately according to the task to be solved. In one example, the cost may be defined according to time, cost, risk, congestion, attractiveness, and the like. The cost may be defined according to factors such as gradient, presence or absence of obstacles, kinematics of the device, asymmetry (different for going and returning), and the like. The factors related to kinematics may include, for example, motion restrictions (e.g., turning radius of the vehicle, etc.), aging deterioration, amount of acceleration / deceleration, type of cargo, and the like. In one example, in a scene where the transition of the above image is solved, the cost may be defined according to the naturalness of the image. In one example, each point in the space may be configured to indicate a character string (sentence). The geodesic line may indicate the transition of the character string. In this scene, the cost may be defined according to the naturalness of the character string. This cost function may be omitted.

[0080] (System Configuration) In one example, as shown in Fig. 1, a model generating device 1, an inference device 2, and an inference device 3 may be connected to each other via a network. The type of network may be appropriately selected from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, and the like. However, the method of exchanging data between each device is not limited to this example, and may be appropriately selected depending on the embodiment. In another example, data may be exchanged using a storage medium.

[0081] 1, the model generation device 1, the inference device 2, and the inference device 3 are each separate computers. However, the configuration of the system according to this embodiment is not limited to such an example, and may be appropriately determined according to the embodiment. In another example, a combination of at least one of the model generation device 1, the inference device 2, and the inference device 3 may be configured by a single computer. In yet another example, at least one of the model generation device 1, the inference device 2, and the inference device 3 may be configured by multiple computers.

[0082] §2 Configuration Example [Hardware configuration] (Model Generator) Fig. 4 illustrates a schematic example of a hardware configuration of the model generating device 1 according to this embodiment. In the example of Fig. 4, the model generating device 1 according to this embodiment is a computer to which a control unit 11, a storage unit 12, a communication interface 13, an input device 14, an output device 15, and a drive 16 are electrically connected.

[0083] The control unit 11 includes a hardware processor, such as a CPU (Central Processing Unit), The control unit 11 includes a random access memory (RAM), a read only memory (ROM), and the like, and is configured to execute information processing based on a program and various data. The control unit 11 (CPU) is an example of a processor resource. The storage unit 12 may be configured, for example, by a hard disk drive, a solid state drive, or the like. The storage unit 12, the RAM, and the ROM are examples of memory resources. In this embodiment, the storage unit 12 stores various information such as a model generation program 81, first learning result data 40, second learning result data 50, and the like.

[0084] The model generation program 81 is a program for causing the model generation device 1 to execute information processing (FIG. 10 described below) related to the generation of the trained first model 4 and the trained second model 5. The model generation program 81 includes a series of instructions for the information processing. The first learning result data 40 is configured to indicate information related to the trained first model 4 that has been generated. The second learning result data 50 is configured to indicate information related to the trained second model 5 that has been generated. In this embodiment, the first learning result data 40 and the second learning result data 50 may be generated as a result of executing the model generation program 81.

[0085] As long as information for executing the computational processing of each trained model (4, 5) can be held, the configuration of each learning result data (40, 50) is not particularly limited and may be appropriately determined depending on the embodiment. In one example, each learning result data (40, 50) may be configured to include information indicating values ​​of computational parameters adjusted by machine learning. In some cases, each learning result data (40, 50) may be configured to include information indicating the configuration of each model (4, 5) (e.g., the structure of a neural network, etc.).

[0086] The communication interface 13 is configured to perform wired or wireless communication via a network. The communication interface 13 may be configured, for example, by a wired LAN (Local Area Network) module, a wireless LAN module, etc. The model generation device 1 may perform data communication with other computers (for example, the inference device 2, the inference device 3, etc.) via the communication interface 13.

[0087] The input device 14 is a device for performing input, such as a mouse, a keyboard, etc. The output device 15 is a device for performing output, such as a display, a speaker, etc. An operator can operate the model generation device 1 by using the input device 14 and the output device 15. The input device 14 and the output device 15 may be connected via an external interface. The input device 14 and the output device 15 may be integrally configured, for example, by a touch panel display, etc.

[0088] The drive 16 is a device for reading various information such as a program stored in a storage medium 91. At least one of the model generation program 81, the first learning result data 40, and the second learning result data 50 may be stored in the storage medium 91 instead of the storage unit 12 or together with the storage unit 12. The storage medium 91 is configured to accumulate various information (stored programs, etc.) by electrical, magnetic, optical, mechanical, or chemical action so that a machine such as a computer can read the information. The model generation device 1 may acquire the model generation program 81 from the storage medium 91. The storage medium 91 may be a disk-type storage medium such as a CD or a DVD, or may be a non-disk-type storage medium such as a semiconductor memory (e.g., a flash memory). The type of the drive 16 may be appropriately selected according to the type of the storage medium 91. The drive 16 may be connected via an external interface. The external interface may be appropriately configured to connect to an external device by wire or wirelessly, for example, via a Universal Serial Bus (USB) port, a dedicated port, or the like.

[0089] In addition, with regard to the specific hardware configuration of the model generating device 1, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processor may be a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), a GPU, or the like. The communication interface 13, the input device 14, the output device 15, etc. may be configured by a PU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), etc. , and at least one of the drive 16 may be omitted. The model generating device 1 may be composed of a plurality of computers. In this case, the hardware configurations of the computers may or may not be the same. Furthermore, the model generating device 1 may be an information processing device designed specifically for the service to be provided, as well as a general-purpose server device, a general-purpose PC (Personal Computer), a tablet PC, a terminal device, etc.

[0090] (1st inference device) Fig. 5 shows a schematic diagram of an example of a hardware configuration of the inference device 2 according to this embodiment. In the example shown in Fig. 5, the inference device 2 according to this embodiment is a computer in which a control unit 21, a storage unit 22, a communication interface 23, an input device 24, an output device 25, and a drive 26 are electrically connected.

[0091] The control unit 21 to the drive 26 and the storage medium 92 of the inference device 2 may be configured similarly to the control unit 11 to the drive 16 and the storage medium 91 of the above-mentioned model generation device 1. The control unit 21 (CPU) is an example of a processor resource of the inference device 2, and the storage unit 22 (and RAM, ROM) is an example of a memory resource of the inference device 2. In this embodiment, the storage unit 22 stores various information such as an inference program 82, inference model data 60, etc.

[0092] The inference program 82 is a program for causing the inference device 2 to execute information processing relating to the generation of geodesic lines (FIG. 11 described later). The inference program 82 includes a series of instructions for the information processing. The inference model data 60 is configured to indicate information relating to the inference model 6. When a trained first model 4 is adopted as the inference model 6, the inference model data 60 may be the first learning result data 40. At least one of the inference program 82 and the inference model data 60 may be stored in a storage medium 92 instead of or together with the storage unit 22. The inference device 2 may acquire at least one of the inference program 82 and the inference model data 60 from the storage medium 92.

[0093] The inference device 2 may perform data communication with another computer (e.g., the model generation device 1) via the communication interface 23. An operator can operate the inference device 2 by using the input device 24 and the output device 25.

[0094] In addition, regarding the specific hardware configuration of the inference device 2, components can be omitted, replaced, and added as appropriate depending on the embodiment. For example, the control unit 21 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, FPGA, DSP, GPU, ASIC, etc. At least one of the communication interface 23, the input device 24, the output device 25, and the drive 26 may be omitted. The inference device 2 may be configured with multiple computers. In this case, the hardware configurations of the computers may be the same or may not be the same. The inference device 2 may be an information processing device designed exclusively for the service provided, as well as a general-purpose server device, a general-purpose PC, a tablet PC, a terminal device, etc.

[0095] (Second reasoning device) Fig. 6 shows a schematic diagram of an example of a hardware configuration of the inference device 3 according to this embodiment. In the example shown in Fig. 6, the inference device 3 according to this embodiment is a computer in which a control unit 31, a storage unit 32, a communication interface 33, an input device 34, an output device 35, and a drive 36 are electrically connected.

[0096] The control unit 31 to the drive 36 and the storage medium 93 of the inference device 3 may be configured similarly to the control unit 11 to the drive 16 and the storage medium 91 of the above-described model generation device 1, respectively. The control unit 31 (CPU) is an example of a processor resource of the inference device 3, and the storage unit 32 (and RAM, ROM) is an example of a memory resource of the inference device 3. In this embodiment, the storage unit 32 stores various information such as an inference program 83, second learning result data 50, etc.

[0097] The inference program 83 is a program for causing the inference device 3 to execute information processing related to distance estimation (FIG. 12 described below). The inference program 83 includes a series of instructions for the information processing. At least one of the inference program 83 and the second learning result data 50 may be stored in a storage medium 93 instead of or together with the storage unit 32. The inference device 3 may acquire at least one of the inference program 83 and the second learning result data 50 from the storage medium 93.

[0098] The inference device 3 may perform data communication with another computer (e.g., the model generation device 1) via the communication interface 33. An operator can operate the inference device 3 by using the input device 34 and the output device 35.

[0099] In addition, regarding the specific hardware configuration of the inference device 3, components can be omitted, replaced, and added as appropriate depending on the embodiment. For example, the control unit 31 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, FPGA, DSP, GPU, ASIC, etc. At least one of the communication interface 33, the input device 34, the output device 35, and the drive 36 may be omitted. The inference device 3 may be configured with multiple computers. In this case, the hardware configurations of the computers may be the same or may not be the same. The inference device 3 may be an information processing device designed exclusively for the service provided, as well as a general-purpose server device, a general-purpose PC, a tablet PC, a terminal device, etc.

[0100] [Software configuration] (Model Generator) 7 shows an example of a software configuration of the model generation device 1 according to this embodiment. The control unit 11 of the model generation device 1 loads a model generation program 81 stored in the storage unit 12 into the RAM, and executes instructions included in the model generation program 81 by the CPU. As a result, the model generation device 1 operates as a computer including a learning processing unit 111 and an output unit 112 as software modules.

[0101] The learning processing unit 111 is configured to supervise the execution of machine learning of the first model 4 and the second model 5. The machine learning includes training the first model 4 to acquire the ability to estimate the position of the midpoint using the distance estimation result by the second model 5, and training the second model 5 to acquire the ability to estimate the distance using the midpoint estimation result of the first model 4 and a spatial metric. The output unit 112 is configured to output the result of the machine learning.

[0102] (1st inference device) 8 shows a schematic diagram of an example of the software configuration of the inference device 2 according to this embodiment. The control unit 21 of the inference device 2 loads an inference program 82 stored in the storage unit 22 into the RAM, and executes instructions included in the inference program 82 by the CPU. This causes the inference device 2 to operate as a computer including a position inference unit 211 and an output unit 212 as software modules.

[0103] The position inference unit 211 has an inference model 6 (trained first model 4) by holding inference model data 60 (first learning result data 40). The position inference unit 211 is configured to infer the position of a midpoint between a first point and a second point given in space using the inference model 6. The position inference unit 211 is configured to generate a geodesic line by recursively repeating the process of adding the inferred midpoint as a new point and inferring a midpoint between two adjacent points. The output unit 212 is configured to output information related to the generated geodesic line.

[0104] (Second reasoning device) 9 shows a schematic diagram of an example of the software configuration of the inference device 3 according to this embodiment. The control unit 31 of the inference device 3 loads an inference program 83 stored in the storage unit 32 into the RAM, and executes instructions included in the inference program 83 by the CPU. As a result, the inference device 3 is a computer having a distance inference unit 311 and an output unit 312 as software modules. It acts as a

[0105] The distance inference unit 311 has a trained second model 5 by holding the second learning result data 50. The distance inference unit 311 is configured to infer a distance between two given points in space using the trained second model 5. The output unit 312 is configured to output information related to the inferred distance.

[0106] (others) In the present embodiment, an example is described in which each software module of the model generating device 1, the inference device 2, and the inference device 3 is realized by a general-purpose CPU. However, some or all of the above software modules may be realized by one or more dedicated processors or chipsets. Each of the above modules may be realized as a hardware module. With regard to the software configurations of the model generating device 1, the inference device 2, and the inference device 3, modules may be omitted, replaced, or added as appropriate depending on the embodiment.

[0107] §3 Example of operation (Model Generator) 10 is a flowchart showing an example of the processing procedure of the model generating device 1 according to this embodiment. The following processing procedure is an example of a model generating method executed by a computer. However, the processing procedure of the model generating device 1 is merely an example, and each step may be changed as much as possible. In addition, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0108] In step S101, the control unit 11 operates as a learning processing unit 111 and directs the execution of machine learning of the first model 4 and the second model 5. The machine learning of the first model 4 and the second model 5 may be executed in the model generating device 1, or may be executed in an external computer other than the model generating device 1. In one example, the training of the first model 4 may include the above steps (A) to (E) (FIG. 2A). The training of the second model 5 may include the above steps (1) to (4) (FIG. 2B). In one example, the midpoint may be a midpoint, and the training process of the first model 4 and the second model 5 may be executed according to the pseudocode shown in FIG. 2C. When the training of the first model 4 and the second model 5 is completed, the control unit 11 proceeds to the next step S102.

[0109] In step S102, the control unit 11 operates as the output unit 112 and outputs the result of the machine learning. The output destination and the content of the information to be output may be appropriately selected according to the embodiment. In one example, the control unit 11 may generate learning result data (first learning result data 40, second learning result data 50) indicating the result of the machine learning as the output process, and store the generated learning result data in a predetermined storage area. The predetermined storage area may be, for example, a RAM in the control unit 11, the storage unit 12, an external storage device, a storage medium, or a combination of these. The storage medium may be, for example, a CD, a DVD, a semiconductor memory, or the like. The external storage device may be, for example, a data server such as a NAS (Network Attached Storage). The external storage device may be, for example, an external storage device. When the machine learning of the model is executed on an external computer, the learning result data may be generated by the external computer. In another example, the control unit 11 may output a convergence result of training in the machine learning as the output process. The output destination may be, for example, the RAM in the control unit 11, the output device 15, an external computer, an external storage device, a storage medium, or a combination of these.

[0110] When the output of the machine learning result is completed, the control unit 11 ends the processing procedure of the model generation device 1 according to this operation example. When using the inference device 200, the generated first learning result data 40 may be provided from the model generation device 1 or an external computer to the inference device 2 at any timing and in any manner. In one example, the first learning result data 40 may be provided to the inference device 2 via a network. In another example, the first learning result data 40 may be provided to the inference device 2 via a storage medium. In another example, the first learning result data 40 may be pre-installed in the inference device 2. Similarly, the second learning result data 50 may be provided from the model generation device 1 or an external computer to the inference device 3 at any timing and in any manner.

[0111] Furthermore, the control unit 11 may update or newly generate the first learning result data 40 and the second learning result data 50 by periodically or irregularly repeating the processes of steps S101 and S102. During this repetition, at least a part of the machine learning conditions may be changed, modified, added, deleted, or the like as appropriate. Then, the control unit 11 may provide the updated or newly generated first learning result data 40 to the inference device 2, thereby updating the first learning result data 40 held by the inference device 2. Furthermore, the control unit 11 may update the second learning result data 50 held by the inference device 3, by providing the updated or newly generated second learning result data 50 to the inference device 3.

[0112] (1st inference device) 11 is a flowchart showing an example of a processing procedure of the inference device 2 according to this embodiment. The following processing procedure is an example of an inference method executed by a computer. However, the processing procedure of the inference device 2 is merely an example, and each step may be modified as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0113] In step S201, the control unit 21 operates as a position inference unit 211 and uses the inference model 6 to infer the position of a midpoint between a first point and a second point given in space. The initial first point and second point may be given appropriately. The control unit 21 inputs information on the given first point and second point into the inference model 6 and executes the calculation processing of the inference model 6. As a result, the control unit 21 obtains the inference result of the position of the midpoint from the inference model 6.

[0114] In step S202, the control unit 21 operates as the position inference unit 211 and determines whether or not to repeat the process of step S201. As described above, the number of times to repeat may be specified by the user or may be automatically specified by computer processing. The number of times to repeat may be given according to any criterion. For example, when it is determined that the process of step S201 is to be repeated depending on the fact that the set number of times has not been reached, the criterion is not met, etc., the control unit 21 adds the estimated midpoint as a new point, returns the process to step S201, and executes again the estimation of the midpoint between two adjacent points. By repeatedly executing the process of step S201 recursively, the control unit 21 can generate a geodesic line between the first point and the second point that are initially given, and that is composed of a plurality of estimated midpoints. On the other hand, when it is determined that the process of step S201 is not to be repeated, the control unit 21 advances the process to the next step S203.

[0115] In step S203, the control unit 21 operates as the output unit 212 and outputs information related to the generated geodesic line. The output destination and the content of the information to be output may each be appropriately selected depending on the embodiment. In one example, the control unit 21 may output the generation result of the geodesic line as it is. In another example, the control unit 21 may execute any information processing related to the generated geodesic line. Then, the control unit 21 may output the result of executing the information processing as information related to the geodesic line. The output destination may be, for example, a RAM in the control unit 21, the output device 25, an external computer, an external storage device, a storage medium, or a combination of these.

[0116] When the output of the information is completed, the control unit 21 ends the processing procedure of the inference device 2 according to this operation example. Note that the control unit 21 may execute the series of information processing from step S201 to step S203 at any timing, such as a user request.

[0117] (Second reasoning device) 12 is a flowchart showing an example of a processing procedure of the inference device 3 according to this embodiment. The following processing procedure is an example of an inference method executed by a computer. However, the processing procedure of the inference device 3 is merely an example, and each step may be modified as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0118] In step S301, the control unit 31 operates as the distance inference unit 311 and uses the trained second model 5 to infer the distance between two given points in space. The two points (first point and second point) may be appropriately given, similar to the above step S201. The control unit 31 inputs information on the two given points to the trained second model 5 and executes the calculation process of the trained second model 5. As a result, the control unit 31 obtains the result of inferring the distance between the two points from the trained second model 5. When the distance estimation result is obtained, the control unit 31 proceeds to the next step S302.

[0119] In step S302, the control unit 31 operates as the output unit 312 and outputs information related to the estimated distance. The output destination and the content of the information to be output may be appropriately selected according to the embodiment. In one example, the control unit 31 may directly output the result of estimating the distance. In another example, the control unit 31 may execute any information processing based on the result of estimating the distance. Then, the control unit 31 may output the result of executing the information processing as information related to the distance. The output destination may be, for example, a RAM in the control unit 31, the output device 35, an external computer, an external storage device, a storage medium, or a combination of these.

[0120] When the output of the information is completed, the control unit 31 ends the processing procedure of the inference device 3 according to this operation example. Note that the control unit 31 may execute the series of information processing of the above steps S301 and S302 at any timing, such as at the request of a user.

[0121] [Features] In this embodiment, the model generating device 1 can generate a trained first model 4 that has acquired the ability to estimate an intermediate point between two points in a given space by the process of step S101. By using the generated trained first model 4, it is possible to generate a geodesic line while densely obtaining waypoints by repeatedly recursively estimating intermediate points located at a predetermined ratio. Therefore, according to this embodiment, when generating a geodesic line, it is possible to obtain sufficiently dense waypoints. Furthermore, the inference device 2 can obtain sufficiently dense waypoints by adopting a method of recursively estimating intermediate points located at a predetermined ratio as a method of generating a geodesic line by using the inference model 6 by the process of steps S201 and S202. Furthermore, the inference device 3 can estimate the distance between two points by using the trained second model 5 by the process of step S301.

[0122] § 4 Variations Although the embodiments of the present disclosure have been described above in detail, the above description is merely illustrative of the present disclosure in all respects. In the above-described embodiments, various improvements or modifications may be made as appropriate. For example, in the system according to the above-described embodiment, the inference device 3 may be omitted. One of the model generation device 1 and the inference device 2 may be omitted. Furthermore, for example, the inference device 2 may further use a trained second model 5 to calculate the distance between each waypoint that constitutes a geodesic line. The distance may be calculated.

[0123] §5 Example The following experiments were carried out to verify the effectiveness of the above-described embodiment, however, the present invention is not limited to the following examples.

[0124] (First embodiment) The method of the above embodiment and the method of FIG. 2C were adopted as the model generation method of the first example. The above (Equation 9) was adopted for the loss of the first model, and the above (Equation 8) was adopted for the loss of the second model. The number of epochs (N epochs) was set to 10, and the batch size was set to 256. We also gradually increased the iteration depth for collecting sample data to a defined depth D.

[0125] (Second Example) In the model generation method of the second embodiment, the loss of the first model was replaced from the above (Equation 9) to the following (Equation 12). Other conditions in the second embodiment were set to be the same as those in the first embodiment. From the following (Equation 13), the loss of (Equation 12) means that the first model is trained to estimate the 2:1 midpoint.

[0126]

number

number

[0127] (First Comparative Example) In the first comparative example, we use the reference 1 (John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov, “Proximal policy optimization algorithms”, [online], [Retrieved September 22, 2023], Internet<URL:https: / / arxiv.org / abs / 1707.06347> ) and constructed a method for generating a model that sequentially acquires waypoints using general goal-conditional reinforcement learning. The reward function is defined using the pseudo-Finsler metric.

[0128] (Second Comparative Example) As a second comparative example, the reinforcement learning method described in Non-Patent Document 1 was adopted as the model generation method. As a subgoal, the value to be optimized during training was changed so as to estimate the midpoint.

[0129] (First experiment) The environment for the first experiment was the same as that of Reference 2 (Makoto Matsumoto, “A slope of a mountain is a finsler surface with respect to a time measure”, [online], [Retrieved September 22, 2023], Internet <URL:https: / / www.semanticscholar.org / paper / A-slope-of-a-mountain-is-a-Finsler-surface-with-to-Matsumoto / a7ab70674cbf5dd4d7d89b5aab10c4316e81f895> We prepared a space of Matsumoto metrics proposed in

[14] . Matsumoto metrics are asymmetric Finsler metrics that take into account the travel time on an inclined surface. Matsumoto metrics F:TM→[0,∞) are defined as follows (Eq. 14) and (Eq. 15). Ta.

[0130]

number

number

[0131] A field was set with a disk M of radius 1 centered at the origin and f(p) in the following (Equation 16). For this space, a model was trained using the methods of the first and second examples and the first and second comparative examples. The number of time steps was approximately 2×10 7 was set to.

[0132]

number

[0133] In addition, two points s, g∈M were randomly sampled in this space, and a geodesic line was generated between the two obtained points (s, g) using the models of each embodiment and each comparative example. In the point sequence constituting the geodesic line, it was evaluated whether or not all the local distances between two adjacent points were less than the threshold ε. When all the local distances were less than the threshold ε, it was evaluated that the generation of a geodesic line having sufficiently dense waypoints was successful. In the first experiment, the number of segments n in the model of the first comparative example was set to 64. Correspondingly, the depth of repetition D in the models of the first to second embodiments and the second comparative example was set to 6 (i.e., 63 waypoints were obtained). The threshold ε was set to 0.1. The success rate of each embodiment and each comparative example was measured for 100 pairs of randomly selected two points, and the progress due to learning was plotted. Furthermore, after the machine learning was completed, a geodesic line was generated between the two randomly sampled points using the models of the first to second embodiments and the first to second comparative examples.

[0134] (Second experiment) For the environment of the second experiment, we used reference 3 ("Kinodynamic trajectory optimization and control for car-like robots", [online], [searched on September 22, 2023], Internet). <URL:https: / / rst.etit.tu-dortmund.de / storages / rst-etit / r / Global / Paper / Roesmann / 2017_Roesmann_IROS.PDF> We prepared a space of car-like metrics proposed in

[1999] . Car-like metrics are pseudo-Finsler metrics that consider the trajectory of a vehicle moving in one direction. M is defined as the following (Equation 17), and the Car-Like metric is defined as the following (Equations 18) to (Equation 20).

[0135]

number

number

number

number

[0136] where c p is the penalty coefficient, and r min is the lower limit of the radius of curvature. p Set to 100 and r min was set to 0.5. U was set to a disk of radius 1 centered at the origin. A one-way field was set by adding the term h2 to the term proposed in Reference 3. For this space, models were trained using the methods of the first embodiment and the first to second comparative examples. The number of time steps was approximately 8 × 10 7 In the second experiment, the number of segments n in the model of the first comparative example was set to 256. Correspondingly, the depth of iteration D in the models of the first comparative example and the second comparative example was set to 8 (i.e., 255 waypoints were obtained). The threshold ε was set to 0.1. The success rate of the first example and each comparative example was measured for 100 pairs of randomly selected two points, and the progression due to learning was plotted.

[0137] (Experimental Results) FIG. 13 shows the plot results of the first experiment. FIG. 14 shows the results of generating a geodesic line in the first experiment. FIG. 15 shows the plot results of the second experiment. As shown in FIG. 13, in each example, the success rate of generating a geodesic line having sufficiently dense waypoints was higher than in each comparative example. As a result, as shown in FIG. 14, in each example, a geodesic line closer to the true value of the shortest route could be obtained than in each comparative example. Also, in the result of FIG. 15, in the first example, the success rate of generating a geodesic line having sufficiently dense waypoints was higher than in each comparative example. From these results, it was found that the model generation method of this embodiment is effective in generating a geodesic line having sufficiently dense waypoints. It was also found that the model generation method of this embodiment is effective for route planning.

[0138] This specification includes the following disclosure. [Appendix 1] A computer (1) directing the execution of machine learning of the first model (4) and the second model (5); outputting the results of the machine learning; Run The first model (4) is configured to infer, from the positions of two points in a given space, the position of an intermediate point lying between the two points; The second model (5) is configured to infer a distance between the two points from the positions of the two points; The machine learning method includes: training the first model (4) to acquire the ability to estimate the location of the waypoint using the distance estimation results of the second model (5); and training the second model (5) to acquire the ability to estimate the distance using the midpoint estimation results of the first model (4) and a metric of the space; Including, Model generation method. [Appendix 2] The midpoint is a midpoint. 2. A method for generating a model as described in Appendix 1. [Appendix 3] Training the first model (4) Sampling two points from the space, the first sample point and the second sample point; estimating a first location of a midpoint between the first sample point and the second sample point using the first model (4); using the second model (5) to estimate a first distance between the first sample point and the estimated midpoint; using the second model (5) to estimate a second distance between the estimated midpoint and the second sample point; and training the first model (4) such that the estimated first and second distances satisfy the midpoint condition; Including, 3. The model generation method according to claim 1 or 2. [Appendix 4] Training the second model (5) comprises: (I) Two points consisting of the third and fourth sample points are sampled from the space. Pulling, (II) estimating a third distance between the third sample point and the fourth sample point using the second model (5); (III) if the third sample point and the fourth sample point satisfy a proximity condition, training the second model (5) such that the determined third distance is equal to a value given by a metric of the space; and (IV) if the third sample point and the fourth sample point do not satisfy a proximity condition, (IV-1) estimating a second location of a midpoint between the sampled third sample point and the sampled fourth sample point using the first model (4); (IV-2) adopting the combination of the third sample point and the intermediate point, and the combination of the intermediate point and the fourth sample point as new combinations of the third sample point and the fourth sample point, respectively, and repeating the process from (II); and (IV-3) After repeating the process from (II) by (IV-2), the third sample point sampled in (I) and the previous sample point are estimated using the second model (5). training the second model (5) such that an estimate of the third distance of the fourth sample point is equal to a sum of values ​​given by a metric in the space; Including, 4. The model generation method of any one of claims 1 to 3. [Appendix 5] The spatial metric includes a cost function. 5. The model generation method of any one of claims 1 to 4. [Appendix 6] On the computer (2), - a step of estimating the position of a midpoint between a first point and a second point given in space using an inference model (6), said inference model (6) being configured to estimate the position of a midpoint between two points, the midpoint being between the first point and the second point and having a predetermined ratio of distances from the two points, from the positions of the two points; generating a geodesic line between the first point and the second point, the geodesic line being composed of a plurality of the estimated intermediate points, by repeatedly performing the estimation step recursively; outputting information about the generated geodesic curve; In order to execute Inference programs (82). [Appendix 7] The geodesic curve is a path. The inference program (82) described in Appendix 6. [Appendix 8] The path is a motion path of a robotic device. The inference program (82) described in Appendix 7. [Appendix 9] The route is a travel route of a moving object. The inference program (82) described in Appendix 7. [Appendix 10] each point in the space is configured to indicate a state of the device; the geodesic curve represents a state transition of the device; The inference program (82) described in Appendix 6. [Appendix 11] The device is a robot device or a mobile object. The inference program (82) described in Appendix 10. [Appendix 12] Each point in the space is configured to represent an image; the geodesic curves represent transitions in the image; The inference program (82) described in Appendix 6. [Appendix 13] each point in the space is configured to represent a state of an element given by a simulator; The geodesic curve represents a locus of states in the element that obeys physical laws. The inference program (82) described in Appendix 6. [Appendix 14] A computer (3) storing the trained second model (5) generated by the model generation method according to any one of appendices 1 to 5, using said trained second model (5) to estimate the distance between two given points in space; outputting information regarding the estimated distance; In order to execute Inference programs (83). [Explanation of symbols]

[0139] 1...Model generation device, 11: control unit, 12: storage unit, 13: communication interface, 14...input device, 15...output device, 16...drive, 81...model generation program, 91...storage medium, 111: learning processing unit; 112: output unit; 2... Reasoning device, 21: control unit, 22: storage unit, 23: communication interface, 24...input device, 25...output device, 26...drive, 82...inference program, 92...storage medium, 211: position inference unit; 212: output unit; 3... Reasoning device, 31: control unit, 32: storage unit, 33: communication interface, 34...input device, 35...output device, 36...drive, 83...inference program, 93...storage medium, 311: distance inference unit; 312: output unit; 4...First model, 40...First learning result data, 5…Second model, 50…Second learning result data, 6...Inference model, 60...Inference model data

Claims

1. The computer directing the execution of machine learning of the first model and the second model; outputting the results of the machine learning; Run the first model is configured to infer, from the positions of two points in a given space, the position of an intermediate point between the two points; the second model is configured to infer a distance between the two points from the positions of the two points; The machine learning method includes: training the first model to acquire the ability to estimate the location of the waypoint using the distance estimates by the second model; and training the second model to acquire the ability to estimate the distance using the midpoint estimates of the first model and a metric of the space; Including, Model generation method.

2. The midpoint is a midpoint. The method of claim 1 .

3. Training the first model comprises: Sampling two points from the space, the first sample point and the second sample point; using the first model to estimate a first location of a midpoint between the first sample point and the second sample point; using the second model to estimate a first distance between the first sample point and the estimated midpoint; using the second model to estimate a second distance between the estimated midpoint and the second sample point; and training the first model such that the estimated first and second distances satisfy the waypoint condition; Including, The method of claim 1 .

4. Training the second model comprises: (1) sampling two points from the space, the two points being a third sample point and a fourth sample point; (2) estimating a third distance between the third sample point and the fourth sample point using the second model; (3) training the second model such that, if the third sample point and the fourth sample point satisfy a proximity condition, the estimated third distance is equal to a value given by a metric of the space; and (4) If the third sample point and the fourth sample point do not satisfy a proximity condition, (4-1) estimating a second location of a midpoint between the third sample point and the fourth sample point using the first model; (4-2) adopting the combination of the third sample point and the midpoint, and the combination of the midpoint and the fourth sample point as new combinations of the third sample point and the fourth sample point, respectively, and repeating the process from (2); and (4-3) After repeating the process from (2) according to (4-2), training the second model such that an estimate of the third distance between the third sample point and the fourth sample point sampled in (1), estimated using a model, is equal to the sum of values ​​given by a metric of the space; Including, The method of claim 1 .

5. The spatial metric includes a cost function. The method of claim 1 .

6. On the computer, inferring a location of a midpoint between a first and a second point given in space using an inference model, the inference model being configured to infer, from the locations of the two points, a location of a midpoint that lies between the two points and has a predetermined ratio of distances from the two points; generating a geodesic line between the first point and the second point, the geodesic line being composed of a plurality of the estimated intermediate points, by repeatedly performing the estimation step recursively; outputting information about the generated geodesic curve; In order to execute Inference program.

7. The geodesic curve is a path. The inference program according to claim 6.

8. The path is a motion path of a robotic device. The inference program according to claim 7.

9. The route is a travel route of a moving object. The inference program according to claim 7.

10. each point in the space is configured to indicate a state of the device; the geodesic curve represents a state transition of the device; The inference program according to claim 6.

11. The device is a robot device or a mobile object. The inference program according to claim 10.

12. Each point in the space is configured to represent an image; the geodesic curves represent transitions in the image; The inference program according to claim 6.

13. each point in the space is configured to represent a state of an element given by a simulator; The geodesic curve represents a locus of states in the element that obeys physical laws. The inference program according to claim 6.

14. A computer that stores the trained second model generated by the model generation method according to any one of claims 1 to 5, using the trained second model to estimate the distance between two given points in space; outputting information regarding the estimated distance; In order to execute Inference program.

Images