MODEL GENERATION DEVICE, ROUTE SEARCH DEVICE, MODEL GENERATION METHOD, AND MODEL GENERATION PROGRAM

Abstract

To improve the efficiency of a route search and reduce the generation cost of a search module.SOLUTION: A model generation device pertaining to one aspect of the present invention gives a training map as an input map with regard to each training data set by machine learning so as to train a search module so that the route searched by the search module conforms to a recommended route indicated by correct-answer information. During machine learning, the model generation device calculates an operation to extract and an operation to select directly as are in a forward propagation phase and replaces each of the operation to extract and the operation to select with a differentiable alternative operation in a reverse propagation phase and, by calculation of the alternative operations having been replaced with, calculates an approximate gradient that corresponds to the differential calculation of each of the operation to extract and the operation to select.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a model generation device, a route search device, a model generation method, and a model generation program. [Background technology]

[0002] There is a problem (route search problem) of searching for a recommended route from a start node on a map (graph) to a target node (goal). The recommended route is, for example, the shortest route, and the route search problem is solved for the purpose of searching, for example, a recommended route for a moving object to move, a recommended route for a robot device to operate, etc. Methods for solving this route search problem include Dijkstra's algorithm, best-first search algorithm, A * Classical algorithms such as search algorithms are widely known.

[0003] The basic idea behind the method for searching for the minimum cost route is as follows. (1) The search space is a set of N nodes (N is a natural number) V (V = {V1,...,V N}), one of which is the start node (v s ∈V), and another one is the goal node (v g ∈V). (2) Prepare a set O (O ⊂ V) of candidate nodes to be checked for whether they constitute a recommended path, and a set C (C ⊂ V) of checked nodes. In the initial state, set O contains the start node v s Only belongs to (O={v s}), the set C is empty. (3) target node v g Repeat the following steps (A) and (B) until v is included in set O. (A) Select node v from set O with the smallest cost. * (v * ∈O) and select a node v * Add to set C and add node v from set O. * Remove. (B) Node v * The nodes that are close to (e.g., adjacent to)SUCC ⊂ V) from V and add the extracted node to set O. (4) target node v g is included in set O, then the goal node v g Select to end the search.

[0004]

number

[0005] When a=1 and b=0, the above search algorithm becomes Dijkstra's algorithm. With Dijkstra's algorithm, it is possible to reliably find the optimal recommended route with the smallest actual cost. However, Dijkstra's algorithm searches exhaustively for the route with the smallest actual cost g. Therefore, the search efficiency is not good and the search takes a long time. When a=1 and b=1, the above search algorithm becomes the best-first search method. The best-first search method is often more efficient than the Dijkstra algorithm because it preferentially searches the node with the smallest cost to the goal node (i.e., it continues to move in the direction approaching the goal node). However, there is a possibility that the optimal recommended route is not obtained (i.e., the obtained route is not the optimal route) due to reasons such as the presence of an obstacle between the start node and the goal node. When a=1 and b=1, the above search algorithm becomes the best-first search method. * This is a search algorithm. * The search efficiency of the search algorithm is better than that of the Dijkstra algorithm, but worse than that of the best-first search method. However, by appropriately designing a tolerant heuristic function (cost), it is possible to reliably find the optimal recommended route.

[0006] That is, the best-first search method and A * The search algorithm can improve the efficiency of search compared to the Dijkstra algorithm. However, there are cases where obstacles that block progress are present on the map, and the best-first search method or A * It is known that even if a search algorithm is adopted, the efficiency of the search may be deteriorated depending on the shape of the obstacle.

[0007] FIG. 1 is a schematic diagram illustrating an example of a map including an obstacle that deteriorates the efficiency of a search. In the map of FIG. 1, a U-shaped obstacle exists between a start node and a goal node, and the concave part of the obstacle faces the start node. * When a search algorithm searches for a recommended route in this map, it may search inside the recess to get closer to the goal node from the starting node, in which case the search will collide with an obstacle inside the recess, and then search inside and around the recess to find a detour route, resulting in poor search efficiency.

[0008] In the case of the map in FIG. 1, as long as the inside of the recess is searched, the goal node cannot be reached. Therefore, no matter which search algorithm is adopted, the efficiency of the search will deteriorate as long as the inside of the recess is searched. In this case, if the inside of the recess can be avoided and the search (for example, the dotted arrow in FIG. 1) can be advanced, the efficiency of the search can be improved no matter which search algorithm is adopted (even when Dijkstra's algorithm is adopted). In other words, when an obstacle that impedes progress exists on the map and the obstacle forms an area that deteriorates the efficiency of the search, if the search of that area can be avoided, the efficiency of the search can be improved no matter which search algorithm is adopted.

[0009] Non-Patent Document 1 proposes adjusting the parameters of a heuristic function for calculating costs in a best-first search method by machine learning. According to the method of Non-Patent Document 1, the heuristic function can be trained so that the cost of areas that deteriorate the efficiency of the search is increased. Therefore, by using the trained heuristic function, it is possible to avoid searching such areas as much as possible, thereby improving the efficiency of the search. [Prior art documents] [Non-patent literature]

[0010] [Non-Patent Document 1] Mohak Bhardwaj, Sanjiban Choudhury, Sebastian Scherer, “Learning Heuristic Search via Imitation”, CoRL 2017, 271-280. Summary of the Invention [Problem to be solved by the invention]

[0011] The present inventors have found that the method of directly training a cost function by machine learning proposed in Non-Patent Document 1 and the like has the following problems. In other words, the shapes of obstacles that deteriorate the efficiency of search are not limited to the above-mentioned U-shape, but exist in a variety of shapes. In order to deal with each case, it is desirable to train the cost function for a variety of training maps. However, in order to directly train the cost function, correct answer information indicating the true value of the cost for all nodes included in the training map must be prepared for each training map. Identifying the true value of the cost for all nodes according to the search algorithm (i.e., creating correct answer information) takes time proportional to the number of nodes included in the training map, and as a result, the effort required to generate a route search module increases. Furthermore, the method of Non-Patent Document 1 often fails to search for a route with the minimum cost because it employs a best-first search method.

[0012] In one aspect, the present invention has been made in consideration of the above circumstances, and its object is to provide a technology that improves the efficiency of route searches and reduces the effort required to generate search modules. [Means for solving the problem]

[0013] In order to solve the above-mentioned problems, the present invention employs the following configuration.

[0014] That is, a model generation device according to one aspect of the present invention includes a data acquisition unit that acquires a plurality of learning data sets, each of which is constituted by a combination of a training map and correct answer information indicating a recommended route in the training map, and a learning processing unit that performs machine learning of a search module using the plurality of learning data sets, the search module including: a neural network configured to estimate a cost of each of a plurality of nodes included in an input map; an operation to extract one or more candidate nodes adjacent to a via node from a plurality of nodes included in the input map; and an operation to select a next via node from the extracted one or more candidate nodes so as to minimize a total cost based on an estimation result of the neural network, the search module setting a start node to the first via node, and a learning processing unit configured to search for a path from a starting node to a target node by repeating the extraction operation and the selection operation until a node is selected as a target node, and the machine learning is configured by training the search module for each of the learning data sets by providing the training map as the input map so that the path searched by the search module conforms to a recommended path indicated by the correct answer information, and during the machine learning, in a forward propagation phase, the extraction operation and the selection operation are calculated as they are, while in a back propagation phase, the extraction operation and the selection operation are each replaced with a differentiable alternative operation, and an approximate gradient corresponding to each of the differential calculations of the extraction operation and the selection operation is calculated by a differential calculation of the replaced alternative operation.

[0015] In this configuration, the search module (search model) includes a first calculation module configured to execute a calculation to estimate the cost of each node, and a second calculation module configured to execute a calculation to search for a route with the smallest total cost based on the estimated cost. The first calculation module is configured by a neural network. The second calculation module is configured by a calculation to extract one or more candidate nodes adjacent to a route node, and a calculation to select a next route node from the extracted one or more candidate nodes. If only the first calculation module is treated as a machine learning target, a true value of the cost is prepared for each training map, and as described above, the effort required to generate the search module is large. In contrast, in this configuration, both the first calculation module and the second calculation module are treated as machine learning processing targets. As a result, in this configuration, it is only necessary to prepare correct answer information indicating the recommended route itself, rather than preparing a true value of the cost for each training map. As a result, it is not necessary to search all the nodes included in the training map, and for example, the search results, track records, etc. of an existing algorithm can be adopted as the recommended route as they are. Therefore, compared to identifying the true values ​​of the costs of all the nodes included in the training map, the work time and burden required for preparing correct answer information can be reduced. According to this configuration, the effort required for generating the search module can be reduced. In addition, in the process of machine learning, the search module is trained to optimally search for a recommended route, so that the neural network can be indirectly trained from the recommended route indicated by the correct answer information so that the cost of areas that deteriorate the efficiency of the search is increased. As a result, the efficiency of the route search can be appropriately improved by using the cost estimation result by the trained neural network.

[0016] However, when performing machine learning of a search module including a neural network, the gradient of the error between the result of searching for a route by the forward propagation operation and the correct answer information is back-propagated in order from the output side. In order to perform this back-propagation operation, all operations that back-propagate the gradient of the error are required to be differentiable, but the operation of the second calculation module that performs a route search using the cost estimation result by the neural network (i.e., the operation of extracting one or more candidate nodes and the operation of selecting the next via node from the extracted one or more candidate nodes) is an operation of extracting elements that meet a condition from a set, and is not differentiable. Therefore, machine learning of the search module cannot be performed by a simple method. In other words, by making the second calculation module the processing target of the machine learning, the final result can be treated as correct answer information, and as a result, the effort required for generating the search module can be reduced as described above, but a new problem occurs in that it is difficult to perform the machine learning by the known method.

[0017] Therefore, in this configuration, during machine learning, in the forward propagation phase, the operations to be extracted and the operations to be selected are calculated as they are, whereas in the backward propagation phase, the operations to be extracted and the operations to be selected are replaced with differentiable alternative operations, and the approximate gradients corresponding to the differential calculations of the operations to be extracted and the operations to be selected are calculated by the differential calculation of the replaced alternative operations. This allows machine learning to be properly performed even for a search module that includes non-differentiable operations. Therefore, according to this configuration, the efficiency of route search can be improved, and a trained search module (a learned search model) can be properly generated, thereby reducing the effort required for its generation.

[0018] In the model generating device according to the above aspect, the selecting operation may be configured to select a next via node from one or more extracted candidate nodes based on the estimation result of the neural network so that a total sum of costs is minimized by a predetermined search algorithm. With this configuration, it is possible to reduce the effort required to generate a search module configured to perform the predetermined search algorithm. Moreover, by using an existing search algorithm as the predetermined search algorithm, it is possible to further reduce the effort required to generate the search module.

[0019] In the model generating device according to the above aspect, the predetermined search algorithm is a Dijkstra algorithm, a best-first search algorithm, or an A * The algorithm may be a search algorithm. According to this configuration, the search module can be easily configured, so that the effort required for generating the search module can be further reduced. Note that each algorithm may include a modified algorithm in addition to the original algorithm.

[0020] In the model generation device according to the above aspect, the training map may include a range in which movement of a moving object is permitted, and the recommended route indicated by the correct answer information may be a route recommended for the moving object to move. According to this configuration, in a form in which a search module usable for a movement plan of a moving object is generated, it is possible to improve efficiency of route search and reduce the effort required for generating the search module.

[0021] In the model generating device according to the above aspect, the training map is The recommended route indicated by the correct answer information may include a range of allowances, and the recommended route indicated by the correct answer information may be a route recommended for operation of the robot device. According to this configuration, in a form in which a search module usable for a robot operation plan is generated, it is possible to improve efficiency of route search and reduce the effort required for generating the search module.

[0022] The model generation device according to the above aspect can generate a trained search module that has acquired the ability to search for a route with the smallest cost evaluated by an arbitrary index. In the model generation device according to the above aspect, the recommended route may be the shortest route by adopting distance as the evaluation index. This configuration can improve the efficiency of searching for the shortest route and reduce the effort required to generate a search module that has acquired the ability to search for the shortest route.

[0023] In the model generating device according to the above aspect, the training map may be composed of an image, and the nodes may be composed of pixels. With this configuration, in a form in which a route is searched for within an image, it is possible to improve the efficiency of the route search and reduce the effort required for generating a search module.

[0024] In the model generating device according to the above aspect, the image may be a real image or a binary image. With this configuration, in a form using a real image or a binary image, it is possible to improve the efficiency of the route search and reduce the effort required for generating the search module.

[0025] Moreover, the form of the present invention may not be limited to the form of the model generating device. One aspect of the present invention may be a route search device that uses a trained search module generated by the model generating device. For example, one aspect of the present invention may be a route search device configured to include a map acquisition unit that acquires an object map, a search unit that uses the trained search module to search for a route from a start node to a goal node in the acquired object map, and an output unit that outputs information regarding the result of searching the route on the object map. This route search device may be read as, for example, a planning device, a movement planning device, a guidance device, an operation planning device, a control device, or the like depending on the type of task in an application scene.

[0026] In the path search device according to the above aspect, the object map may include a range in which movement of a moving object is permitted. In response to this, the search unit may use the trained search module to search for a recommended path for the moving object to move from the start node to the goal node in the acquired object map. Alternatively, in the path search device according to the above aspect, the object map may include a range in which driving of a robotic device is permitted. In response to this, the search unit may use the trained search module to search for a recommended path for the robotic device to operate from the start node to the goal node in the acquired object map.

[0027] In the route search device according to the above aspect, the information on the result of the route search may be configured by navigation according to the searched route. With this configuration, it is possible to efficiently search for a recommended route, and to execute travel guidance or operation control according to the search result (i.e., the searched recommended route).

[0028] As another aspect of each of the model generating device and the route searching device according to the above-mentioned embodiments, one aspect of the present invention may be an information processing method or a program for realizing all or part of each of the above-mentioned configurations, or a storage medium having such a program stored therein and readable by a computer or other device, machine, etc. Here, a storage medium readable by a computer, etc. refers to a storage medium that stores information such as a program electrically, magnetically, optically, mechanically, etc. Another aspect of the present invention may be a route search system including the model generating device and the route search device according to any one of the above aspects.

[0029] For example, a model generation method according to one aspect of the present invention includes a step of acquiring a plurality of learning data sets, each of which is composed of a combination of a training map and correct answer information indicating a recommended route in the training map, by a computer, and a step of implementing machine learning of a search module using the plurality of learning data sets, the search module including: a neural network configured to estimate a cost of each of a plurality of nodes included in an input map; an operation of extracting one or more candidate nodes adjacent to a via node from a plurality of nodes included in the input map; and an operation of selecting a next via node from the extracted one or more candidate nodes so as to minimize a total sum of costs based on an estimation result of the neural network, the search module setting a start node as a first via node, and selecting a target node as a via node from the extracted candidate nodes based on an estimation result of the neural network, the search module setting a start node as a first via node, and selecting a target node as a via node from the extracted candidate nodes based on an estimation result of the neural network, the step of selecting a next via node from the extracted candidate nodes so as to minimize a total sum of costs based on an estimation result of the neural network, the step of selecting a next via node from the extracted candidate nodes ... the machine learning is configured to search for a path from a start node to a goal node by repeating the extracting operation and the selecting operation until a path from a start node to a goal node is selected as a recommended path indicated by the ground truth information, and the machine learning is configured to train the search module for each of the learning datasets by providing the training map as the input map, so that a path searched by the search module matches a recommended path indicated by the ground truth information, and during the machine learning, in a forward propagation phase, the extracting operation and the selecting operation are calculated as they are, whereas in a back propagation phase, the extracting operation and the selecting operation are each replaced with a differentiable alternative operation, and an approximate gradient corresponding to each of the differential calculations of the extracting operation and the selecting operation is calculated by a differential calculation of the replaced alternative operation.

[0030] Also, for example, a model generation program according to one aspect of the present invention includes a step of instructing a computer to acquire a plurality of learning data sets each composed of a combination of a training map and correct answer information indicating a recommended route in the training map, and a step of implementing machine learning of a search module using the plurality of learning data sets, the search module including a neural network configured to estimate a cost of each of a plurality of nodes included in an input map, an operation of extracting one or more candidate nodes adjacent to a via node from a plurality of nodes included in the input map, and an operation of selecting a next via node from the extracted one or more candidate nodes so as to minimize a total sum of costs based on an estimation result of the neural network, the search module setting a start node as a first via node, and selecting a target node as a via node from the extracted candidate nodes based on an estimation result of the neural network, the step of selecting a next via node from the extracted candidate nodes so as to minimize a total sum of costs ... the program is configured to search for a path from a start node to a goal node by repeating the extracting operation and the selecting operation until a node is selected as a recommended path from a start node to a goal node, the machine learning being configured by training the search module for each of the learning datasets by providing the training map as the input map so that a path searched by the search module matches a recommended path indicated by the correct answer information, and during the machine learning, in a forward propagation phase, the extracting operation and the selecting operation are calculated as they are, whereas in a back propagation phase, the extracting operation and the selecting operation are each replaced with a differentiable alternative operation and an approximate gradient corresponding to each of the differential calculations of the extracting operation and the selecting operation is calculated by a differential calculation of the replaced alternative operation. Effect of the Invention

[0031] According to the present invention, it is possible to improve the efficiency of route search and reduce the cost of generating a search module. [Brief description of the drawings]

[0032] [Figure 1] FIG. 1 illustrates a schematic example of a map that includes obstacles that reduce the efficiency of a search. [Diagram 2] FIG. 2 illustrates a schematic example of a situation in which the present invention is applied. [Diagram 3] FIG. 3 illustrates an example of a hardware configuration of the model generating device according to the embodiment. [Figure 4] FIG. 4 illustrates an example of a hardware configuration of a route search device according to an embodiment. [Diagram 5] FIG. 5 illustrates an example of a software configuration of the model generating device according to the embodiment. [Figure 6] FIG. 6 illustrates an example of a software configuration of the route search device according to the embodiment. [Figure 7] FIG. 7 is a flowchart illustrating an example of a processing procedure of the model generating device according to the embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an example of a processing procedure of the route search device according to the embodiment. [Figure 9] FIG. 9 illustrates a schematic example of another scene to which the present invention is applied. [Figure 10] FIG. 10 illustrates a schematic example of another scene to which the present invention is applied. [Figure 11A] FIG. 11A shows the target map used for the evaluation. [Figure 11B] FIG. 11B shows the target map used for the evaluation. [Figure 11C] FIG. 11C shows the target map used for the evaluation. [Figure 12A] FIG. 12A shows the results of searching for a route in the object map of FIG. 11A using the A* search algorithm (comparative example) and the nodes that were the objects of the search. [Figure 12B] FIG. 12B shows the results of searching for a route in the object map of FIG. 11B using the A* search algorithm (comparative example) and the nodes that were the objects of the search. [Figure 12C] FIG. 12C shows the results of searching for a route in the object map of FIG. 11C using the A* search algorithm (comparative example) and the nodes that were the objects of the search. [Figure 13A] FIG. 13A shows the results of estimating the cost of each node in the object map of FIG. 11A using the neural network of the embodiment. [Figure 13B] FIG. 13B shows the results of estimating the cost of each node in the object map of FIG. 11B using the neural network of the embodiment. [Figure 13C] FIG. 13C shows the results of estimating the cost of each node in the object map of FIG. 11C using the neural network of the embodiment. [Figure 14A] FIG. 14A shows the results of searching for a route in the object map of FIG. 11A by the search module of the embodiment, and the nodes that are the object of the search. [Figure 14B] FIG. 14B shows the results of searching for a route in the target map of FIG. 11B by the search module of the embodiment, and the nodes that are the targets of the search. [Figure 14C] FIG. 14C shows the results of searching for a route in the object map of FIG. 11C by the search module of the embodiment, and the nodes that are the object of the search. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] An embodiment according to one aspect of the present invention (hereinafter, also referred to as "the present embodiment") will be described below with reference to the drawings. However, the present embodiment described below is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, in implementing the present invention, a specific configuration according to the embodiment may be appropriately adopted. Note that while the data appearing in this embodiment is described in natural language, more specifically, it is specified in pseudo-language, commands, parameters, machine language, etc. that can be recognized by a computer.

[0034] §1 Examples of application FIG. 2 is a schematic diagram illustrating an example of a scene to which the present invention is applied. As shown in FIG. 2, a route search system 100 according to the present embodiment includes a model generating device 1 and a route search device 2. is.

[0035] The model generation device 1 according to this embodiment is a computer configured to generate a trained search module 5 (trained search model) by machine learning. The model generation device 1 according to this embodiment acquires a plurality of learning data sets 3 each composed of a combination of a training map 31 and correct answer information 32 indicating a recommended route in the training map 31. The model generation device 1 according to this embodiment performs machine learning of the search module 5 using the acquired plurality of learning data sets 3.

[0036] The search module 5 according to this embodiment includes a first calculation module configured to execute a calculation to estimate the cost of each of a plurality of nodes included in the input map, and a second calculation module configured to execute a calculation to search for a route with the minimum total cost based on the estimated costs. The first calculation module is configured with a neural network 50. The second calculation module is configured with a calculation (hereinafter also simply referred to as "extraction calculation") 51 to extract one or more candidate nodes adjacent to a via node from a plurality of nodes included in the input map, and a calculation (hereinafter also simply referred to as "selection calculation") 52 to select a next via node from the extracted one or more candidate nodes so that the total cost from the start node to the goal node is the minimum based on the estimation result of the neural network 50.

[0037] The search module 5 is configured to search for a path from the start node to the goal node by setting the start node as the first via node and repeating the extraction operation 51 and the selection operation 52 until the goal node is selected as the via node. That is, the search module 5 is configured to search for a path with the minimum cost by performing the above operations (1)-(4) using the first calculation module and the second calculation module. The estimation result of the neural network 50 is used for the calculation of the above formula 1. The extraction operation 51 corresponds to the above operation (3)(B). Setting the start node as the first via node and the selection operation 52 correspond to the above operation (3)(A).

[0038] Machine learning is configured by training the search module 5 so that the route searched by the search module 5 conforms to the recommended route indicated by the correct answer information 32 by providing the training map 31 as an input map for each learning data set 3. Specifically, the neural network 50 has a plurality of calculation parameters, such as the weight of the connection between each neuron and the threshold value of each neuron. First, the model generation device 1 uses the search module 5 to try a route search for the training map 31 of each learning data set 3. In this trial process, the model generation device 1 uses the neural network 50 to estimate the cost of each node included in the training map 31 of each learning data set 3. That is, the model generation device 1 inputs the training map 31 to the neural network 50 and executes a forward propagation calculation process of the neural network 50. Through this calculation, the model generation device 1 can obtain an estimated result 60 of the cost of each node included in the training map 31. Next, the model generation device 1 sets the start node S1 as the first via node. Then, the model generation device 1 uses the obtained estimation result 60 to repeat the extraction operation 51 and the selection operation 52 until the target node G1 is selected as a via node. Through this series of forward propagation operations, the model generation device 1 can obtain a trial search result, i.e., a result of searching for a route on the training map 31 (hereinafter also referred to as a "search result") 62. Next, the model generation device 1 calculates the error between this search result 62 and the recommended route (correct answer) indicated by the correct answer information 32. The model generation device 1 adjusts the values ​​of each calculation parameter of the neural network 50 by backpropagating the gradient of the calculated error using the error backpropagation method. Training the search module 5 is a process of finding a route that matches the correct answer information 32 through this series of processes (i.e., the trial search process by forward propagation and the adjustment process of the calculation parameters by backpropagation). It is configured by adjusting the values ​​of the calculation parameters of the neural network 50 so as to obtain the search result 62.

[0039] During this machine learning, the model generating device 1 calculates the extraction operation 51 and the selection operation 52 as they are in the forward propagation (search trial) phase, whereas in the back propagation (calculation parameter adjustment) phase, the model generating device 1 replaces each of the extraction operation 51 and the selection operation 52 with a differentiable alternative operation, and calculates approximate gradients corresponding to the differential calculations of each of the extraction operation 51 and the selection operation 52 by differential calculation of the replaced alternative operation. That is, in the search trial phase of the machine learning process, the model generating device 1 employs the extraction operation 51 and the selection operation 52 as they are, and obtains a search result 62 for the training map 31 of each learning dataset 3. In contrast, both the extraction operation 51 and the selection operation 52 are not differentiable, and the error gradient cannot be backpropagated as they are. Therefore, in the phase of backpropagating the error gradient derived by the trial phase, the model generation device 1 adopts the approximate gradients calculated by the differential calculation of each alternative calculation as the results of the differential calculation of each of the extraction calculation 51 and the selection calculation 52, backpropagates the error gradient, and adjusts the values ​​of each calculation parameter of the neural network 50.

[0040] The content of the alternative operation of the extraction operation 51 is not particularly limited as long as it is an operation capable of calculating the approximate gradient of the extraction operation 51, and may be appropriately determined according to the embodiment. The alternative operation of the extraction operation 51 may be configured, for example, as a linear convolution function using a fixed-size rectangular filter, a graph convolution operation using a fixed-size graph, or a function similar thereto, and the approximate gradient of the extraction operation 51 may be the gradient of the function. The alternative operation of the extraction operation 51 may be configured by a lookup table corresponding to the function. Similarly, the content of the alternative operation of the selection operation 52 is not particularly limited as long as it is an operation capable of calculating the approximate gradient of the selection operation 52, and may be appropriately determined according to the embodiment. The alternative operation of the selection operation 52 may be configured, for example, as a straight-through soft-min function, straight-through soft-min with temperature, weighted straight-through soft-min, straight-through Gumbel softmin function, or a function similar thereto, and the approximate gradient of the selection operation 52 may be the gradient of the function. As a result of this machine learning, the model generation device 1 can generate a trained search module 5 that has acquired the ability to search for a recommended route that minimizes the cost for an input map.

[0041] Meanwhile, the route search device 2 according to this embodiment is a computer configured to perform a route search for a given map using a trained search module 5. First, the route search device 2 acquires a target map 221 that is the target of the route search. Next, the route search device 2 uses the trained search module 5 to search for a route from a start node S2 to a goal node G2 in the acquired target map 221.

[0042] That is, the route search device 2 inputs the object map 221 to the trained neural network 50, and executes the forward propagation arithmetic process of the trained neural network 50, in the same manner as in the trial phase of the model generation device 1. The route search device 2 can obtain an estimation result 223 of the cost of each node included in the object map 221 through this arithmetic process. Next, the route search device 2 sets the start node S2 as the first via node. Then, the route search device 2 uses the obtained estimation result 223 to repeat the extraction calculation 51 and the selection calculation 52 until the goal node G2 is selected as the via node. In the calculation of this second calculation module, the route search device 2 adopts the extraction calculation 51 and the selection calculation 52 as they are, in the same manner as in the trial phase of the model generation device 1.

[0043] Through this series of calculations, the route search device 2 can obtain a result 225 of the route search on the target map 221 (hereinafter, also referred to as the "search result"). The search device 2 outputs information related to a search result 225 of a route for the target map 221. Note that the route search device 2 according to this embodiment may be interpreted as a planning device, a movement planning device, a guidance device, an action planning device, a control device, or the like, depending on the type of task in an application scene.

[0044] In addition, as long as the maps (training map 31, target map 221) used in this embodiment include multiple nodes and are appropriately configured to be the target of route search processing, the data format of the map is not particularly limited and may be appropriately selected depending on the embodiment. In one example, the maps (training map 31, target map 221) may be represented in a list format, a graph format, etc. Obstacles (immovable areas) may be represented by the presence or absence of connections between nodes, information given to the nodes, etc.

[0045] As an example of a graph format, the map (training map 31, target map 221) may be composed of an image (image data). In this case, the nodes may be composed of pixels. The image can be treated as a graph of a two-dimensional grid composed of four or eight neighboring nodes connected to each node. The four neighboring nodes are four pixels arranged above, below, left and right of the target pixel. The eight neighboring nodes are eight pixels surrounding the periphery of the target pixel. This image may be composed of a real image or a binary image. When searching for a path of a moving object, the real image may be an image generated by photographing a range in which the movement of the moving object is permitted, such as an aerial photograph or a photograph taken by a drone. When searching for a movement path of a robot, the real image may be an image generated by photographing a range in which the robot operates, such as a work site. Each pixel of the binary image may be appropriately configured to express a movable area and an immovable area in two values. The binary image may be generated by appropriately binarizing the real image. In the model generation device 1 according to this embodiment, by using an image as a map, it is possible to generate a trained search module 5 that has acquired the ability to search for a path within the image through the above-mentioned machine learning.

[0046] The correct answer information 32 may be appropriately configured to indicate the correct answer of the recommended route in the training map 31. The data format of the correct answer information 32 may be appropriately selected according to the expression of the route search result. When the route search result is configured as an image, an example of the correct answer information 32 may be configured as an image indicating the correct answer of the recommended route (an example of the image is shown in FIG. 2).

[0047] Moreover, the cost may be appropriately expressed as a numerical value, so long as it is configured to correspond to the degree of recommending or not recommending passing through the node by an arbitrary index. In one example, the cost may be expressed to be proportional to the numerical value (i.e., the larger the numerical value, the higher the cost). In another example, the cost may be expressed to be inversely proportional to the numerical value (i.e., the smaller the numerical value, the higher the cost). With a focus on not recommending, the cost may be read as, for example, a load, a penalty, or the like. Conversely, with a focus on recommending, the cost may be read as, for example, a recommendation level, a reward, or the like. In this case, minimizing the cost may be replaced with maximizing the recommendation level, or the like. The index for evaluating the cost may be designed from any viewpoint. In this embodiment, for convenience, the cost will be described as being designed to have a higher value as the degree of not recommending is higher. When designing the cost to have a higher value as the degree of recommending is higher, the selection operation 52 may be replaced from selecting an element with a minimum value to selecting an element with a maximum value, and the alternative operation of the selection operation 52 may also be replaced appropriately. As a specific example, the alternative operation of the selection operation 52 may be, for example, a straight-through soft-max function, a straight-through soft-max with temperature, a weighted straight-through soft-max, a straight-through Gumbel softmax function, or a similar function.

[0048] The index for evaluating the degree of recommendation may be, for example, distance, time, cost, risk, congestion, attractiveness, etc. As an example, distance may be adopted as an evaluation index. In this case, the recommended path to be searched is the shortest path (path with the smallest distance) from the start node to the goal node.

[0049] As another example, time may be used as the evaluation index. The cost may be set to be higher as the time increases. In this case, the recommended route to be searched is the route that minimizes the time required to travel from the start node to the goal node.

[0050] As another example, cost may be adopted as the evaluation index. The cost may be set to be higher as the cost increases. In this case, the recommended route to be searched is the route that minimizes the cost of traveling from the start node to the goal node. The cost may be, for example, fuel cost, electricity cost, transportation fee, highway toll, etc.

[0051] As another example, the degree of danger may be adopted as an evaluation index. The cost may be set to be higher as the degree of danger increases. In this case, the recommended route to be searched is the route with the lowest risk of movement from the start node to the goal node (i.e., the safest). When searching for a route for a moving object, the degree of danger may be set according to the risk of the moving object encountering an accident such as an accident or crime. When searching for a route for a robot, the degree of danger may be set according to the risk of failure such as collision or destruction.

[0052] As another example, the congestion degree may be adopted as an evaluation index. The cost may be set to be higher as the congestion degree increases. In this case, the recommended route to be searched for is a route that avoids congested places. Examples of congested places include places with a high density of people (when searching for a route for people to move), places with traffic jams (when searching for a route for vehicles to move), and places with a high density of obstacles (when searching for a route for the operation of a robot device).

[0053] As another example, attractiveness may be adopted as an evaluation index. The lower the attractiveness, the higher the cost may be set. In this case, the recommended route to be searched is a route that passes through places with high attractiveness. The attractiveness may be given according to an artificial index such as the quality of scenery (when searching for a movement route of a moving object) or the elegance of the movement (when searching for a movement route of a robot device).

[0054] These cases are applicable, for example, to a situation where a route for a moving body (e.g., a person, a vehicle, a drone, etc.) is planned, a situation where a route for a robot to operate is planned, etc. The cost may be designed appropriately depending on the situation to which it is applied. The cost may be designed by using a combination of the above-mentioned multiple indexes. In the model generation device 1 according to this embodiment, a trained search module 5 that has acquired the ability to search for a recommended route according to the designed cost can be generated by the above-mentioned machine learning.

[0055] As described above, the model generating device 1 according to the present embodiment does not treat only the first calculation module (neural network 50) as the target of machine learning, but treats both the first calculation module and the second calculation module (the extraction calculation 51 and the selection calculation 52) as the target of machine learning. As a result, in the present embodiment, it is not necessary to prepare a true value of the cost for each training map 31, and it is sufficient to prepare the correct answer information 32 indicating the recommended route itself. Therefore, it is not necessary to search all the nodes included in the training map 31, and for example, the search results and track records of an existing algorithm can be adopted as the recommended route as they are. As a result, the work time and burden required for preparing the correct answer information 32 can be reduced compared to identifying the true values ​​of the costs of all the nodes included in the training map 31. Therefore, according to the present embodiment, it is possible to reduce the effort required for generating the trained search module 5.

[0056] In addition, during the machine learning process, the search module 5 is trained to optimally search for recommended routes. Therefore, the neural network 50 can be indirectly trained from the recommended route indicated in the correct answer information 32 so that the cost of an area that deteriorates the efficiency of the search is increased. As a result, the route search device 2 can appropriately improve the efficiency of the route search by using the cost estimation result 223 by the trained neural network 50.

[0057] Furthermore, in the model generation device 1 according to the present embodiment, in the machine learning process, the extraction operation 51 and the selection operation 52 are calculated as they are in the forward propagation phase, whereas in the backward propagation phase, the extraction operation 51 and the selection operation 52 are replaced with differentiable alternative operations. Then, the model generation device 1 calculates approximate gradients corresponding to the differential calculations of the operation to be extracted and the operation to be selected by differential calculations of the replaced alternative operations. This allows the machine learning of the search module 5 (particularly, adjustment of the calculation parameters of the neural network 50) to be appropriately performed even if the search module 5 includes non-differentiable operations (the extraction operation 51 and the selection operation 52).

[0058] In the example of Fig. 2, the model generating device 1 and the route searching device 2 are connected to each other via a network. The type of network may be appropriately selected from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, and the like. However, the method of exchanging data between the model generating device 1 and the route searching device 2 is not limited to this example, and may be appropriately selected depending on the embodiment. For example, data may be exchanged between the model generating device 1 and the route searching device 2 using a storage medium.

[0059] 2, the model generating device 1 and the route searching device 2 are each configured by separate computers. However, the configuration of the route searching system 100 according to this embodiment is not limited to such an example and may be appropriately determined according to the embodiment. For example, the model generating device 1 and the route searching device 2 may be an integrated computer. Also, for example, at least one of the model generating device 1 and the route searching device 2 may be configured by multiple computers.

[0060] §2 Configuration Example [Hardware configuration] <Model generation device> Fig. 3 illustrates a schematic example of a hardware configuration of the model generating device 1 according to this embodiment. As shown in Fig. 3, the model generating device 1 according to this embodiment is a computer to which a control unit 11, a storage unit 12, a communication interface 13, an external interface 14, an input device 15, an output device 16, and a drive 17 are electrically connected. In Fig. 3, the communication interface and the external interface are described as a "communication I / F" and an "external I / F".

[0061] The control unit 11 includes a hardware processor, such as a CPU (Central Processing Unit), The storage unit 12 includes a random access memory (RAM), a read only memory (ROM), and the like, and is configured to execute information processing based on a program and various data. The storage unit 12 is an example of a memory, and is configured, for example, by a hard disk drive, a solid state drive, or the like. In this embodiment, the storage unit 12 stores various information such as a model generation program 81, a plurality of learning datasets 3, learning result data 125, and the like.

[0062] The model generation program 81 is a program for causing the model generation device 1 to execute machine learning information processing (FIG. 7) described later for generating a trained search module 5. The model generation program 81 includes a series of instructions for the information processing. A plurality of learning data sets 3 are used to generate the trained search module 5. The learning result data 125 indicates information about the generated trained search module 5. In this embodiment, the learning result data 12 5 is generated as a result of executing the model generation program 81. Details will be described later.

[0063] The communication interface 13 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, or the like, and is an interface for performing wired or wireless communication via a network. The model generating device 1 can use the communication interface 13 to execute data communication via a network with other information processing devices. The external interface 14 is, for example, a USB (Universal Serial Bus) port, a dedicated port, or the like, and is an interface for connecting to an external device. The type and number of the external interfaces 14 may be selected arbitrarily. The model generating device 1 may be connected to a device (for example, a camera, another computer, or the like) for obtaining the training map 31 via at least one of the communication interface 13 and the external interface 14.

[0064] The input device 15 is a device for performing input, such as a mouse, a keyboard, etc. The output device 16 is a device for performing output, such as a display, a speaker, etc. An operator such as a user can operate the model generation device 1 by using the input device 15 and the output device 16.

[0065] The drive 17 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading various information such as a program stored in a storage medium 91. The storage medium 91 is a medium that accumulates information such as a program by electrical, magnetic, optical, mechanical, or chemical action so that a computer or other device, machine, or the like can read various information such as the stored program. At least one of the model generation program 81 and the multiple learning data sets 3 may be stored in the storage medium 91. The model generation device 1 may acquire at least one of the model generation program 81 and the multiple learning data sets 3 from the storage medium 91. Note that, in FIG. 3, a disk-type storage medium such as a CD or a DVD is illustrated as an example of the storage medium 91. However, the type of the storage medium 91 is not limited to the disk type, and may be other than the disk type. An example of a storage medium other than the disk type may be a semiconductor memory such as a flash memory. The type of the drive 17 may be selected arbitrarily according to the type of the storage medium 91.

[0066] In addition, with regard to the specific hardware configuration of the model generating device 1, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processor may be a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), etc. The model generating device 1 may be configured as a storage device. The storage unit 12 may be configured by a RAM and a ROM included in the control unit 11. At least one of the communication interface 13, the external interface 14, the input device 15, the output device 16, and the drive 17 may be omitted. The model generating device 1 may be configured by a plurality of computers. In this case, the hardware configurations of the computers may or may not be the same. Furthermore, the model generating device 1 may be an information processing device designed specifically for the service to be provided, as well as a general-purpose server device, a PC (Personal Computer), etc.

[0067] <Route search device> Fig. 4 illustrates a schematic example of a hardware configuration of the route search device 2 according to this embodiment. As shown in Fig. 4, the route search device 2 according to this embodiment is a computer in which a control unit 21, a storage unit 22, a communication interface 23, an external interface 24, an input device 25, an output device 26, and a drive 27 are electrically connected.

[0068] The control unit 21 to the drive 27 and the storage medium 92 of the route search device 2 may be configured similarly to the control unit 11 to the drive 17 and the storage medium 91 of the model generating device 1, respectively. The control unit 21 includes a hardware processor such as a CPU, a RAM, and a ROM, and is configured to execute various information processes based on programs and data. The storage unit 22 is configured, for example, with a hard disk drive, a solid state drive, or the like. In this embodiment, the storage unit 22 stores various information such as a route search program 82, learning result data 125, and the like.

[0069] The route search program 82 is a program for causing the route search device 2 to execute information processing ( FIG. 8 ), described below, for performing a route search task using the trained search module 5. The route search program 82 includes a series of instructions for the information processing. At least one of the route search program 82 and the learning result data 125 may be stored in the storage medium 92. In addition, the route search device 2 may acquire at least one of the route search program 82 and the learning result data 125 from the storage medium 92.

[0070] The route search device 2 may be connected to a device (eg, a camera, another computer, etc.) for obtaining the object map 221 via at least one of the communication interface 23 and the external interface 24.

[0071] Regarding the specific hardware configuration of the route search device 2, components can be omitted, replaced, and added as appropriate depending on the embodiment. For example, the control unit 21 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, FPGA, DSP, etc. The storage unit 22 may be configured with a RAM and a ROM included in the control unit 21. At least one of the communication interface 23, the external interface 24, the input device 25, the output device 26, and the drive 27 may be omitted. The route search device 2 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. In addition, the route search device 2 may be an information processing device designed exclusively for the service provided, as well as a general-purpose server device, a general-purpose PC, a mobile terminal (e.g., a smartphone), a tablet PC, a PLC (programmable logic controller), etc.

[0072] [Software configuration] <Model generation device> 5 illustrates a schematic example of a software configuration of the model generating device 1 according to the present embodiment. The control unit 11 of the model generating device 1 loads the model generating program 81 stored in the storage unit 12 in the RAM. Then, the control unit 11 interprets and executes the instructions included in the model generating program 81 loaded in the RAM by the CPU to control each component. As a result, as shown in FIG. 5, the model generating device 1 according to the present embodiment operates as a computer including a data acquiring unit 111, a learning processing unit 112, and a storage processing unit 113 as software modules. That is, in the present embodiment, each software module of the model generating device 1 is realized by the control unit 11 (CPU).

[0073] The data acquisition unit 111 is configured to acquire a plurality of learning data sets 3. Each learning data set 3 is configured by a combination of a training map 31 and correct answer information 32. The training map 31 is configured to include a plurality of nodes. Each node may correspond to an arbitrary position in a real space or a virtual space. As an example, in a case where a path along which a moving object moves is searched, each node may correspond to an arbitrary point in a spatial area that allows the movement of the moving object. As another example, in a case where a path along which a robot device operates is searched, each node may correspond to an arbitrary point in a working area of ​​the robot device. As another example, in a case where a route to use a transportation facility is searched, each node may correspond to an arbitrary geographical point such as a transportation facility hub. The training map 31 may be configured by an image, and each node may be configured by each pixel included in the image. The image may be a real image or a binary image. The correct answer information 32 is constructed to indicate a solution for a recommended path from a start node S1 to a goal node G1 in the training map 31.

[0074] The learning processing unit 112 is configured to perform machine learning of the search module 5 using the acquired multiple learning data sets 3. The search module 5 is configured to include a neural network 50, an extraction operation 51, and a selection operation 52. The neural network 50 is configured to estimate the cost of each of multiple nodes included in the input map. The extraction operation 51 is configured to extract one or more candidate nodes adjacent to the via node from the multiple nodes included in the input map. The selection operation 52 is configured to select a next via node from the extracted one or more candidate nodes so that the total sum of costs from the start node to the goal node is minimized based on the estimation result of the neural network 50. The search module 5 is configured to search for a path from the start node to the goal node by setting the start node as the first via node and repeating the extraction operation 51 and the selection operation 52 until the goal node is selected as the via node.

[0075] The machine learning is configured by training the search module 5 by providing the training map 31 as an input map for each learning data set 3, so that the route searched by the search module 5 matches the correct answer of the recommended route indicated by the correct answer information 32, using an error backpropagation method. The learning processing unit 112 is configured to calculate the extraction operation 51 and the selection operation 52 as they are in the forward propagation (search trial) phase during machine learning, whereas in the backpropagation phase, replace each of the extraction operation 51 and the selection operation 52 with a differentiable alternative operation, and calculate approximate gradients corresponding to the respective differential calculations of the extraction operation 51 and the selection operation 52 by differential calculation of the replaced alternative operation.

[0076] The storage processing unit 113 is configured to generate information about the trained search module 5 generated by machine learning as learning result data 125, and to store the generated learning result data 125 in a predetermined storage area. The learning result data 125 may be appropriately configured to include information for reproducing the trained search module 5.

[0077] (Search Module) As shown in FIG. 5, the neural network 50 included in the search module 5 according to this embodiment is a fully connected neural network, and includes an input layer 501, one or more intermediate (hidden) layers 502, and an output layer 503.

[0078] The input layer 501 is configured to receive an input of a map for which the cost of each node is to be estimated. The cost may be affected by the location of obstacles and the like in the map as well as the positions of the start node and the goal node. Therefore, the input layer 501 may be configured to further receive input of at least one of information on the start node and the goal node along with the map. The output layer 503 is configured to output an estimation result of the cost of each node included in the input map. The number of intermediate layers 502 may be appropriately determined depending on the embodiment.

[0079] Each of the layers 501-503 includes one or more neurons (nodes). The number of neurons included in each of the layers 501-503 may be appropriately determined depending on the embodiment. The number of neurons in the input layer 501 may be appropriately determined depending on the input form such as the number of dimensions of the input. The number of neurons in the output layer 503 may be appropriately determined depending on the output form such as the number of dimensions of the output. In this embodiment, each neuron included in each of the layers 501-503 is connected to all neurons in the adjacent layers.

[0080] A weight (connection weight) is set for each connection in each layer 501 to 503. A threshold is set for each neuron, and basically, it is judged whether the sum of the products of each input and each weight exceeds the threshold. The output of each neuron is determined by the threshold. The threshold may be expressed by an activation function. In this case, the sum of the products of each input and each weight is input to the activation function, and the output of each neuron is determined by executing the calculation of the activation function. The type of activation function may be selected arbitrarily. The weight of the connection between each neuron included in each layer 501 to 503 and the threshold of each neuron are examples of calculation parameters.

[0081] In the above machine learning, the learning processing unit 112 uses the training map 31 of each learning data set 3 as training data (input data), and uses the correct answer information 32 as correct answer data (teacher signal, label).

[0082] That is, in the search trial phase, first, the learning processing unit 112 inputs the training map 31 of each learning data set 3 to the input layer 501, and executes the forward propagation calculation process of the neural network 50. Through this calculation, the learning processing unit 112 acquires an estimation result 60 of the cost of each node included in the training map 31 from the output layer 503. The learning processing unit 112 sets the start node S1 as the first via node. Then, the learning processing unit 112 uses the obtained estimation result 60 to repeat the extraction calculation 51 and the selection calculation 52 until the goal node G1 is selected as the via node. In the forward propagation calculation process, the learning processing unit 112 calculates the extraction calculation 51 and the selection calculation 52 as they are. As a result of executing a series of these forward propagation calculations (search trials), the learning processing unit 112 acquires a search result 62 of a route for the training map 31.

[0083] The learning processing unit 112 calculates an error between the obtained search result 62 and the correct answer indicated by the correct answer information 32 associated with the input training map 31. Then, the learning processing unit 112 calculates the gradient of the error, and back-propagates the calculated gradient of the error by the error back-propagation method to adjust the value of each calculation parameter. In the back-propagation calculation process, the learning processing unit 112 adopts each of the approximate gradients calculated by the differential calculation of the alternative calculation as the result of the differential calculation of the extraction calculation 51 and the selection calculation 52.

[0084] The learning processing unit 112 repeats the above series of processes for adjusting the values ​​of the calculation parameters so as to reduce the sum of the calculated errors for each learning data set 3. As a result of this machine learning, a trained search module 5 that has acquired the ability to search for a recommended route with the minimum cost for an input map can be generated.

[0085] The storage processing unit 113 generates learning result data 125 for reproducing the trained search module 5 generated by the machine learning. As long as the trained search module 5 can be reproduced, the configuration of the learning result data 125 is not particularly limited and may be appropriately determined according to the embodiment. As an example, the learning result data 125 may include information indicating the values ​​of each calculation parameter obtained by adjusting the machine learning. In some cases, the learning result data 125 may include information indicating at least one of the structure of the neural network 50, the content of the extraction calculation 51, and the content of the selection calculation 52. The structure of the neural network 50 may be specified by, for example, the number of layers from the input layer to the output layer, the type of each layer, the number of neurons included in each layer, the connection relationship between neurons in adjacent layers, etc. The storage processing unit 113 stores the generated learning result data 125 in a predetermined storage area.

[0086] <Route search device> Fig. 6 illustrates an example of a software configuration of the route search device 2 according to this embodiment. The control unit 21 of the route search device 2 loads a route search program 82 stored in the storage unit 22 onto the RAM. The control unit 21 then uses the CPU to interpret and execute instructions included in the route search program 82 loaded onto the RAM to control each component. As a result, as shown in Fig. 6, the route search device 2 according to this embodiment comprises a map acquisition unit 211, a search unit 223, and a map acquisition unit 224. The model generating device 1 operates as a computer having, as software modules, a control unit 212, and an output unit 213. That is, in this embodiment, each software module of the route search device 2 is also realized by the control unit 21 (CPU) in the same way as the model generating device 1.

[0087] The map acquisition unit 211 is configured to acquire an object map 221 to be a target of a route search task. The search unit 212 has a trained search module 5 by holding the learning result data 125. The search unit 212 is configured to search for a route from a start node S2 to a goal node G2 in the acquired object map 221 using the trained search module 5. That is, the search unit 212 inputs the acquired object map 221 to an input layer 501 of a trained neural network 50, and executes a forward propagation arithmetic process of the trained neural network 50. Through this arithmetic process, the search unit 212 can acquire an estimated result 223 of the cost of each node included in the object map 221 from the output layer 503. Next, the search unit 212 uses the obtained estimation result 223 to repeat the extraction operation 51 and the selection operation 52 until the goal node G2 is selected as a route node. In the calculation of this second calculation module, the search unit 212 employs the extraction calculation 51 and the selection calculation 52 as they are, similar to the forward propagation phase (search trial phase) of the above machine learning. As a result of executing this series of calculation processes, the search unit 212 is configured to obtain a search result 225 of a route for the target map 221. The output unit 213 is configured to output information related to the obtained search result 225.

[0088] <Other> Each software module of the model generating device 1 and the route searching device 2 will be described in detail in an operation example described later. In this embodiment, an example is described in which each software module of the model generating device 1 and the route searching device 2 is realized by a general-purpose CPU. However, some or all of the above software modules may be realized by one or more dedicated processors (e.g., graphics processing units). Each of the above modules may be realized as a hardware module. Furthermore, with regard to the software configurations of the model generating device 1 and the route searching device 2, software modules may be omitted, replaced, or added as appropriate depending on the embodiment.

[0089] §3 Example of operation [Model generation device] 7 is a flowchart showing an example of a processing procedure related to machine learning by the model generation device 1 according to the present embodiment. The processing procedure of the model generation device 1 described below is an example of a model generation method. However, the processing procedure of the model generation device 1 described below is merely an example, and each step may be changed as much as possible. In addition, steps may be omitted, replaced, or added to the following processing procedure as appropriate depending on the embodiment.

[0090] (Step S101) In step S101, the control unit 11 operates as the data acquisition unit 111 and acquires a plurality of learning data sets 3.

[0091] Each learning data set 3 may be generated as appropriate. As an example, first, a training map 31 including a plurality of nodes constituting a search space is generated. The search space may be set according to the purpose of a task, such as a movement plan for a moving body, an operation plan for a robot device, or a transfer plan for transportation. The search space may also be set in a real space or a virtual space. The training map 31 may be generated manually as appropriate. A plurality of different training maps 31 may be generated by changing at least a part of the search space. When a real image is used for the map, the training map 31 may be generated by photographing a spatial region that can be the target of a route search with a camera. When a binary image is used for the map, each pixel of the real image obtained by the camera may be arbitrarily converted into a binary image. The training map 31 may be generated by binarizing the images using the method. The binary images may be generated manually. The training map 31 may be composed of multiple images.

[0092] Next, for the generated training map 31, correct answer information 32 indicating a correct answer of a recommended route from a start node to a goal node is generated. The recommended route may be obtained as appropriate. As an example, the recommended route may be given manually by an operator. As another example, the recommended route may be obtained using, for example, the Dijkstra algorithm, the best-first search method, the A *The recommended route may be obtained by executing an existing search algorithm such as a search algorithm on the training map 31. As another example, when a search space is set in the real space, the performance obtained by actually moving from a start node to a target node in the search space or by planning the movement may be used as the recommended route. It is desirable to treat the route with the smallest total cost as the correct solution of the recommended route, but the correct solution indicated by at least a part of the correct solution information 32 does not necessarily have to be the route with the smallest total cost. The cost may be evaluated according to, for example, distance, time, cost, risk level, congestion level, attractiveness, etc. As an example, by adopting distance as an evaluation index of the cost, the correct solution information 32 may be configured to indicate the shortest distance as the correct solution of the recommended route. Note that, as long as the recommended route can be expressed, the data format of the correct solution information 32 is not particularly limited and may be appropriately selected according to the embodiment. As an example, when the training map 31 is configured by an image, the correct solution information 32 may also be configured by an image, and the recommended route may be indicated on the image.

[0093] The generated supervised answer information 32 is then associated with the training map 31. This allows each learning data set 3 to be generated. Note that for the same training map 31, multiple pairs of start nodes and target nodes may be generated such that at least one of the start node and the target node is different. Supervised answer information 32 may be generated for each generated pair, and the generated supervised answer information 32 for each pair may be associated with the training map 31. This allows multiple different learning data sets 3 to be generated from the same training map 31.

[0094] Each learning data set 3 may be generated automatically by the operation of a computer, or may be generated manually by at least partially including an operation of an operator. Moreover, each learning data set 3 may be generated by the model generating device 1, or may be generated by a computer other than the model generating device 1. When each learning data set 3 is generated by the model generating device 1, the control unit 11 may acquire each learning data set 3 by executing the above-mentioned generation process automatically or manually by an operation of an operator via the input device 15. On the other hand, when each learning data set 3 is generated by another computer, the control unit 11 may acquire each learning data set 3 generated by the other computer, for example, via a network, a storage medium 91, or the like. Some of the multiple learning data sets 3 may be generated by the model generating device 1, and the rest may be generated by one or more other computers.

[0095] The number of acquired learning data sets 3 is not particularly limited, and may be appropriately determined according to the embodiment so that machine learning can be performed. After acquiring a plurality of learning data sets 3, the control unit 11 proceeds to the next step S102.

[0096] (Step S102) In step S102, the control unit 11 operates as the learning processing unit 112 and performs machine learning of the search module 5 using the acquired multiple learning datasets 3.

[0097] As an example of machine learning processing, first, the search module 5 to be processed by machine learning is initialized. In the initial setting of the neural network 50 of the search module 5, the structure of the neural network 50 (e.g., the number of layers, the type of each layer, the number of neurons in each layer, the connection relationship between neurons in adjacent layers, etc.) and the initial values ​​of each calculation parameter are set. The structure of the neural network 50 and the initial values ​​of each calculation parameter may be provided by a template or by an operator's input. When re-learning is performed, the neural network 50 may be initially set based on the learning result data obtained by past machine learning.

[0098] Moreover, the extraction operation 51 and the selection operation 52 may be constructed appropriately. The candidate node extracted by the extraction operation 51 (i.e., a node adjacent to the via node) is a node that is a candidate for passing through the via node next, and is typically a node adjacent to the via node. The candidate node extracted by the extraction operation 51 may include, in addition to a node physically adjacent to the via node, for example, a node that can be moved (transitioned) from the via node within a certain cost. As described above in (3)(B), the extraction operation 51 selects a via node (v * ) is specified, the specified transit node (v * ) to one or more candidate nodes (set V SUCC ⊂ V) from the set of nodes contained in the input map (V) and add the extracted candidate nodes to a set O.

[0099] On the other hand, as described in (3)(A) above, the selection operation 52 may be appropriately configured to select the next via node with the smallest cost from the set O based on the cost estimation result by the neural network 50, and add the selected node to the set C and remove it from the set O. The cost function f(v) (Equation 1) for calculating the cost may be set arbitrarily. In other words, the search algorithm adopted by the search module 5 is not particularly limited as long as it searches for a route by extracting and selecting nodes, and may be appropriately selected depending on the embodiment.

[0100] As an example, the selection operation 52 may be configured to select a next transit node from one or more extracted candidate nodes so that the total sum of costs is minimized by a predetermined search algorithm. The predetermined search algorithm may be, for example, a Dijkstra algorithm, a best-first search algorithm, or an A *The selection operation 52 may be a search algorithm. In other words, the selection operation 52 may be constructed using an existing search algorithm. This can reduce the effort required to construct the search module 5. Note that each algorithm may include a modified algorithm (an algorithm derived from the original algorithm) in addition to the original algorithm. For example, the predetermined search algorithm may be a modified Dijkstra algorithm, a modified best-first search method, and a modified A * A search algorithm may be included. For example, * The search algorithm is the original A * In addition to the search algorithm, weighted A* (a=1, b>1 in the above formula 1), bidirectional A* Variation A such as (searching for a path from both the start node and the goal node) * It may include a search algorithm.

[0101] Accordingly, the output of the neural network 50 (i.e., the estimation result 60) may be configured to derive the value of the cost function f(v) in the above formula 1 for each node included in the input map. As an example, the evaluation function f(v) of each node (the above formula 1) may be replaced by the following formula 2.

[0102]

number

[0103] As an example of the search module 5, when a map is formed by a two-dimensional binary image and cost values ​​are given in the range of [0,1], the neural network 50 may be configured to output the estimated cost of each node (each pixel) as a two-dimensional image (each pixel has a value of [0,1]) by using a sigmoid function in the output layer, for example. The map (binary image) may be configured so that the value of an obstacle node is 0 and the value of a movable node is 1. The starting node v s is the starting node v sThe goal node v may be represented by a binary image in which the value of v is 1 and the other values ​​are 0. g is the target node v g may be represented by a binary image in which the value of each node in the set is 1 and the other values ​​are 0. The above sets C and O may also be represented by binary images in which the value of a node belonging to the set is 1 and the other values ​​are 0. In this case, the selection operation 52 may be given by the following Equation 3.

[0104]

number

[0105]

number

number

[0106]

number

number

[0107] The forward propagation operation of the second calculation module, which is composed of the extraction operation 51 and the selection operation 52, can be expressed as the following processes (i) to (iii). (i) Execute the selection operation 52 (equation 3). (ii) Update set C using Equation 4. Add node v to set C. * is the goal node v g If so, end the route search. (iii) Execute the extraction operation 51 (equation 6) and update the set O according to equations 5 and 7, and return to (i). The forward propagation operation of the second calculation module is executed after the forward propagation operation of the neural network 50 (first calculation module) is completed. When a binary image is used for the map, the neural network 50, the extraction operation 51, and the selection operation 52 may be configured as described above by default.

[0108] Next, the control unit 11 trains the search module 5 by providing the training map 31 as an input map for each learning data set 3 so that the route searched by the search module 5 matches the correct answer of the recommended route indicated by the correct answer information 32. For this training process, a stochastic gradient descent method, a mini-batch gradient descent method, or the like may be used.

[0109] As an example of the training process, first, the control unit 11 attempts a route search for the training map 31 of each learning data set 3 using the search module 5. That is, the control unit 11 inputs the training map 31 of each learning data set 3 to the input layer 501 of the neural network 50. When the neural network 50 is configured to further receive input of at least one of information on a start node and a goal node, the control unit 11 further inputs at least one of information on a start node S1 and a goal node G1 of the training map 31 together with the training map 31 to the input layer 501. Then, the control unit 11 executes a forward propagation calculation process of the neural network 50 (first calculation module). As a result of this calculation process, the control unit 11 obtains an estimation result 60 of the cost of each node included in the training map 31 from the output layer 503. Next, the control unit 11 executes a forward propagation calculation of the second calculation module using the estimation result 60 of the cost by the neural network 50. Specifically, the control unit 11 prepares a set O and a set C as described in (1) and (2) above, and empties (i.e., initializes) each of them. The control unit 11 adds the start node S1 to the set O, thereby setting the start node S1 as the first via node. Then, the control unit 11 uses the obtained estimation result 60 as described above, and repeats the extraction operation 51 and the selection operation 52 until the goal node G1 is selected as the via node. For example, when the search module 5 is configured as in the example using the binary image described above, the control unit 11 executes the above processes (i)-(iii). As a result of executing a series of these forward propagation operations (search trials), the control unit 11 can obtain a search result 62 (i.e., set C) of a route for the training map 31. In this forward propagation calculation process, the control unit 11 directly calculates the extraction operation 51 and the selection operation 52.

[0110] Next, the control unit 11 calculates an error between the obtained search result 62 (set C) and the correct answer of the recommended route indicated by the correct answer information 32 associated with the input training map 31. A loss function may be used to calculate the error (loss). The type of loss function used to calculate the error may be appropriately selected according to the embodiment. As an example, when the search result 62 and the correct answer information 32 are both composed of images, the control unit 11 may calculate the error (difference) between the two images using an index for evaluating the identity of the images, such as Hamming distance, L1 distance, L2 distance, etc., as a loss function. That is, the loss function may be configured to be capable of evaluating, as an error, the fact that a node on the recommended route is not included in the search result 62 and the fact that a node other than the recommended route is included in the search result 62. As a specific example, when the search module 5 is configured as in the example using the binary image, and the correct answer information 32 and the search result 62 are expressed as binary images, the error L may be given by the following formula 8.

[0111]

number

[0112] Next, the control unit 11 calculates the gradient of the calculated error. Then, the control unit 11 uses the calculated gradient of the error to calculate the error of each calculation parameter value from the output side in order by the error backpropagation method. When the search module 5 is configured as the example using the binary image, the set C can be expressed as the following formula 9. Accordingly, the calculation of backpropagating the gradient of the error can be given by the following formula 10.

[0113]

number

number

[0114] In the calculation process for backpropagating the error, the control unit 11 replaces the extraction calculation 51 and the selection calculation 52 with differentiable alternative calculations. The control unit 11 calculates approximate gradients corresponding to the differential calculations of the extraction calculation 51 and the selection calculation 52 by differential calculations of the replaced alternative calculations. In one example, the alternative calculation for the extraction calculation 51 may be a convolution calculation with a filter N as a kernel. In addition, the alternative calculation V for the selection calculation 52 * may be calculated using Equation 11 (an example using straight-through soft-min with temperature).

[0115]

number

number

[0116] The alternative operation of Equation 11 is argmax n (F(-f(n))), δv in Eq. * t (n) / δf(n) can be expressed as δF / δf(n). * can be thought of as an array in which the selected node has a value of 1 and the other nodes have a value of 0. * For the n-th element of f, the gradient with respect to the m-th element can be calculated as shown in the above formula 12. Also, if the term φ(m) in formula 2 is regarded as the term g in formula 1, δf(n) / δφ(n) can be expressed as the product of δf(n) / δg(n) and δg(n) / δφ(n). Among these, when tracing the path of P, the extraction calculation 51 is replaced with the above alternative calculation, so that the approximate gradient (i.e., the term δg(n) / δφ(n)) can be calculated. Therefore, by these replacements, the control unit 11 can calculate the δv * t It is possible to calculate approximate gradients of the terms Δf(n) / Δf(n) and Δf(n) / Δφ(n). This makes it possible to calculate the gradient (ΔL / Δθ). Then, the control unit 11 can calculate the error in the value of each calculation parameter of each of the layers 501-503 using the calculated gradient (ΔL / Δθ).

[0117] The control unit 11 updates the value of each calculation parameter of each layer 501 to 503 based on each calculated error. The degree to which each calculation parameter value is updated may be adjusted by a learning rate. The learning rate may be specified by an operator or may be set as a setting value in the program.

[0118] The control unit 11 adjusts the values ​​of the calculation parameters of the layers 501 to 503 through the series of update processes described above so as to reduce the sum of the calculated errors (so as to minimize the error L in the above example) for each learning dataset 3. For example, the control unit 11 may repeat the adjustment of the values ​​of the calculation parameters through the series of update processes described above until a predetermined condition is satisfied, such as executing the process a specified number of times or the sum of the calculated errors becoming equal to or less than a threshold value.

[0119] As a result of this machine learning process (for example, a result of optimizing the values ​​of each parameter so as to minimize the error L), the control unit 11 can generate a trained search module 5 that has acquired the ability to perform a desired route search task according to the used learning data set 3. In other words, the machine learning can cause the search module 5 to acquire the ability to search for a recommended route, and can cause the neural network 50 to acquire the ability to estimate the cost of each node so as to reduce redundant searches. When the machine learning process is completed, the control unit 11 proceeds to the next step S103.

[0120] (Step S103) In step S103, the control unit 11 operates as the storage processing unit 113 and generates information about the trained search module 5 generated by machine learning as learning result data 125. Then, the control unit 11 stores the generated learning result data 125 in a predetermined storage area.

[0121] The predetermined storage area may be, for example, a RAM in the control unit 11, the storage unit 12, an external storage device, a storage medium, or a combination of these. The storage medium may be, for example, a CD, a DVD, or the like, and the control unit 11 may store the learning result data 125 in the storage medium via the drive 17. The external storage device may be, for example, a data server such as a NAS (Network Attached Storage). In this case, the control unit 11 may use the communication interface 13 to store the learning result data 125 in the data server via a network. The external storage device may also be, for example, an external storage device connected to the model generation device 1 via the external interface 14.

[0122] When the storage of the learning result data 125 is completed, the control unit 11 ends the processing procedure of the model generating device 1 according to this operation example.

[0123] The generated learning result data 125 may be provided to the route search device 2 at any timing. For example, the control unit 11 may transfer the learning result data 125 to the route search device 2 as part of the process of step S103 or separately from the process of step S103. The route search device 2 may obtain the learning result data 125 by receiving this transfer. Also, for example, the route search device 2 may use the communication interface 23 to communicate with the model generation device 1 or The learning result data 125 may be acquired by accessing a data server via a network. Alternatively, for example, the route search device 2 may acquire the learning result data 125 via the storage medium 92. Alternatively, for example, the learning result data 125 may be incorporated in the route search device 2 in advance.

[0124] Furthermore, the control unit 11 may update or newly generate the learning result data 125 by periodically or irregularly repeating the processes of steps S101 to S103. During this repetition, at least a part of the learning dataset 3 used for machine learning may be changed, modified, added, deleted, or the like as appropriate. Then, the control unit 11 may update the learning result data 125 held by the route search device 2 by providing the updated or newly generated learning result data 125 to the route search device 2 by any method.

[0125] [Route search device] 8 is a flowchart showing an example of a processing procedure for performing a route search task by the route search device 2 according to this embodiment. The processing procedure of the route search device 2 described below is an example of a route search method. However, the processing procedure of the route search device 2 described below is merely an example, and each step may be changed as much as possible. Furthermore, steps may be omitted, replaced, or added to the following processing procedure as appropriate depending on the embodiment.

[0126] (Step S201) In step S201, the control unit 21 operates as the map acquisition unit 211 and acquires the target map 221.

[0127] The object map 221 is the same type of data as the training map 31. In some cases, the object map 221 may be the same as the training map 31. Any device may be used to acquire the object map 221. As an example, when a map is configured by an image, a camera may be used to acquire the object map 221. The control unit 21 may acquire the object map 221 directly from this device via the external interface 24 or the like. Alternatively, the control unit 21 may acquire the object map 221 indirectly from the device via another computer.

[0128] Along with the object map 221, the control unit 21 appropriately acquires information on the start node S2 and the goal node G2. The information on each node (S2, G2) may be included in the object map 221. Alternatively, the information on each node (S2, G2) may be acquired from a user's designation, the result of any information processing (e.g., the measurement result of the current location), or the like. After acquiring the object map 221 and the information on each node (S2, G2), the control unit 21 advances the process to the next step S202.

[0129] (Step S202) In step S202, the control unit 21 operates as the search unit 212, and sets the trained search module 5 with reference to the learning result data 125. Then, the control unit 21 uses the trained search module 5 to search for a recommended route with the minimum cost from the start node S2 to the goal node G2 in the acquired target map 221.

[0130] The computation process of this search may be the same as the computation process of the search trial phase in the training process of the above machine learning. That is, the control unit 21 inputs the acquired object map 221 to the input layer 501 of the trained neural network 50. If the neural network 50 is configured to further receive input of at least one of information on the start node and the goal node, the control unit 21 further inputs at least one of information on the start node S2 and the goal node G2 to the input layer 501. Then, the control unit 21 executes the computation process of forward propagation of the trained neural network 50. As a result of this computation process, the control unit 21 obtains from the output layer 503 the cost estimation results 223 of each node included in the object map 221. Next, the control unit 21 prepares a set O and a set C as described above in (1) and (2) and empties each of them. The control unit 21 adds the start node S2 to the set O, thereby setting the start node S2 as the first passed node. Then, the control unit 21 uses the obtained estimation results 223 to repeat the extraction operation 51 and the selection operation 52 until the goal node G2 is selected as the passed node.

[0131] As a result of executing a series of these calculation processes, the control unit 21 can obtain a search result 225 of a recommended route with the minimum cost for the target map 221. In one example, the control unit 21 can obtain a recommended route with the minimum cost by referring to the set C and tracing back from the goal node G2 to the parent nodes (previous via nodes) in order to the start node S2. When the control unit 21 obtains the search result 225, the control unit 21 advances the process to the next step S203.

[0132] (Step S203) In step S203, the control unit 21 operates as the output unit 213 and outputs information related to the search result 225 of the recommended route.

[0133] The output destination and the contents of the information to be output may be appropriately determined according to the embodiment. For example, the control unit 21 may output the search result 225 obtained in step S202 directly to the output device 26. The output format may be appropriately selected from, for example, images, sounds, and the like. Also, for example, the control unit 21 may execute some information processing based on the obtained search result 225. Then, the control unit 21 may output the result of the information processing as information related to the search result 225. The output of the result of the information processing may include controlling the operation of the control target device according to the search result 225. The output destination may be, for example, the output device 26, an output device of another computer, the control target device, and the like.

[0134] As an example, the information on the route search result 225 may be configured by navigation according to the searched route. The navigation may be, for example, prompting the user to move along the searched route by images, audio, etc. Alternatively, the navigation may be controlling the operation of the controlled device so as to drive along the searched route. In this way, travel guidance or operation control can be performed according to the obtained search result 225.

[0135] When the output of information related to the search result 225 is completed, the control unit 21 ends the processing procedure of the route search device 2 according to this operation example. The route search device 2 may be configured to execute a series of information processes from step S201 to step S203 at any timing (for example, when an instruction from a user is received, when the operation of a control target device is controlled, etc.). Furthermore, the route search device 2 may be configured to repeatedly perform the above route search task by repeatedly executing a series of information processes from step S201 to step S203.

[0136] [Features] As described above, in step S102, the model generation device 1 according to this embodiment does not treat only the first calculation module (neural network 50) as the target of machine learning, but treats both the first calculation module and the second calculation module (the extraction calculation 51 and the selection calculation 52) as the target of machine learning. This makes it possible to reduce the work time and burden required to prepare the correct answer information 32 of each learning data set 3 acquired in step S101. Therefore, according to this embodiment, it is possible to reduce the effort required to generate a trained search module 5.

[0137] In addition, during the machine learning process in step S102, the search module 5 optimally searches for a recommended route. As the neural network 50 is trained to search for areas that are difficult to search, the neural network 50 is indirectly trained from the recommended route indicated in the correct answer information 32 so as to estimate high costs in areas that deteriorate the efficiency of the search. The route search device 2 according to this embodiment can efficiently perform the route search task by using the cost estimation result 223 by the trained neural network 50 in the process of step S202. * When a search algorithm is employed, it is possible to improve the efficiency of obtaining a route with the minimum cost. Also, when a best-first search method is employed as the search algorithm of the search module 5, it is possible to increase the probability of obtaining a route with the minimum cost by using the results of the cost estimation of each node by the trained neural network 50.

[0138] Furthermore, in the model generation device 1 according to this embodiment, in the backpropagation phase of machine learning in step S102, the extraction operation 51 and the selection operation 52 included in the search module 5 are replaced with alternative operations, and the approximate gradients calculated by differential calculation of the alternative operations are adopted as the gradients of the extraction operation 51 and the selection operation 52. This replacement makes it possible to appropriately perform machine learning of the search module 5 including the non-differentiable second calculation module. As a result, it is possible to appropriately generate a trained search module 5 capable of efficiently performing the above-mentioned route search task by machine learning.

[0139] § 4 Variations Although the embodiment of the present invention has been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements or modifications can be made without departing from the scope of the present invention. For example, the following modifications are possible. In the following, the same reference numerals are used for the same components as in the above embodiment, and the description of the same points as in the above embodiment is omitted as appropriate. The following modifications can be combined as appropriate.

[0140] <4.1> The route search system 100 according to the embodiment may be applied to any situation where a route is searched for on a given map. As a specific example, the route search system 100 according to the embodiment may be applied to a situation where a route recommended for a moving object is searched for, a situation where a flow line recommended for a robot device is searched for, and the like. A specific example of a limited application situation will be described below.

[0141] (A) A scene where a recommended route for a moving object is being searched for. 9 illustrates an example of an application scene of a guidance system 100A according to a first specific example. The first specific example is an example in which the above-described embodiment is applied to a scene in which a route recommended for a moving object is searched for. The guidance system 100A according to the first specific example includes a model generating device 1 and a guidance device 2A. The guidance device 2A is an example of the route search device 2 described above.

[0142] The map (training map, target map) in the first specific example is configured to include a range in which the movement of the moving object is permitted. Each node included in the map may correspond to any point in a spatial area in which the movement of the moving object is permitted (for example, a road, a square, a space inside a building, etc.). This map may be configured by an image. In this case, the map may be obtained appropriately by a camera. The type of camera may be selected arbitrarily. For example, a general RGB camera, an infrared camera, a depth camera, etc. may be used as the camera. In addition, a distance sensor such as a Lidar (Light Detection And Ranging) sensor may be used as the camera. As an example, an actual image obtained by photographing a spatial area in which the moving object can move, such as an aerial photograph or a drone photograph, may be used as the map. Alternatively, an image expressing the spatial area, such as a map or a floor plan, may be used as the map. Alternatively, a binary image expressing the movable area and the non-movable area of ​​the spatial area in two values ​​may be used as the map. The moving object may be, for example, a machine configured to move, such as a vehicle or a drone. Alternatively, the moving object may be, 9, the moving object is a vehicle. Except for these points, the configuration of the first specific example may be the same as that of the above embodiment.

[0143] (Model Generator) In the first specific example, the model generation device 1 can generate a trained search module 5 that has acquired the ability to search for recommended routes for a moving object to travel, by a processing procedure similar to that of the above embodiment.

[0144] That is, in step S101, the control unit 11 acquires a plurality of learning data sets each composed of a combination of a training map and correct answer information. The training map is configured to include a range that allows the moving body to move. The correct answer information is configured to indicate a route that is recommended for the moving body to move as a recommended route. In step S102, the control unit 11 performs machine learning of the search module 5 using the acquired plurality of learning data sets. As in the above embodiment, in the backpropagation phase of the machine learning, the control unit 11 adopts each approximate gradient calculated by the differential calculation of the alternative calculation as the result of the differential calculation of the extraction calculation 51 and the selection calculation 52. This machine learning can generate a trained search module 5 that has acquired the ability to search for a recommended route for the moving body. In step S103, the control unit 11 saves learning result data indicating the generated trained search module 5 in a predetermined storage area. The learning result data may be provided to the guidance device 2A at any timing.

[0145] (Guidance device) The hardware configuration and the software configuration of the guidance device 2A may be similar to those of the route search device 2 according to the above embodiment. In the first specific example, the guidance device 2A can search for a route recommended for a moving object by using a processing procedure similar to that of the above route search device 2.

[0146] That is, in step S201, the control unit of the guidance device 2A operates as a map acquisition unit and acquires the target map 221A including an area in which the movement of the moving object is permitted (i.e., the area is a movement target of the moving object). The method of acquiring the target map 221A is not particularly limited and may be appropriately selected according to the embodiment. As an example, in order to make a movement plan for a moving object moving on the ground, an image obtained by photographing the area in which the moving object exists and the surrounding area by a drone may be used as the target map 221A.

[0147] The control unit also acquires information on the start node SA2 and the goal node GA2 as appropriate. The start node SA2 may correspond to the current location of the mobile object or a desired starting point. The goal node GA2 may correspond to the destination of the mobile object. As an example, information on each node (SA2, GA2) may be obtained by user specification. As another example, if the guidance device 2A is equipped with a measuring device such as a GPS (Global Positioning System), information on the start node SA2 may be obtained by measuring The information on the target node GA2 may be obtained from the results of measuring the current location by a measuring device. When the target map 221A is an image and a moving object moves toward an object (e.g., a specific building, spot, etc.) shown on the target map 221A, the information on the target node GA2 may be obtained from the results of image recognition on the target map 221A. Any method of image recognition may be used. After the start node SA2 and the target node GA2 are identified, the target map 221A may be appropriately acquired so as to include each of them.

[0148] In step S202, the control unit operates as a search unit and uses the trained search module 5 to search for a recommended route for the moving object to move from the start node SA2 to the goal node GA2 in the acquired object map 221A. Specifically, the control unit uses the trained neural network 50 to estimate the cost of each node included in the object map 221A. As a result of the calculation process of the trained neural network 50, the control unit , and obtains cost estimation results 223A for each node. The control unit prepares sets O and C and empties each of them. The control unit adds start node SA2 to set O, thereby setting start node SA2 as the first via node. Then, using the obtained estimation results 223A, the control unit repeats extraction operation 51 and selection operation 52 until goal node GA2 is selected as the via node. In this way, the control unit can obtain search results 225A for a recommended route with the minimum cost from start node SA2 to goal node GA2.

[0149] In step S203, the control unit operates as an output unit and outputs information related to the search result 225A of the recommended route. As an example, the control unit may output the obtained recommended route as it is. As another example, the control unit may navigate the movement along the searched recommended route based on the obtained search result 225A. The navigation may be performed, for example, by images, sounds, and the like. When the moving body is, for example, a machine such as a vehicle or a drone, the navigation may be outputting information such as a message to the user so as to encourage the movement along the recommended route by the user's manual operation. Alternatively, when the guidance device 2A is configured to be able to control the operation of the machine, the navigation may be controlling the operation of the machine so as to move along the searched recommended route. When the moving body is a person, the navigation may be outputting information such as a message to the person so as to encourage the movement along the recommended route.

[0150] (Features) According to the model generation device 1 of the first specific example, it is possible to reduce the effort required to generate a trained search module 5 that has acquired the ability to search for a route recommended to a moving object. Furthermore, the guidance device 2A can efficiently search for a route recommended to a moving object by using the trained search module 5. Furthermore, in the model generation device 1, machine learning of the search module 5 can be appropriately performed by replacing the extraction operation 51 and the selection operation 52 in the backpropagation phase of machine learning. As a result, it is possible to appropriately generate a trained search module 5 that can efficiently perform the task of searching for a route recommended to a moving object.

[0151] (B) Searching for a recommended path for the robot device 10 illustrates an example of an application scene of a control system 100B according to a second specific example. The second specific example is an example in which the above-described embodiment is applied to a scene in which a route recommended for a robot device RB is searched for. The control system 100B according to the second specific example includes a model generating device 1 and a control device 2B. The control device 2B is an example of the route search device 2 described above.

[0152] The map (training map, target map) in the second specific example is configured to include a range in which the robot device RB is allowed to be driven. Each node included in the map may correspond to any point in the working area of ​​the robot device RB. This map may be configured by an image. In this case, the map may be obtained as appropriate by a camera. The type of camera may be selected arbitrarily. As an example, an actual image obtained by photographing a spatial area in which the robot device RB can be driven may be used as the map. Alternatively, a binary image that expresses movable areas and immovable areas of the spatial area in binary may be used as the map. The map may be obtained by modeling the working environment of the robot device RB. The robot device RB may be, for example, an industrial robot (for example, a robot arm, etc.), an autonomous robot, etc. Except for these points, the configuration of the second specific example may be the same as that of the above embodiment.

[0153] (Model Generator) In the second specific example, the model generation device 1 can generate a trained search module 5 that has acquired the ability to search for a recommended path for the robot device RB to operate, using a processing procedure similar to that of the above embodiment.

[0154] That is, in step S101, the control unit 11 acquires a plurality of learning data sets each composed of a combination of a training map and correct answer information. The training map is configured to include a range that allows the robot device RB to be driven. The correct answer information is configured to indicate a route that is recommended for the robot device RB to operate as a recommended route. In step S102, the control unit 11 performs machine learning of the search module 5 using the acquired plurality of learning data sets. As in the above embodiment, in the backpropagation phase of the machine learning, the control unit 11 adopts each approximate gradient calculated by the differential calculation of the alternative calculation as the result of the differential calculation of the extraction calculation 51 and the selection calculation 52. This machine learning can generate a trained search module 5 that has acquired the ability to search for a recommended route for the robot device RB. In step S103, the control unit 11 saves learning result data indicating the generated trained search module 5 in a predetermined storage area. The learning result data may be provided to the control device 2B at any timing.

[0155] (Control device) The hardware configuration and software configuration of the control device 2B may be similar to those of the path search device 2 according to the above embodiment. The robot device RB to be driven may be connected to the control device 2B via a communication interface or an external interface. In the second specific example, the control device 2B can search for a recommended path for the operation of the robot device RB by the same processing procedure as that of the path search device 2.

[0156] That is, in step S201, the control unit of the control device 2B operates as a map acquisition unit and acquires an object map 221B including a range in which the robot device RB is permitted to be driven. The method of acquiring the object map 221B is not particularly limited and may be appropriately selected depending on the embodiment. As an example, an image obtained by photographing the working range of the robot device RB with a camera provided on or around the robot device RB may be used as the object map 221B. As another example, environmental information generated by modeling the surrounding environment of the robot device RB may be used as the object map 221B.

[0157] The control unit appropriately acquires information on the start node SB2 and the target node GB2. The start node SB2 may correspond to the current position of the robot device RB or a desired start position. If the robot device RB is equipped with an end effector, the start node SB2 may correspond to the current position of the end effector or a desired start position. The target node GB2 may correspond to the target position of the robot device RB. The target position of the robot device RB may be, for example, a position where a target object such as a workpiece is placed. As an example, information on each node (SB2, GB2) may be obtained by specification by an operator. As another example, information on the start node SB2 may be obtained from information on the own device (for example, the current position of the end effector). Information on the target node GB2 may be obtained from setting information on the target position. As yet another example, when the target map 221B is an image and the robot device RB and the target object are shown in the image, information on each node (SB2, GB2) may be obtained from the result of image recognition on the target map 221B. Any method of image recognition may be used. Object map 221B may be optionally obtained to include start node SB2 and goal node GB2 after they have been identified.

[0158] In step S202, the control unit operates as a search unit, and uses the trained search module 5 to search for a recommended route for the robot device RB to move from the start node SB2 to the goal node GB2 in the acquired object map 221B. Specifically, the control unit uses the trained neural network 50 to estimate the cost of each node included in the object map 221B. As a result of the calculation process of the trained neural network 50, the control unit obtains the estimated result 223B of the cost of each node. The control unit prepares a set O and a set C, and empties each of them. The control unit adds the start node SB2 to set O, thereby setting the start node SB2 as the first via node. Then, the control unit calculates the estimated cost 223B of each node based on the obtained estimated cost 223B. Using the selection result 223B, the extraction operation 51 and the selection operation 52 are repeated until the goal node GB2 is selected as the intermediate node. In this way, the control unit can obtain a search result 225B for a recommended route from the start node SB2 to the goal node GB2 that has the minimum cost.

[0159] In step S203, the control unit operates as an output unit and outputs information related to the search result 225B of the recommended route. As an example, the control unit may output the obtained recommended route as it is. As another example, the control unit may navigate so as to drive the robot apparatus RB along the searched recommended route based on the obtained search result 225B. The navigation may be performed, for example, by an image, a sound, or the like. The navigation may be outputting information such as a message to the operator to prompt the operator to drive the robot apparatus RB along the recommended route by manual operation of the operator (i.e., to instruct the operation). Alternatively, when the control device 2B is configured to be able to control the operation of the robot apparatus RB, the navigation may be controlling the operation of the robot apparatus RB so as to drive along the searched recommended route.

[0160] (Features) According to the model generation device 1 of the second specific example, it is possible to reduce the effort required to generate a trained search module 5 that has acquired the ability to search for a route recommended to the robot device RB. Furthermore, the control device 2B can efficiently search for a route recommended to the robot device RB by using the trained search module 5. Furthermore, the model generation device 1 can appropriately perform machine learning of the search module 5 by replacing the extraction operation 51 and the selection operation 52 in the backpropagation phase of machine learning. As a result, it is possible to appropriately generate a trained search module 5 that can efficiently perform the task of searching for a route recommended to the robot device RB.

[0161] (C) Other Although two specific examples have been described above, the above-described embodiment is not limited to these examples in terms of the applicable scenes. As another example, the route search system 100 according to the above-described embodiment may be applied to a scene in which a route using transportation (e.g., an airplane, a train, a bus, etc.) is searched for. In this case, the map (training map, target map) is configured to include any range of a transportation network formed by the transportation. Each node included in the map may correspond to any geographical point such as a base (e.g., a station) of the transportation. The correct answer information may be configured to indicate a route recommended for traveling using the transportation as a recommended route. This allows the model generation device 1 to generate a trained search module 5 that has acquired the ability to search for a route recommended for traveling using the transportation. In addition, the route search device 2 can use the trained search module 5 to search for a route recommended for traveling using the transportation.

[0162] <4.2> The configuration of the neural network 50 may not be limited to the above-mentioned embodiment, and may be changed as appropriate depending on the embodiment. For example, each neuron may be connected to a specific neuron in an adjacent layer, or to a neuron in a layer other than the adjacent layer. The connection relationship of each neuron may be determined as appropriate depending on the embodiment. The intermediate layer 502 may be omitted. The neural network 50 may include other types of layers, such as a convolutional layer, a pooling layer, a normalization layer, and a dropout layer. The neural network 50 may be configured by other types of neural networks, such as a convolutional neural network, a recurrent neural network, and a graph neural network.

[0163] In the above embodiment, the input and output formats of the neural network 50 are as follows: It is not particularly limited and may be appropriately determined depending on the embodiment. For example, the neural network 50 may be configured to further receive input of information other than the map. Also, for example, the neural network 50 may be configured to output information other than the estimation result of the cost of each node.

[0164] In the above embodiment, the extraction operation 51 and the selection operation 52 may be appropriately changed depending on the embodiment. For example, the selection operation 52 may be given by the following formula 13 or formula 14. Accordingly, an alternative operation for the selection operation 52 may be configured by an operation obtained by removing O(n) from each of formulas 13 and 14, similar to the relationship between formulas 3 and 11.

[0165]

number

number

[0166] §5 Experimental Examples In order to verify the effectiveness of the search using the replacement of the backpropagation phase and the cost estimation result obtained by the neural network, the search modules according to the following examples and comparative examples were created, although the present invention is not limited to the following examples.

[0167] First, a trained search module according to the embodiment that has the ability to search for the shortest path from a start node to a goal node in a map composed of a binary image was generated in the same manner as in the above embodiment. The neural network used was the VGG-16 Network (see Reference<URL: https: / / pytorch.org / hub / pytorch_vision_vgg / > We used U-Net, which is composed of 10 layers on the input side of the network (network published in ). For input, we used a two-channel map obtained by combining the input map (where obstacle nodes have a value of 0 and other nodes have a value of 1) and a map where the start node and goal node have a value of 1 and other nodes have a value of 0. We also used a sigmoid function for the activation function of the output layer, and configured the neural network to output values ​​normalized in the range of [0,1] as estimates of the cost of each node. For the training dataset, we used<URL: https: / / github.com / mohakbhardwaj / motion_planning_datasets> We used the samples published in. The map size was 32x32. 800 samples were obtained for each type of obstacle, and a search module was prepared. Then, machine learning was performed for each search module for each type of obstacle. Adam was used as the optimization algorithm for machine learning. The learning rate was set to 0.001. The batch size was set to 100. The number of epochs was set to 100. For the alternative operation of the extraction operation, a convolution operation with kernels of [[0,1,0],[1,0,1],[0,1,0]] was used. For the search algorithm of the selection operation, the A* search algorithm was used. From node v to the target node v g Let h(v,v) be the function that calculates the cost based on the straight-line distance (Manhattan distance) to v. g) was adopted. The above formula 11 (straight-through soft-min with temperature) was used as the alternative calculation for the selection calculation. The square root of 32 was substituted for τ. On the other hand, the neural network was omitted. Then, a search module according to the comparative example was created that is configured to search for the shortest path by the A* search algorithm in the above steps (1) to (4). As in the selection operation of the embodiment, in the comparative example, g Let h(v,v) be the function that calculates the cost based on the straight-line distance (Manhattan distance) to v. g ) was adopted. Three object maps for evaluation were given to the search modules according to the embodiment and each comparative example, and a search for the shortest path from the start node to the goal node was performed in each object map.

[0168] 11A, 11B, and 11C show the object maps used in the evaluation. FIG. 12A, 12B, and 12C show the results of searching for a route in the object maps of FIG. 11A, 11B, and 11C by the search module according to the comparative example (dark gray) and the nodes (set C, light gray) that were the object of the search. FIG. 13A, 13B, and 13C show the results of estimating the cost of each node included in the object maps of FIG. 11A, 11B, and 11C by the neural network of the search module according to the embodiment. FIG. 14A, 14B, and 14C show the results of searching for a route in the object maps of FIG. 11A, 11B, and 11C by the search module according to the embodiment (dark gray) and the nodes (light gray) that were the object of the search.

[0169] As shown in Figs. 14A, 14B, and 14C, the trained search module according to the embodiment was able to properly find the shortest path from the start node to the goal node in the evaluation target map. From this result, it was found that by replacing the backpropagation phase, even if a non-differentiable operation is included, the machine learning of the search module can be properly executed, and thus it is possible to properly generate a trained search module that has acquired the ability to search for a recommended route. Also, as shown in Figs. 12A, 12B, 12C, 14A, 14B, and 14C, the number of nodes to be searched in the embodiment was reduced compared to the comparative example. As shown in Figs. 13A, 13B, and 13C, the trained neural network was able to estimate the cost of each node so that the cost of the area that deteriorates the efficiency of the search is high (i.e., so as to avoid searching the nodes in this area). From this result, it was found that the efficiency of the route search can be properly improved by using the estimated result of the cost of each node obtained by the neural network indirectly trained in the above machine learning. [Explanation of symbols]

[0170] 1...Model generation device, 11: control unit, 12: storage unit, 13: communication interface, 14...External interface, 15...input device, 16...output device, 17...drive, 81...model generation program, 91...storage medium, 111: data acquisition unit; 112: learning processing unit; 113...storage processing section, 125...Learning outcome data, 2...Route search device, 21: control unit, 22: storage unit, 23: communication interface, 24…External interface, 25...input device, 26...output device, 27...drive, 82...Route search program, 92...Storage medium, 211: map acquisition unit, 212: search unit, 213: output unit, 221…Target map, 3…Training data set, 31...Training map, 32...Correct answer information, 5…Exploration module, 50…Neural network, 501...input layer, 502...intermediate (hidden) layer, 503: Output layer, 51...Extraction operation, 52...Selection operation

Claims

1. A data acquisition unit that acquires a plurality of learning data sets each composed of a combination of a training map and correct answer information indicating a recommended route in the training map; A learning processing unit that performs machine learning of a search module using the plurality of learning data sets, The search module includes: a neural network configured to estimate a cost for each of a plurality of nodes in an input map; An operation of extracting one or more candidate nodes adjacent to the via node from a plurality of nodes included in the input map; and and selecting a next transit node from the extracted one or more candidate nodes so as to minimize a total cost based on an estimation result of the neural network; The search module is configured to search for a path from a start node to a target node by setting a start node as a first via node and repeating the extraction and selection operations until a target node is selected as a via node; The machine learning is configured by training the search module for each of the learning data sets by providing the training map as the input map so that a route searched by the search module matches a recommended route indicated by the correct answer information; During the machine learning, in a forward propagation phase, the operation to be extracted and the operation to be selected are calculated as they are, whereas in a backward propagation phase, the operation to be extracted and the operation to be selected are each replaced with a differentiable alternative operation, and an approximate gradient corresponding to each of the differential calculations of the operation to be extracted and the operation to be selected is calculated by a differential calculation of the replaced alternative operation. A learning processing unit; Equipped with Model generation device.

2. The selection operation is configured to select a next via node from one or more extracted candidate nodes so that a total sum of costs is minimized by a predetermined search algorithm based on an estimation result of the neural network. The model generating device according to claim 1 .

3. The predetermined search algorithm is a Dijkstra algorithm, a best-first search algorithm, or an A ＊ A search algorithm, The model generating device according to claim 2 .

4. the training map includes a range in which movement of the moving object is permitted; The recommended route indicated by the correct answer information is a route recommended for the moving object to travel. A model generating device according to any one of claims 1 to 3.

5. the training map includes a range in which the robot device is permitted to be driven; the recommended route indicated by the correct answer information is a route recommended for the operation of the robot device; A model generating device according to any one of claims 1 to 3.

6. The recommended route is the shortest route. A model generating device according to any one of claims 1 to 5.

7. The training map is composed of images; The nodes are composed of pixels. A model generating device according to any one of claims 1 to 6.

8. The image may be a real image or a binary image.

8. The model generating apparatus of claim 7.

9. A map acquisition unit that acquires a target map; A search unit that searches a path from a start node to a goal node in the obtained object map using a trained search module, The trained search module is generated by machine learning using a plurality of learning data sets each of which is composed of a combination of a training map and correct answer information indicating a recommended route in the training map; The search module includes: a neural network configured to estimate a cost for each of a plurality of nodes in an input map; An operation of extracting one or more candidate nodes adjacent to the via node from a plurality of nodes included in the input map; and and selecting a next transit node from the extracted one or more candidate nodes so as to minimize a total cost based on an estimation result of the neural network; The search module is configured to search for a path from a start node to a target node by setting a start node as a first via node and repeating the extraction and selection operations until a target node is selected as a via node; The machine learning is configured by training the search module for each of the learning data sets by providing the training map as the input map so that a route searched by the search module matches a recommended route indicated by the correct answer information; During the machine learning, in the forward propagation phase, the operation to be extracted and the operation to be selected are calculated as they are, whereas in the backward propagation phase, the operation to be extracted and the operation to be selected are each replaced with a differentiable alternative operation, and an approximate gradient corresponding to the differential calculation of each of the operation to be extracted and the operation to be selected is calculated by a differential calculation of the replaced alternative operation. A search section; an output unit that outputs information regarding the result of searching the route on the target map; Equipped with Route finding device.

10. the target map includes a range in which movement of the moving object is permitted; The search unit uses the trained search module to search for a recommended route for the moving object to travel from the start node to the goal node in the acquired object map. The route search device according to claim 9.

11. the object map includes a range in which the robot device is permitted to be driven; The search unit uses the trained search module to search for a recommended path for the robotic device to move from the start node to the goal node in the obtained object map. The route search device according to claim 9.

12. The information regarding the result of the route search is configured by navigation according to the searched route. A route search device according to any one of claims 9 to 11.

13. The computer acquiring a plurality of learning data sets each composed of a combination of a training map and correct answer information indicating a recommended route in the training map; performing machine learning of a search module using the plurality of training data sets; The search module includes: a neural network configured to estimate a cost for each of a plurality of nodes in an input map; An operation of extracting one or more candidate nodes adjacent to the via node from a plurality of nodes included in the input map; and and selecting a next transit node from the extracted one or more candidate nodes so as to minimize a total cost based on an estimation result of the neural network; The search module is configured to search for a path from a start node to a target node by setting a start node as a first via node and repeating the extraction and selection operations until a target node is selected as a via node; The machine learning is configured by training the search module for each of the learning data sets by providing the training map as the input map so that a route searched by the search module matches a recommended route indicated by the correct answer information; During the machine learning, in a forward propagation phase, the operation to be extracted and the operation to be selected are calculated as they are, whereas in a backward propagation phase, the operation to be extracted and the operation to be selected are each replaced with a differentiable alternative operation, and an approximate gradient corresponding to each of the differential calculations of the operation to be extracted and the operation to be selected is calculated by a differential calculation of the replaced alternative operation. Steps and Execute Model generation method.

14. On the computer, acquiring a plurality of learning data sets each composed of a combination of a training map and correct answer information indicating a recommended route in the training map; performing machine learning of a search module using the plurality of training data sets; The search module includes: a neural network configured to estimate a cost for each of a plurality of nodes in an input map; An operation of extracting one or more candidate nodes adjacent to the via node from a plurality of nodes included in the input map; and and selecting a next transit node from the extracted one or more candidate nodes so as to minimize a total cost based on an estimation result of the neural network; The search module is configured to search for a path from a start node to a target node by setting a start node as a first via node and repeating the extraction and selection operations until a target node is selected as a via node; The machine learning is configured by training the search module for each of the learning data sets by providing the training map as the input map so that a route searched by the search module matches a recommended route indicated by the correct answer information; During the machine learning, in the forward propagation phase, the extracted operation and the selected operation are calculated as they are, whereas in the back propagation phase, the extracted operation and the selected operation are replaced with differentiable alternative operations, and the derivatives of the replaced alternative operations are calculated. calculating an approximate gradient corresponding to a differential calculation of each of the extracted operation and the selected operation by calculation; Steps and In order to execute Model generator.

Images