Information processing device, control method, and program

Abstract

To enable a mobile object to act autonomously even in areas where map information is not available in advance.SOLUTION: An information processing device includes: object characteristic group information input means for inputting object characteristic group information including at least two or more pieces of object characteristic information consisting of type information indicating a type name of an object and three-dimensional position information of the object in a space; an object arrangement characteristic database for storing the type information in association with position information; purpose information input means for inputting purpose information which is information regarding a purpose of the moving object; and prediction means for predicting, as a first prediction, based on the object arrangement characteristic database, an arrangement relation of at least one or more objects including a first object and a second object that are present between a first position and a second position, using as input the object characteristic group information regarding the first object present at the first position and the type information regarding the second object present at the second position different from the first position determined based on the purpose information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, a control method, and a program. [Background technology]

[0002] In recent years, technologies have been proposed that allow moving objects such as automated guided vehicles to act autonomously in environments such as factories and logistics warehouses. An example of an automated guided vehicle is an AGV. AGV is an abbreviation for Automated Guided Vehicle. For example, a moving object can be moved autonomously from a starting point to a destination via predetermined waypoints.

[0003] When a mobile object is caused to move autonomously, a map of the real space may be created, and self-position and orientation measurement may be performed to measure the position and orientation of the mobile object itself in the real space.

[0004] In Patent Document 1, a moving body that moves autonomously within a predetermined area measures its own position within the predetermined area, and attempts to make it possible to observe various landmarks that exist throughout the area. For this reason, Patent Document 1 discloses a method for allowing a moving body to reach a destination by generating a moving route that allows the landmarks to be observed, using map information that includes information about the landmarks within the predetermined area. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-86588 A

[0006] [Non-Patent Document 1] Tateno et al., “CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction”, CVPR2019 [Non-Patent Document 2] Ashish et al., “Attention is All you Need”, NeuralIPS2017 [Non-Patent Document 3] Jacob et al., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, arXiv 2018 [Non-Patent Document 4] Diederik et al., “ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION”, ICLR2015 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the method of Patent Document 1, map information about a specific area is required before the moving object moves autonomously within the specific area. Therefore, if map information cannot be obtained in advance, the moving object cannot move autonomously.

[0008] The present invention has been made in consideration of the above problems, and has an object to allow a mobile object to act autonomously even in an area where map information is not available in advance. [Means for solving the problem]

[0009] An information processing device according to one embodiment of the present invention comprises an object characteristic group information input means for inputting object characteristic group information including at least two or more pieces of object characteristic information consisting of type information indicating a type name of an object and three-dimensional position information of the object in space, an object placement characteristic database for retaining the type information in association with position information, a destination information input means for inputting destination information which is information regarding the destination of a moving body, and a prediction means for predicting, as a first prediction based on the object placement characteristic database, the placement relationship of at least one or more objects including the first object and the second object that are present between the first position and the second position, using the object characteristic group information regarding a first object present at a first position and the type information regarding a second object present at a second position different from the first position determined based on the destination information. Effect of the Invention

[0010] According to the present invention, a mobile object can be caused to act autonomously even in an area where map information is not available in advance. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of a usage scene of a mobile system including an information processing device according to a first embodiment of the present invention. [Diagram 2] 1 is an overall configuration diagram of a mobile system including an information processing device according to a first embodiment of the present invention. [Diagram 3] 1 is a hardware configuration diagram of an information processing device according to a first embodiment of the present invention. [Figure 4] 1 is a functional configuration diagram of a mobile system including an information processing device according to a first embodiment of the present invention. [Diagram 5] 4 is a flowchart showing a processing procedure of the information processing device according to the first embodiment of the present invention. [Figure 6] 1 is a flowchart showing a prediction processing procedure of the information processing device according to the first embodiment of the present invention. [Figure 7] 4 is an example of a data structure showing an object arrangement relationship in an object arrangement characteristic database according to Example 1 of the present invention. [Figure 8] 1 is a specific example showing a prediction process of the information processing device according to the first embodiment of the present invention. [Figure 9] FIG. 11 is a functional configuration diagram of a mobile system including an information processing device according to a second embodiment of the present invention. [Figure 10] 10 is a flowchart showing a processing procedure of an information processing device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the scope of the present invention, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the present invention.

[0013] [Example 1] First, a first embodiment of the present invention will be described. In this embodiment, an example will be described in which the method of the present invention is applied to a mobile system that recognizes objects existing in the surrounding space by image recognition and autonomously travels near the recognized objects.

[0014] In this embodiment, the mobile body autonomously moves within a building such as an apartment building or an office building, and when the name of a destination such as "Room 103" is specified, the mobile body automatically sets a route from the starting point to the destination and starts autonomous movement. In this embodiment, detailed map information about the building that would be necessary for the mobile body to move autonomously is not given to the mobile body in advance.

[0015] In this embodiment, the system predicts the layout of a group of objects from the layout of objects in a building to the destination based on an object layout characteristic database that consolidates the "layout of objects" in real space. Then, in this embodiment, the system automatically determines a moving path for a moving body to follow the group of objects based on the predicted layout of objects.

[0016] The arrangement relationship of objects in this embodiment refers to the characteristics of the relationship between objects, such as the combination pattern and occurrence frequency of the positional relationship and contact relationship between objects in real space, their changes over time, and their occurrence probability. For example, it refers to the relationship between objects that is common sense for humans, such as there is often a chair next to a desk, but there is almost never a chair on the desk, or there are desks and chairs indoors but rarely outdoors.

[0017] 1 is a diagram showing an example of a usage scene of a mobile body system including an information processing device according to a first embodiment of the present invention. A space 500 inside a building is a predetermined area in which a mobile body system 100 moves autonomously in this embodiment. The mobile body system 100 is an autonomously moving body including an information processing device according to the present invention.

[0018] Room 101 501 is a room (room 101) as an object within a building that exists in space 500. Room 102 502 and room 103 503 are also rooms as objects within a building that exists in space 500, similar to room 101 501. Hallway 504 is a hallway as an object within a building that exists in space 500 within the building.

[0019] Furthermore, rooms 101 501 and 102 502 are positioned "adjacent" to each other as objects within a building, and similarly rooms 102 502 and 103 503 are positioned "adjacent" to each other as objects within a building.

[0020] In this embodiment, the objects existing in the building in which the mobile system 100 moves autonomously are the building's structural elements such as floors, doors, walls, ceilings, elevators, and stairs. Furthermore, in this embodiment, specific spaces constructed from a plurality of building's structural elements, such as the entrance, corridors, elevator halls, and rooms such as Room 101, are also treated as objects existing in the building. In these specific spaces, the objects are arranged in such a way that they are "adjacent to each other," "on the left side as you face them," "on the right side as you face them," "in front of them," or "above them."

[0021] 2 is a diagram showing an example of the overall configuration of a mobile body system including an information processing device according to this embodiment. The mobile body system 100 includes a first sensor 101, a second sensor 102, a driving unit 103, an object characteristic group information generating device 110, an information processing device 120, a position and orientation measuring device 130, and a mobile body control device 140.

[0022] The first sensor 101 measures a predetermined area in which the mobile system 100 travels and generates first sensor data. The first sensor 101 in this embodiment is a camera that captures an image of the surroundings of the mobile system 100, and generates a captured image as the first sensor data, that is, image data in which a brightness value is recorded for each pixel.

[0023] The second sensor 102 generates sensor data by measuring the area ahead of the mobile system 100. In this embodiment, the second sensor 102 is a distance sensor, and generates distance data to an object present ahead of the mobile system 100 as the second sensor data.

[0024] The driving unit 103 is a driving device for moving the mobile body system 100. The driving unit 103 includes, for example, wheels, a motor for rotating the wheels, and a steering device for changing the traveling direction.

[0025] The object characteristic group information generating device 110 acquires first sensor data generated by the first sensor 101, and generates three-dimensional shape data with an object label based on the first sensor data. The object characteristic group information generating device 110 also generates object characteristic group information based on the generated three-dimensional shape data with an object label.

[0026] The information processing device 120 predicts the relative positions of objects existing between the departure point and the destination based on the object position characteristic database, and determines the travel route of the moving object based on the predicted relative positions.

[0027] The position and orientation measurement device 130 acquires first sensor data generated by the first sensor 101, and performs position and orientation measurement in a predetermined area based on the first sensor data.

[0028] The mobile body control device 140 controls the driving unit 103 based on the position and orientation information acquired from the position and orientation measurement device 110 and the movement route acquired from the information processing device 120, thereby controlling the autonomous movement of the mobile body system 100. In addition, the mobile body control device 140 acquires second sensor data from the second sensor 102, detects an obstacle present in front of the mobile body system 100, and stops the movement of the mobile body system 100 when an obstacle is detected.

[0029] 3 is a diagram showing a hardware configuration of the information processing device 120. Reference numeral 311 denotes a CPU, which executes the processing of the present invention. CPU is an abbreviation for Central Processing Unit. The CPU 311 also controls various devices connected to a system bus 320.

[0030] Reference numeral 312 denotes a ROM, which stores a BIOS program and a boot program. ROM is an abbreviation for Read Only Memory. BIOS is an abbreviation for Basic Input Output System. Reference numeral 313 denotes a RAM, which is used as a main storage device for the CPU 311. RAM is an abbreviation for Random Access Memory. Reference numeral 314 denotes an external memory, which stores programs to be processed by the information processing device 120.

[0031] The input unit 315 is a keyboard, mouse, or robot controller, and performs processing related to input of information, etc. The display unit 316 outputs the calculation results of the information processing device 120 to a display device according to instructions from the CPU 311. The display device may be of any type, such as a liquid crystal display device, a projector, or an LED indicator. 317 is an I / O. I / O is an abbreviation for Input / Output.

[0032] 4 is a diagram showing an example of the functional configuration of a mobile system including an information processing device according to this embodiment. An object characteristic group information generating device 110 includes a first sensor data acquiring unit 111, a three-dimensional shape data generating unit 112, a three-dimensional shape data holding unit 113, an object recognizing unit 114, a three-dimensional shape data labeling unit 115, and an object characteristic group information calculating unit 116.

[0033] Moreover, the information processing device 120 has a target information input unit 121, an object characteristic group information input unit 122, an object arrangement characteristic database 123, a prediction unit 124, and an action plan determination unit 125. Moreover, the position and orientation measurement device 130 has a first sensor data acquisition unit 131, and a position and orientation measurement unit 132. Moreover, the mobile object control device 140 has an input unit 141, a display content determination unit 142, a display unit 143, and a movement control unit 144.

[0034] The first sensor data acquisition unit 111 acquires the first sensor data from the first sensor 101. In this embodiment, the first sensor data is image data.

[0035] The three-dimensional shape data generating unit 112 generates three-dimensional shape data based on the first sensor data. The three-dimensional shape data is data indicating a three-dimensional shape model. The three-dimensional shape model is a data structure in which an object instance ID (which object exists in the environment) and the type of the object are assigned to a three-dimensional point group. That is, each point group is assigned an ID indicating which object it is and what type of object it is. Such a three-dimensional shape model can be generated by, for example, a known method. The known method is, for example, the method described in Non-Patent Document 1.

[0036] The object recognition unit 114 performs object labeling of pixels by semantic segmentation based on the first sensor data.

[0037] The three-dimensional shape data labeling unit 115 assigns an object label to the three-dimensional shape data based on the three-dimensional shape model and the object label. The three-dimensional shape data storage unit 113 stores the generated three-dimensional shape data with object labels. The details of this series of processes are described in Non-Patent Document 1, so a detailed description will be omitted.

[0038] The object characteristic group information calculation unit 116 extracts object type and position information as object characteristic group information from the labeled three-dimensional shape model created as described above. The object characteristic information includes object type information in which a numeric label is assigned to each type of object present in the environment (e.g., desk, chair, wall, floor, etc.), and position information consisting of the three-dimensional coordinates (X, Y, Z) of the object. These are assumed to be generated from a three-dimensional shape model three-dimensionally restored by SfM or SLAM. SfM is an abbreviation for Structure From Motion. SLAM is an abbreviation for Simultaneous Localization and Mapping. The object characteristic group information includes multiple object characteristic information included in a certain environment (i.e., a certain three-dimensional shape model). The object characteristic group information includes at least two or more pieces of object characteristic information consisting of type information indicating the type name of the object and three-dimensional position information of the object in space.

[0039] In this embodiment, the object characteristic group information calculation unit 116 extracts the label of each object in the 3D shape model as object type information and the center of gravity of the object as position information as object characteristic information. For object regions to which a label has not yet been assigned by the above method, the object type is extracted as unknown, that is, object type information that holds only position information.

[0040] The destination information input unit 121 inputs information about the destination of the mobile system 100 from the input unit 141 .

[0041] The object characteristic group information input unit 122 inputs the object characteristic group information created by the object characteristic group information calculation unit 116 .

[0042] The object placement characteristic database 123 is a database that holds placement characteristics that indicate the positional relationship of a plurality of objects. The placement characteristics are knowledge data that generalize the three-dimensional positional relationship of objects in the real world. Specifically, the object placement characteristic database 123 holds characteristics such as "chairs are often found next to desks, but never on top of them" and "desks and chairs are found indoors, but rarely outdoors," as mentioned above. The placement characteristics held in the object placement characteristic database 123 can also be said to be characteristics such as what kind of placement of objects is common and what kind of placement is not common in reality.

[0043] The object location characteristic database 123 in this embodiment is a pre-trained neural network trained to infer unknown object characteristic group information related to objects at two certain locations from object characteristic information at the current location. Specifically, it is a trained neural network in which 24 layers of Transformers described in Non-Patent Document 2 are stacked. In this embodiment, the number of input dimensions and the number of output dimensions of the Transformer are 512, that is, a maximum of 512 pieces of object characteristic information are input, and the same number of 512-dimensional outputs are obtained. Specifically, the encoder network used in Non-Patent Document 3 is used.

[0044] Furthermore, the object arrangement characteristic database 123 in this embodiment has information on a priori probability indicating the proportion of arrangement relationships between one or more objects that exist in a given arrangement relationship between objects.

[0045] The prediction unit 124 predicts unknown object characteristic information from the object characteristic group information input by the object characteristic group information input unit 122 and the objective information input by the objective information input unit 121. In this prediction, the prediction unit 124 predicts unknown object characteristic information related to objects at two certain points from the object characteristic information of the current location, among the object characteristic information included in the object characteristic group information, using the object arrangement characteristic database 123. The prediction unit 124 outputs this prediction result. The prediction result by the prediction unit 124 is held in a holding unit (not shown). The holding unit is provided, for example, outside the information processing device 100.

[0046] The behavior plan determination unit 125 determines a behavior plan for the mobile body system 100 based on the positional relationship of the objects predicted by the prediction unit 124.

[0047] The first sensor data acquiring unit 131 is the same as the first sensor data acquiring unit 111. That is, the first sensor data acquiring unit 131 acquires the first sensor data from the first sensor 101.

[0048] The position and orientation measurement unit 132 measures the position and orientation of the mobile system 100 based on the first sensor data acquired by the first sensor data acquisition unit 131. In this embodiment, the position and orientation measurement unit 117 performs position and orientation measurement using SLAM.

[0049] The input unit 141 receives information about the purpose of the mobile system 100 from the user.

[0050] The display content determination unit 142 determines the content to be displayed on the display unit 143 based on the action plan of the mobile system 100 determined by the action plan determination unit 125 .

[0051] The display unit 143 displays the display content determined by the display content determination unit 142 on a screen of the mobile system 100 or a peripheral device of the mobile system 100 .

[0052] The movement control unit 144 controls the driving unit 103 to make the mobile system 100 travel automatically, based on the action plan of the mobile system 100 determined by the action plan determination unit 125 and the position and orientation measurement results by the position and orientation measurement unit 117 .

[0053] Fig. 5 is a flowchart showing a processing procedure of the information processing device 120 according to this embodiment. The processing in Fig. 5 is automatically started, for example, when the information processing device 120 is powered on and the information processing device 120 starts up.

[0054] In step S101, the information processing device 120 initializes the system. That is, the CPU 311 reads and executes a program from the external memory 314 to put the information processing device 120 into an operable state. Furthermore, the CPU 311 reads weight parameters of a neural network, which is the object arrangement characteristic database 123, from the external memory 314 as necessary, and expands the weight parameters in the RAM 313. After the series of initialization processes in step S101 are completed, the information processing device 120 executes the process in step S102.

[0055] In step S102, the destination information input unit 121 inputs information related to the destination of the mobile system 100 from the input unit 141. In this embodiment, the destination information input unit 121 inputs a name indicating the destination of the mobile system 100 in a building (e.g., "Room 103").

[0056] In step S103, the object characteristic group information input unit 122 inputs object characteristic group information. In this embodiment, the object characteristic group information input unit 122 inputs object characteristic group information related to an object detected at a certain point by the mobile system 100. Specifically, the object characteristic group information input unit 122 inputs object characteristic group information related to an object (such as "entrance" or "room 101") in a building detected by the mobile system 100 at a starting point or current location. Then, the object characteristic group information input unit 122 converts the input object characteristic group information into a data structure that can be recognized by the object arrangement characteristic database 123, and outputs the data to the prediction unit 124. The data structure that can be recognized by the object arrangement characteristic database 123 is a feature vector (object type vector) in which object type labels are arranged, and a position vector that indicates the positions of the objects.

[0057] In step S104, the prediction unit 124 inputs the destination information input from the destination information input unit 121 and object characteristic group information based on the object characteristic group information input from the object characteristic group information input unit 122 to the object arrangement characteristic database 123, and executes a prediction process. Details of the prediction process will be described later with reference to Fig. 6. Through this prediction process, the prediction unit 124 predicts the arrangement relationship of objects that exist between the departure point and the destination, including objects at the departure point and objects at the destination.

[0058] In step S105, the behavior plan determination unit 125 determines a behavior plan for the mobile body system 100 based on the positional relationship of the objects predicted by the prediction unit 124. In this embodiment, a movement route for moving to the vicinity of the next object adjacent to the object at the current location is determined based on the positional relationship of the objects existing between the departure point and the destination predicted in step S104. By repeating the process of step S105, the mobile body system 100 can move to the destination by passing through the vicinity of the objects existing between the departure point and the destination one after another.

[0059] In step S106, the information processing device 120 judges whether or not the destination has been reached. Specifically, it judges whether or not the object type label of the object in the building detected at the current location input in step S103 matches the name of the destination input in step S102. If the information processing device 120 judges that the destination has been reached, the process of step S107 is executed. On the other hand, if the information processing device 120 judges that the destination has not been reached, the process of step S103 is executed.

[0060] In step S107, the information processing device 120 judges whether or not to end the process. Specifically, if new object information has been input, the information processing device 120 judges not to end the process. If new object information has not been input, the information processing device 120 judges to end the process. If the information processing device 120 judges not to end the process, the process of step S102 is executed. On the other hand, if the information processing device 120 judges to end the process, the process is ended as it is.

[0061] As described above, the object arrangement characteristic database 123 in this embodiment is a pre-trained neural network that is trained to infer unknown object characteristic information related to objects at two certain locations from object characteristic information at the current location. In this embodiment, an object type vector and a position vector are input to the object arrangement characteristic database 123 as object characteristic group information.

[0062] The object type vector is a one-dimensional column vector. The object type vector is a vector in which the object type labels of objects existing in the environment are arranged in a row, with the first element being a CLS token, followed by the object label corresponding to the first object characteristic group information, followed by the object label corresponding to the second object characteristic group information, and so on. The CLS token is a special label that indicates the beginning of the data. In addition, a MASK token (a special label that indicates that the object type is unknown) is placed as a label corresponding to unknown objects that have not yet been identified.

[0063] The position vector is a one-dimensional column vector consisting of three elements, each of which is the three-dimensional position X, Y, and Z of each object. In other words, each column of the position vector stores the X, Y, and Z values, which are the position coordinates of the object in the corresponding column included in the object type vector. In addition, if the position is still unknown, a MASK token is placed.

[0064] Then, for the input object type vector and position vector, the object placement characteristic database 123 forward propagates the calculation result using the neural network held in the object placement characteristic database 123, and outputs an output vector. At this time, the MASK tokens included in the input object type vector and position vector are replaced with specific object labels and position information based on the weights of the neural network held in the object placement characteristic database 123. This means that unknown objects and unknown positions are determined, and in this embodiment, this is a prediction process using the object placement characteristic database 123.

[0065] In this embodiment, object characteristic group information is generated at a first location, and object characteristic information is generated by masking the name of the object at a second location as an object type label and the position information, and added to the object characteristic group information. The object characteristic group information generated at the first location includes an object type vector and a position vector.

[0066] Furthermore, object characteristic information in which the object type and position information are MASKed is created and added to the object characteristic group information. When the object characteristic group information (object type vector and position vector including MASK token) is input to the object arrangement characteristic database 123, an output vector in which the MASK token has been replaced is obtained. In other words, the arrangement relationship of one or more objects that are directly or indirectly adjacent to an object at a first location and an object at a second location and that include an object at the first location and an object at the second location is output. In this embodiment, the object arrangement characteristic database 123 learns to output such an object arrangement relationship.

[0067] 6 is a flowchart showing a prediction process procedure of the information processing device 120 according to the present embodiment, specifically, the details of the process procedure shown in step S104 in FIG.

[0068] In step S201, the prediction unit 124 acquires object characteristic group information (object type vector and position vector) input at the previous point, which is stored in a storage unit (not shown), and an output vector indicating the arrangement relationship of objects as a prediction result at the previous point. The object characteristic group information input at the previous point includes the object type vector and the position vector.

[0069] Here, the object type vector of the object characteristic group information input from the object characteristic group information input unit 122 at the current location is called the first object type vector. Also, the object type label indicating the object detected at the current location is called the second object type label. Also, the object type vector of the output vector at the previous location acquired in step S201 is called the third object type label. Also, the object type label indicating the object detected at the previous location is called the fourth object type label. In step S202, the prediction unit 124 compares the second object type vector from the first object type vector with the object type label that appears next to the fourth object type label from the third object type label. This means that it is determined whether the predicted result up to the current location is correct for the positional relationship of the object predicted at the previous location.

[0070] In step S203, the prediction unit 124 judges whether the object type labels match in step S202, that is, whether the previous prediction result is correct. If the prediction unit 124 judges that the object type labels match, that is, that the previous prediction result is correct, the process ends. If the prediction unit 124 judges that the object type labels do not match, that is, that the previous prediction result is incorrect, the process ends. The process of step S204 is executed.

[0071] In step S204, the prediction unit 124 adds the object characteristic group information (object type vector and position vector) input from the object characteristic group information input unit 122 at the current location to the object characteristic group information input at the previous location. In step S204, the prediction unit 124 further generates object characteristic information with the name indicating the destination input from the destination information input unit 121 as an object type label and masked with the position information, and adds it to the object characteristic group information.

[0072] In step S204, the prediction unit 124 further creates object characteristic information in which the object type and position information are MASKed, and adds it to the object characteristic group information. In step S204, the prediction unit 124 further inputs the object characteristic group information (object type vector and position vector) to the object arrangement characteristic database 123. For the input object type vector and position vector, the object arrangement characteristic database 123 replaces MASK tokens included in the object type vector and position vector based on the neural network held by the object arrangement characteristic database 123. As a result, the object arrangement characteristic database 123 outputs an output vector (object type vector and position vector). This output vector indicates the arrangement relationship of the objects, and the object arrangement characteristic database 123 outputs at least one output vector.

[0073] Here, the one or more output vectors output by the object arrangement characteristic database 123 in step S204 are called first output vectors. Also, an output vector that can reach an object type label corresponding to a destination from an object type label corresponding to a current location via an adjacent object type label is called a second output vector. In step S205, the prediction unit 124 extracts a second output vector from the first output vector. If one or more output vectors are extracted, in step S205, the prediction unit 124 selects one output vector from the multiple output vectors. In this embodiment, in step S205, the prediction unit 124 selects an output vector indicating the most likely arrangement relationship of objects as a prediction result based on the prior probability of the arrangement relationship of objects.

[0074] In step S206, the prediction unit 124 stores the object characteristic group information (object type vector and position vector) input at the current location in the storage unit. In step S206, the prediction unit 124 further stores the output vector (object type vector and position vector) indicating the positional relationship of the objects output in step S204 in the storage unit, and ends the process (ends the process of step S104 in FIG. 5).

[0075] In this embodiment, the positional relationship of objects (specific spaces in a building) that are "common sense" expected to exist between the starting point and the destination of a moving object is predicted based on the object position characteristic database. For example, if the name "Room 103" is set as the destination of a moving object, the following object positional relationship is predicted to exist between the starting point of the moving object, "Entrance", and the destination, "Room 103". That is, the following object positional relationship is predicted to exist: "Entrance" → (front) → "Hallway" → (next door) → "Room 101" → (next door) → "Room 102" → (next door) → "Room 103". Here, the parentheses "()" are prepositions indicating positional relationships. Then, a moving route of the moving object is set so that the moving object moves from the starting point to the destination via the vicinity of the object predicted to exist.

[0076] This allows the mobile body to use its knowledge of "common sense layout relationships" of buildings to set a route to its destination, even if it cannot be given detailed map information about buildings in advance that is necessary for the mobile body's autonomous movement, enabling it to move autonomously.

[0077] FIG. 7 is an example of a data structure of the arrangement relationship of objects held by object arrangement characteristic database 123 in this embodiment. Reference numeral 1100 denotes the entire data structure of object arrangement characteristic database 123. Object information 1101 is data indicating the type of object in object arrangement characteristic database 123. In this embodiment, object information 1101 indicates "entrance" as an object within a building. Like object information 1101, object information 1104 is data indicating the type of object in object arrangement characteristic database 123. In this embodiment, object information 1104 indicates "corridor" as an object within a building.

[0078] The position information 1102 is data indicating the positional relationship between the object information 1101 and the object information 1104. In this embodiment, the position information 1102 indicates "forward" as the position from the "entrance" of the object information 1101 to the "corridor" of the object information 1104.

[0079] The prior probability 1103 is data indicating the probability that the positional relationship between the object information 1101 and the object information 1104 will occur. In this embodiment, the prior probability 1103 is indicated as a percentage or a decimal value between 0 and 1. In this embodiment, the probability that the "hallway" will appear after the "entrance" as an object in a building is 100%, so the prior probability 1103 indicates 100%. The same applies below, so explanations of other data will be omitted.

[0080] Fig. 8 is a specific example of prediction processing in the information processing device 120 described in Fig. 5 and Fig. 6 in this embodiment. Column 1201 indicates the number of trials of the flowchart described in Fig. 5. Column 1202 indicates object characteristic information detected by the mobile system 100 at the current location. In this embodiment, column 1202 indicates an object indicating a specific space within a building.

[0081] Column 1203 indicates the previous prediction result acquired by the prediction unit 124 in step S201 of Fig. 6. Column 1204 indicates whether the previous prediction result determined by the prediction unit 124 in step S203 of Fig. 6 was correct.

[0082] Column 1205 indicates object characteristic group information input by the prediction unit 124 to the object arrangement characteristic database 123 in step S204 of FIG. 6. Column 1206 indicates a predicted result pattern of the arrangement relationship of one or more objects output by the object arrangement characteristic database 123 in step S204 of FIG. 6. Column 1207 indicates a prior probability calculated for each predicted result pattern of the arrangement relationship of one or more objects output by the object arrangement characteristic database 123 in step S204 of FIG. 6. In this embodiment, column 1207 is a value obtained by multiplying all the prior probabilities between objects in the arrangement relationship patterns of each object that can be taken from the departure point to the destination. Column 1208 indicates the most likely pattern selected by the prediction unit 124 based on the prior probability from among the predicted result patterns of the arrangement relationship of one or more objects output by the object arrangement characteristic database 123 in step S204 of FIG. 6. A column 1209 indicates an object that is predicted to be detected at the next point in the selected prediction result pattern indicated by the column 1208.

[0083] Row 1210 shows the result of the first processing of the flowchart. The current location in row 1210 is the "entrance", which is the starting point. Three patterns of object arrangement relationships are output as the predicted result patterns in row 1210, and the most likely pattern is selected from among them based on the prior probability. Here, the pattern with the highest prior probability in column 1207, <<"Entrance" → (forward) → "Hallway" → (next door) → "Room 101" → (next door) → "Room 102" → (next door) → "Room 103", is selected.

[0084] Line 1220 shows the result of the second processing of the flowchart. The current location in line 1220 is the "corridor", which is the same object as predicted previously, so no further prediction is performed.

[0085] Line 1230 shows the result of the third processing of the flowchart. The current location in line 1230 is "Room 104". The previous prediction result predicted "Room 101", so the prediction was incorrect. Therefore, the object characteristic group information to which the currently detected object "Room 104" has been added is input into the object arrangement characteristic database 123 for re-prediction. The result of the re-prediction is <<"Entrance" → (forward) → "Hallway" → (next door) → "Room 104" → (next door) → "Room 103">>.

[0086] Line 1240 shows the result of the fourth processing of the flowchart. The current location in line 1240 is "Room 103," and since the destination has been reached, the processing of the flowchart ends.

[0087] According to the method of this embodiment, even if there is no detailed map information of the inside of a building in advance, a travel route to a destination can be automatically set for the mobile body system 100, and the mobile body system 100 can move autonomously.

[0088] <Modification of Example 1> In the first embodiment, when the previous prediction result was incorrect, the prediction unit 124 added the object characteristic group information input from the object characteristic group information input unit 122 at the current location to the object characteristic group information input at the time of the previous prediction, inputted the information into the object arrangement characteristic database 123, and performed the prediction process again. However, the present invention is not limited to this.

[0089] For example, the prediction unit 124 may input only the object characteristic group information of the current location and the type information of the destination into the object arrangement characteristic database 123 and perform the prediction process again. This makes it possible to limit the prediction to only the object arrangement relationship from the point where the prediction was wrong to the destination, compared to the prediction of the object arrangement relationship from the departure point to the destination, and thus makes it possible to speed up the prediction process of the prediction unit 124.

[0090] Furthermore, the prediction unit 124 holds only one prediction result pattern selected as the previous prediction result, but the present invention is not limited to this.

[0091] For example, multiple or all of the prediction result patterns output from the object arrangement characteristic database 123 at the time of the previous prediction may be stored. This allows the prediction unit 124 to determine whether there is a prediction result pattern that was not selected last time but was actually predicted correctly from among the multiple or all of the previous prediction result patterns stored, if the result of this determination shows that there is a prediction result pattern that was actually predicted correctly, and selects it as the current prediction result pattern. This makes it unnecessary to perform prediction processing again using the object arrangement characteristic database 123, thereby making it possible to make the prediction processing of the prediction unit 124 more efficient.

[0092] In addition, the prediction unit 124 may determine that the previous prediction result was incorrect and perform a re-prediction, and as a result, the route from the previous location to the destination may be closer in distance than the route from the current location to the destination. In this case, the action plan determination unit 125 may determine a travel route to return to the previous location. This allows a more efficient travel route to be set for the mobile system 100.

[0093] Furthermore, when the prediction unit 124 determines that the previous prediction result was incorrect, the action plan determination unit 125 may determine a moving route for the mobile system 100 to search the surroundings, or may change the operation mode. Searching the surroundings means moving thoroughly around the surroundings centered on the current point, moving to a place around the current point that has not yet been visited between the starting point and the current point, etc. Changing the operation mode means moving the first sensor 101 (camera) provided in the mobile system 100 to search the surroundings, slowing down the moving speed of the mobile system 100, etc. This makes it easier to detect surrounding objects, and increases the possibility that the mobile system 100 will reach the destination.

[0094] In this embodiment, the prediction unit 124 selects one most probable pattern of the object arrangement relationship based on the prior probability of the object arrangement relationship from among the prediction result patterns of the arrangement relationship of one or more objects output by the object arrangement characteristic database 123. However, the present invention is not limited to this.

[0095] For example, distance information indicating a general distance between objects or direction information indicating a direction may be held in place of the prior probability (prior probability 1103 in FIG. 7) in the object location characteristic database 123. This allows the prediction unit 124 to select a pattern that allows the shortest route from the departure point to the destination based on the distance information or direction information. This allows a more efficient travel route to be determined for the mobile system 100 in terms of the time required for travel.

[0096] The prediction unit 124 may select a prediction result pattern according to the type of object predicted to appear. For example, when the prediction result includes an object that requires attention in terms of safety when the mobile body passes through or a moving object, the prediction result pattern is selected so as to avoid the object. Specifically, this is the case when stairs, escalators, passenger elevators for people, etc. are included. Conversely, when the prediction result includes an object that requires consideration in terms of safety when the mobile body passes through, the prediction result pattern is selected so as to pass through the object. Specifically, in the case of a prediction result in which the mobile body system 100 needs to move to an upper floor of a building, a pattern that includes a baggage elevator rather than a passenger elevator is selected with priority, thereby reducing the risk of the mobile body system 100 coming into contact with a human being. This makes it possible to determine a safer movement route for the mobile body system 100.

[0097] Furthermore, in the prediction unit 124, when the reliability of the prediction result is low, it is possible that the destination cannot be reached even if the vehicle proceeds according to the prediction result. A case where the reliability of the prediction result is low is, for example, when there are many patterns of the prediction result of the positional relationship of the object from the current location to the destination. A case where the reliability of the prediction result is low is, for example, when the result of the prediction based on the prior probability is that there is only a pattern where the positional relationship of the object from the current location to the destination has a prior probability lower than a predetermined value. A case where the reliability of the prediction result is low is, for example, when the result of the prediction based on the distance information or the direction information is that there is only a pattern where the distance from the current location to the destination is greater than a predetermined value.

[0098] In this way, when the reliability of the prediction result is low, the prediction unit 124 may temporarily predict another point as a tentative destination instead of the original destination. The other point is, for example, a position where an object exists such that, among the object positional relationships from the predicted current location to the destination, the object positional relationship has a prior probability higher than a predetermined value, or the object positional relationship is shorter than a predetermined distance.

[0099] Similarly, the action plan determination unit 125 may temporarily determine a moving route to another point for the moving body system 100, or may change the operation mode to search the surroundings for the above-mentioned moving body system 100. This allows the moving body system 100 to operate in a way that increases the possibility of reaching the destination.

[0100] In the first embodiment, the object arrangement characteristic database 123 outputs a pattern of the arrangement relationship of one or more objects appearing between the departure point and the destination, but the present invention is not limited to this. For example, a prediction result of only the arrangement relationship of objects from the current location to the next location may be output. This makes it possible to limit the prediction result patterns, and shorten the time required for one prediction process in the prediction unit 124.

[0101] In the first embodiment, the prediction unit 124 compares the positional relationship of the object predicted at the departure point with the object detected at the next point to determine whether the previous prediction result is correct or not, but the present invention is not limited to this. For example, the prediction unit 124 may predict only one pattern of the positional relationship of the object from the departure point to the destination, and may not make a sequential determination at the next point. This can reduce the amount of processing required for sequential determination, and can reduce the processing load of the prediction unit 124.

[0102] Furthermore, the prediction unit 124 may predict the positional relationship of objects by combining related information related to a specific area in which the mobile system 100 moves. Specific examples of related information include information that does not have accurate position or orientation, such as a floor map with room numbers, and information that does not include names related to specific spaces, such as room numbers, in building structure data, such as CAD and BIM. CAD is an abbreviation for Computer Aided Design. BIM is an abbreviation for Building Information Modeling. In other words, related information is an information source that cannot determine a movement route by itself.

[0103] As shown in the first embodiment, the prediction unit 124 can predict various patterns of room arrangement relationships in a building that exist in common sense, with information on room numbers such as "Room 103". As predicted patterns of room arrangement relationships, for example, there is a pattern in which rooms are lined up in a row on one side of a corridor, and there is also a pattern in which rooms are lined up on both sides of the corridor, and both patterns are possible. On the other hand, the above-mentioned CAD and BIM building structure data does not include information on specific spaces such as room numbers. However, since the building structure data has information on the arrangement relationship of objects, such as whether they are lined up in a row on one side of a corridor or on both sides of the corridor, it becomes possible to narrow down multiple prediction result patterns predicted by the prediction unit 124. Similarly, although a floor map is information that does not have accurate position or orientation, it is possible to narrow down multiple prediction result patterns predicted by the prediction unit 124 because room numbers are described.

[0104] Specifically, the prediction unit 124 may analyze the structural arrangement of a building from related information about the building, convert it into text, and select a pattern of a prediction result with a high degree of match by matching it with the text of the arrangement relationship of objects output from the object arrangement characteristic database 123. Also, based on the arrangement of objects appearing in the CAD, the prediction may be made by tracing links of the arrangement relationship of objects in the object arrangement characteristic database 123. This can improve the prediction accuracy of the arrangement relationship of objects.

[0105] In step S202 of Fig. 6, the prediction unit 124 judges whether the object characteristic group information input from the object characteristic group information input unit 122 at the current location matches the type of the next object predicted to be adjacent to the object detected at the previous location. However, the present invention is not limited to this, and it may be judged whether the object type and the object arrangement relationship match. In other words, it is judged not only that the predicted object is adjacent, but also that it is adjacent to the predicted position. This can improve the prediction accuracy.

[0106] In the first embodiment, the behavior plan determination unit 125 determines the movement path of the mobile body system 100 based on the positional relationship of the objects predicted by the prediction unit 124, but the present invention is not limited to this. For example, when applied to a cleaning robot that autonomously cleans a predetermined area, when predicting the positional relationship of objects in the predetermined area, the behavior plan determination unit 125 may determine tools and equipment required for cleaning according to the objects predicted to be present. This makes it possible to determine a more suitable behavior according to the purpose of the mobile body.

[0107] In the first embodiment, the position and orientation measurement unit 117 performs position and orientation measurement using SLAM, but is not limited to this, and other position and orientation measurement units such as GPS may be used. GPS is an abbreviation for Global Positioning System.

[0108] In the first embodiment, the movement control unit 144 performs movement control based on the position and orientation measurement by the position and orientation measurement unit 117. However, the present invention is not limited to this, and for example, movement control may be performed based on the recognition result of the surrounding environment by the image recognition unit 114. Specifically, the image recognition unit 114 detects a passage through which the mobile system 100 can move from the surrounding environment. The movement control unit 144 may perform movement control based on the passage detected by the image recognition unit 114 and the movement route determined by the action plan determination unit 125.

[0109] [Example 2] Next, a second embodiment of the present invention will be described. In the above first embodiment, a method for predicting the layout relationship of objects in a building using an object layout characteristic database is described. In this embodiment, a method for generating (updating) an object layout characteristic database is described.

[0110] Fig. 9 is a diagram showing an example of a functional configuration of a mobile system including an information processing device according to a second embodiment of the present invention. As shown in Fig. 9, the second embodiment has an object arrangement characteristic database update unit 126 that updates an object arrangement characteristic database 123 in addition to the configuration of the first embodiment shown in Fig. 4.

[0111] In the second embodiment, the object characteristic group information input unit 122 outputs the input object characteristic group information to the object arrangement characteristic database update unit 126. Also, unlike the first embodiment, the object characteristic group information, which is partly modified, is output to the prediction unit 124. The modification of the object characteristic group information will be described later.

[0112] The prediction unit 124 outputs the object characteristic group information predicted by the object arrangement characteristic database 123 to the object arrangement characteristic database update unit 126. The object arrangement characteristic database update unit 126 updates the weights of the object arrangement characteristic database 123 based on the object characteristic group information input by the object characteristic group information input unit 122 and the object characteristic group information predicted by the prediction unit 124. The object arrangement characteristic database update unit 126 outputs the updated weights to the object arrangement characteristic database 123.

[0113] Fig. 10 is a flowchart showing a processing procedure of the information processing device 120 according to the embodiment 2. In the explanation of Fig. 10, the explanation of the same steps as in Fig. 5 of the embodiment 1 will be omitted, and only the steps different from the embodiment 1 will be explained. The process explained in Fig. 10 starts when an update start signal for the object arrangement characteristic database 123 is input by the input unit 141.

[0114] In step S101, the information processing device 120 performs initialization processing of the object arrangement characteristic database 123 in addition to the processing described in the first embodiment. That is, it initializes the weights of the object arrangement characteristic database 123, which is a neural network. While any initialization method can be used, in this embodiment, it is initialized with random values ​​generated from a normal distribution with a mean of 0 and a variance of 1.

[0115] In step S103, the object characteristic group information input unit 122 inputs the object characteristic group information. Next, in step S103, the object characteristic group information input unit 122 generates an object type vector and a position vector as described in the first embodiment. The difference from the first embodiment is that some elements of the object type vector and the position vector are rewritten. Specifically, the object type label and the position vector immediately after the object type label indicating the current location to the object type label and the position vector immediately before the object type label indicating the destination are replaced with a MASK token (a label indicating that the corresponding object type is unknown). In step S103, the object characteristic group information input unit 122 outputs the object characteristic group information thus modified to the prediction unit 124. Also, in step S103, the object characteristic group information input unit 122 outputs the object characteristic group information before modification to the object arrangement characteristic database update unit 126.

[0116] In step S104, the prediction unit 124 inputs the object type vector and the position vector included in the modified object characteristic group information to the object arrangement characteristic database 123. The object arrangement characteristic database 123 sequentially propagates the calculation results to the network, and obtains an output vector (predicted object arrangement relationship).

[0117] In step S301, the object arrangement characteristic database update unit 126 updates the object arrangement characteristic database 123 based on the arrangement relationship of the objects predicted by the object arrangement characteristic database 123 and the arrangement relationship of the objects before modification. The object arrangement characteristic database update unit 126 updates the object arrangement characteristic database 123 so that the object arrangement characteristic database 123 predicts the modified MASK part, that is, so as to solve the filling problem of the object characteristic group information.

[0118] The updating of the object arrangement characteristic database 123 is realized by backpropagation of errors, and the neural network is trained using the method described in Non-Patent Document 4 in which the weights of the neural network are changed continuously.

[0119] As described above, in the second embodiment, the object layout characteristic database is updated so as to predict the modified object characteristic information. That is, the object layout characteristic database can predict unknown object characteristic information based on the layout relationship of the object, i.e., the layout relationship with the surrounding objects. By using the object layout characteristic database updated in this way, more accurate recognition can be realized.

[0120] <Modification of Example 2> In the second embodiment, the object placement characteristic database update unit 126 updates the weights of the object placement characteristic database 123 based on the prediction result of the prediction unit 124, but the present invention is not limited to this. For example, when the object placement characteristic database 123 holds the prior probability between objects, the object placement characteristic database update unit 126 may update the prior probability of the object placement characteristic database 123 according to the prediction result of the prediction unit 124. Furthermore, the object placement characteristic database update unit 126 may update both the weights and the prior probability of the object placement characteristic database 123 according to the prediction result of the prediction unit 124.

[0121] Specifically, when it is determined in step S203 of Fig. 6 that the previous prediction result is correct, in step S301 of Fig. 10, the object placement characteristic database update unit 126 updates the object placement characteristic database 123. At this time, the object placement characteristic database update unit 126 updates the object placement characteristic database 123 so as to increase the value of the prior probability related to the output vector (object placement relationship) at the previous point.

[0122] On the other hand, if it is determined that the previous prediction result is incorrect, the object placement characteristic database update unit 126 updates the object placement characteristic database 123 so as to reduce the value of the prior probability related to the output vector (object placement relationship) at the previous point.

[0123] This makes it possible to achieve more accurate predictions when making predictions using prior probabilities in the object characteristic database.

[0124] Similarly, when the object location characteristic database 123 holds distance information between objects, the object location characteristic database update unit 126 may update the distance information with distance information measured when the mobile system 100 actually moves between objects at two points. This allows for more accurate prediction when making predictions using distance information in the object characteristic database.

[0125] In step S101 in Fig. 10 of the second embodiment, the object arrangement characteristic database 123 starts with the weights initialized, but the present invention is not limited to this, and the object arrangement characteristic database 123 may start from a learned state. Specifically, similar to step S101 in Fig. 5 of the first embodiment, the weight parameters of the object arrangement characteristic database 123 are read from the external memory 314 and expanded in the RAM 313. This makes it possible to efficiently learn the object characteristic database.

[0126] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0127] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0128] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) an object characteristic group information input means for inputting object characteristic group information including at least two pieces of object type information representing the type name of an object and object characteristic information consisting of three-dimensional position information of the object in space; an object location characteristic database that stores the type information in association with position information; a destination information input means for inputting destination information which is information regarding the destination of the moving object; using as input the object characteristic group information on a first object present at a first position and the type information on a second object present at a second position different from the first position determined based on the target information, a prediction means for predicting, as a first prediction based on the object layout characteristic database, a layout relationship of at least one or more objects, including the first object and the second object, that exist between the first position and the second position; An information processing device comprising: (Configuration 2) and a behavior plan determination means for planning the behavior of the moving object based on the positional relationship of at least one of the objects predicted by the prediction means. 2. The information processing device according to configuration 1. (Configuration 3) the object characteristic group information input means inputs the object characteristic group information regarding a third object that is present at a third position different from the first position and the second position; the prediction means judges whether or not the third object is present in the arrangement relationship of the objects predicted as the first prediction, or whether or not the third object is present but is different from the predicted arrangement relationship, and judges whether or not the third object is not present; or when it is determined that the third object exists but the positional relationship is different from the predicted positional relationship, it is determined that the first prediction is incorrect, and the type information regarding the second object and the object characteristic group information regarding the third object are input, predicting, as a second prediction, a positional relationship of the third object and at least one or more objects including the third object, which are present between the third position and the second position, based on the object positional characteristic database; 3. The information processing device according to configuration 1 or 2. (Configuration 4) the action plan determination means determines, based on the first prediction, a movement route for the moving object to move from the first position to the second position via a vicinity of an object existing between the first position and the second position; 3. The information processing device according to configuration 2. (Configuration 5) a behavior plan determination means for planning a behavior of the moving body based on the positional relationship of at least one of the objects predicted by the prediction means, When the prediction means determines that the first prediction is incorrect for the object characteristic group information regarding the third object present at the third position, the behavior plan determination means determines a movement route that searches the periphery of the third position. 4. The information processing device according to configuration 3. (Configuration 6) the object arrangement characteristic database holds information on a prior probability for each arrangement relationship between at least two or more objects; The prediction means makes a prediction based on information about the prior probability stored in the object arrangement characteristic database. selecting a most probable arrangement relationship from among arrangement relationships of at least one or more objects, including the first object and the second object, that exist between the first object and the second object; 6. The information processing device according to any one of configurations 1 to 5. (Configuration 7) the object arrangement characteristic database holds information on distance or direction for each arrangement relationship between at least two or more objects; The prediction means makes a prediction based on information about the distance or direction stored in the object arrangement characteristic database. selecting a positional relationship that results in the shortest distance from among positional relationships of at least one or more objects, including the first object and the second object, that exist between the first object and the second object; 6. The information processing device according to any one of configurations 1 to 5. (Configuration 8) When the reliability of the first prediction is low, the prediction means receives as input the type information regarding a fourth object that is different from the second object and is more likely based on information regarding the prior probability or has a shorter distance based on information regarding the distance or direction, re-predicting a positional relationship of at least one or more objects, including the first object and the fourth object, that are present between the first position and the fourth position; 8. The information processing device according to any one of configurations 1 to 7. (Configuration 9) a related information acquiring means for acquiring related information related to a positional relationship of objects in an area in which the moving body is active; the prediction means selects, based on the related information acquired by the related information acquisition means, from among the predicted arrangement relationships of one or more objects, including the first object and the second object, that exist between the first object and the second object; 9. The information processing device according to any one of configurations 1 to 8. (Configuration 10) and an object placement characteristic database update means for updating the object placement characteristic database based on the object characteristic group information input by the object characteristic group information input means and the prediction result by the prediction means. 10. The information processing device according to any one of configurations 1 to 9. (Method 1) an object characteristic group information input step of inputting object characteristic group information including at least two pieces of object type information representing a type name of an object and object characteristic information consisting of three-dimensional position information of the object in space; an object location characteristic database that stores the type information in association with position information; a destination information input step of inputting destination information which is information regarding the destination of the moving object; using as input the object characteristic group information on a first object present at a first position and the type information on a second object present at a second position different from the first position determined based on the target information, a prediction step of predicting, as a first prediction based on the object layout characteristic database, a layout relationship of at least one or more objects including the first object and the second object that are present between the first position and the second position; A control method comprising: (Program 1) A program for causing a computer to function as each of the means according to any one of configurations 1 to 10. [Explanation of symbols]

[0129] 100 Mobile Systems 101 First Sensor 102 Second Sensor 103 Drive unit 110 Object property group information generation device 111 first sensor data acquisition unit 112 Three-dimensional shape data generation unit 113 Three-dimensional shape data storage unit 114 Object recognition section 115 3D Shape Data Labeling Department 116 Object characteristic group information calculation unit 120 Information processing device 121 Purpose information input section 122 Object characteristic group information input section 123 Object Arrangement Characteristics Database 124 Prediction Department 125 Action Plan Decision Division 126 Object placement characteristics database update unit 130 Position and Orientation Measurement Device 131 first sensor data acquisition unit 132 Position and Orientation Measurement Unit 140 Mobile control device 141 Input section 142 Display content determination section 143 Display section 144 Movement control section

Claims

1. an object characteristic group information input means for inputting object characteristic group information including at least two pieces of object type information representing a type name of an object and object characteristic information consisting of three-dimensional position information of the object in space; an object location characteristic database that stores the type information in association with position information; a destination information input means for inputting destination information which is information regarding the destination of the moving object; a prediction means for predicting, as a first prediction, a positional relationship of at least one or more objects including the first object and the second object that are present between the first position and the second position, based on the object positional characteristic database, using as input the object characteristic group information regarding a first object that is present at a first position and the type information regarding a second object that is present at a second position different from the first position that is determined based on the objective information; An information processing device comprising:

2. The system further includes a behavior plan determination unit that plans the behavior of the moving object based on the positional relationship of at least one of the objects predicted by the prediction unit.

2. The information processing apparatus according to claim 1,

3. the object characteristic group information input means inputs the object characteristic group information regarding a third object that is present at a third position different from the first position and the second position; the prediction means judges whether or not the third object is present in the arrangement relationship of the objects predicted as the first prediction, or whether or not the third object is present but the arrangement relationship is different from the predicted arrangement relationship, and when it is judged that the third object is not present, or the third object is present but the arrangement relationship is different from the predicted arrangement relationship, it judges that the first prediction is incorrect, and predicts, as a second prediction, based on the object arrangement characteristic database, the arrangement relationship of the third object and at least one or more objects including the third object that are present between the third position and the second position, using the type information on the second object and the object characteristic group information on the third object as inputs; 2. The information processing apparatus according to claim 1,

4. the action plan determination means determines, based on the first prediction, a movement route for the moving object to move from the first position to the second position via a vicinity of an object existing between the first position and the second position; 3. The information processing apparatus according to claim 2.

5. a behavior plan determination means for planning a behavior of the moving body based on the positional relationship of at least one of the objects predicted by the prediction means, When the prediction means determines that the first prediction is incorrect for the object characteristic group information regarding the third object present at the third position, the action plan determination means determines a movement route for searching the periphery of the third position.

4. The information processing apparatus according to claim 3.

6. the object arrangement characteristic database holds information on a prior probability for each arrangement relationship between at least two or more objects; the prediction means selects a most probable arrangement relationship from among arrangement relationships of at least one or more objects, including the first object and the second object, that exist between the first object and the second object, predicted based on information about the prior probability stored in the object arrangement characteristic database; 2. The information processing apparatus according to claim 1,

7. the object arrangement characteristic database holds information on distance or direction for each arrangement relationship between at least two or more objects; the prediction means selects a placement relationship resulting in the shortest distance from among placement relationships of at least one or more objects, including the first object and the second object, present between the first object and the second object, predicted based on information regarding the distance or direction stored in the object placement characteristic database; 2. The information processing apparatus according to claim 1,

8. When the reliability of the first prediction is low, the prediction means uses as an input the type information on a fourth object which is different from the second object and which is more likely based on the information on the prior probability or which has a shorter distance based on the information on the distance or direction, and predicts again a positional relationship of at least one or more objects including the first object and the fourth object which are present between the first position and the fourth position.

2. The information processing apparatus according to claim 1,

9. a related information acquiring means for acquiring related information related to a positional relationship of objects in an area in which the moving body is active; the prediction means selects, based on the related information acquired by the related information acquisition means, from among the predicted arrangement relationships of one or more objects, including the first object and the second object, that exist between the first object and the second object; 2. The information processing apparatus according to claim 1,

10. and an object placement characteristic database update means for updating the object placement characteristic database based on the object characteristic group information input by the object characteristic group information input means and the prediction result by the prediction means.

2. The information processing apparatus according to claim 1,

11. an object characteristic group information input step of inputting object characteristic group information including at least two pieces of object type information indicating a type name of an object and object characteristic information consisting of three-dimensional position information of the object in space; an object location characteristic database that stores the type information in association with position information; a destination information input step of inputting destination information which is information regarding the destination of the moving object; a prediction step of predicting, as a first prediction based on the object layout characteristic database, a layout relationship of at least one or more objects including the first object and the second object that are present between the first position and the second position, using as input the object property group information on a first object present at a first position and the type information on a second object present at a second position different from the first position determined based on the target information; A control method comprising:

12. A program for causing a computer to function as each of the means according to any one of claims 1 to 10.

Images