Information processing device, information processing method, and information processing program

Abstract

An information processing device according to an embodiment comprises a recognition unit, a trajectory generation unit, a description generation unit, and a selection unit. The recognition unit generates object information relating to objects present around a moving body and situation information relating to the situation in which the moving body is placed. The trajectory generation unit generates a plurality of different trajectory candidates relating to the movement of the moving body on the basis of moving body information relating to the moving body, the object information, and the situation information. The description generation unit generates a prompt, which is a description regarding behavior control of the moving body, on the basis of the moving body information, the object information, the situation information, and the plurality of trajectory candidates. The selection unit selects a trajectory of the moving body from among the plurality of trajectory candidates by inputting the prompt into a large-scale language model.

Description

Information processing device, information processing method, and information processing program

[0001] The embodiments relate to an information processing device, an information processing method, and an information processing program.

[0002] In technologies such as autonomous driving, after generating multiple trajectories to a destination, the decision on which trajectory to ultimately use is mainly left to human judgment, or a method is adopted in which a trajectory with the lowest cost is selected according to a trajectory cost calculation method prepared in advance by a human.In addition, there is a method in which calculations are devised to calculate the trajectory with the lowest cost at the time of trajectory generation, and only one trajectory is output instead of multiple trajectories (for example, see Non-Patent Document 1 and Non-Patent Document 2).

[0003] Iram Noreen, Amna Khan and Zulfiqar Habib, “Optimal Path Planning using RRT* based Approaches: A Survey and Future Directions” International Journal of Advanced Computer Science and Applications(IJACSA), 7(11), 2016. M. Kamezaki, A. Kobayashi, R. Kono, M. Hirayama and S. Sugano, “Dynamic Waypoint Navigation: Model-Based Adaptive Trajectory Planner for Human-Symbiotic Mobile Robots," in IEEE Access, vol. 10, pp. 81546-81555, 2022.

[0004] However, the index used to select the optimal trajectory from multiple options involves many factors that cannot be expressed mathematically, such as not only the travel distance but also the role assigned to the moving object, its relationship with those around it, the rules of the place where it is moving, and the context. Remote robots and cars are monitored by an operator, so the decision can be left to the operator, but for autonomous robots, it is difficult to change the cost calculation formula for trajectory selection by themselves after they have started moving, and if the initial trajectory selection is incorrect, there is the problem that they will continue to move on the incorrect trajectory.

[0005] The present invention has been made in light of the above circumstances, and its object is to provide an information processing device, an information processing method, and an information processing program that can appropriately and efficiently determine the movement route of a moving object.

[0006] An information processing device according to an embodiment includes a recognition unit, a trajectory generation unit, a description generation unit, and a selection unit. The recognition unit generates object information related to objects present around a moving object and situation information related to a situation in which the moving object is placed. The trajectory generation unit generates a plurality of different trajectory candidates related to the movement of the moving object based on moving object information, the object information, and the situation information related to the moving object. The description generation unit generates a prompt, which is a description related to behavior control of the moving object, based on the moving object information, the object information, the situation information, and the plurality of trajectory candidates. The selection unit selects a trajectory of the moving object from the plurality of trajectory candidates by inputting the prompt into a large-scale language model.

[0007] According to the embodiments, it is possible to provide an information processing device, an information processing method, and an information processing program that can appropriately and efficiently determine a movement route of a moving object.

[0008] FIG. 1 is a block diagram illustrating an information processing device according to this embodiment. FIG. 2 is a flowchart illustrating an example of the operation of the information processing device according to this embodiment. FIG. 3 is a diagram illustrating an example of a list related to objects stored in an object information database. FIG. 4 is a diagram illustrating an example of mobile object information stored in a mobile object information database. FIG. 5 is a diagram illustrating a first example of a prompt generated by the description generation unit and an example of a selection result by the selection unit. FIG. 6 is a diagram illustrating a second example of a prompt generated by the description generation unit and an example of a selection result by the selection unit. FIG. 7 is a diagram illustrating a third example of a prompt generated by the description generation unit and an example of a selection result by the selection unit. FIG. 8 is a diagram illustrating a fourth example of a prompt generated by the description generation unit and an example of a selection result by the selection unit. FIG. 9 is a diagram illustrating a first example of a trajectory generation method by the trajectory generation unit. FIG. 10 is a diagram illustrating a second example of a trajectory generation method by the trajectory generation unit. FIG. 11 is a block diagram illustrating the hardware configuration of an information processing device according to this embodiment.

[0009] Each embodiment will be described below with reference to the drawings. Each embodiment illustrates an apparatus or method for embodying the technical idea of ​​the invention. The drawings are schematic or conceptual. Hereinafter, the same reference numerals are used to designate components having substantially the same functions and configurations. The numbers following the letters that make up the reference numerals are used to distinguish between elements that are referred to by the reference numerals containing the same letters and that have similar configurations. When there is no need to distinguish between elements indicated by reference numerals containing the same letters or numbers, these elements will be referred to by reference numerals containing only letters or numbers.

[0010] An information processing device 1 according to this embodiment will be described with reference to the block diagram of FIG. 1 . The information processing device 1 controls the movement of a mobile object. The information processing device 1 may be mounted on the mobile object, or may be incorporated in a location remote from the mobile object, such as an external server, and may control the movement of the mobile object by wirelessly communicating data between the mobile object and the external server. The mobile object may be a self-propelled robot, a robot driven only by an arm such as an industrial robot, or an aerial vehicle such as a drone, as long as it is a device whose drive can be controlled electronically.

[0011] The information processing device 1 according to this embodiment includes an environment recognition unit 10, an object information database 11, an interference determination unit 12, a trajectory generation unit 13, a description generation unit 14, a selection unit 15, a mobile object information database 16, an output unit 17, and a result storage unit 18.

[0012] The environment recognition unit 10 receives signals captured by an external camera and microphone, and generates target information about targets present around the mobile unit and situation information about the situation the mobile unit is in. Targets here include objects, people, and animals that may become obstacles to the movement of the mobile unit.

[0013] The object information database 11 stores the target information and situation information recognized by the environment recognition unit 10. The interference determination unit 12 determines whether or not the mobile object will interfere with an object between the current location and the destination based on the target information and situation information. The interference determination unit 12 determines an object that may interfere with the mobile object as an object to be avoided.

[0014] The trajectory generation unit 13 generates a plurality of different trajectory candidates for the movement of the mobile object based on the mobile object information, target information, and situation information related to the mobile object. The description generation unit 14 generates a prompt, which is a description related to the behavior control of the mobile object, based on the mobile object information, target information, situation information, and the plurality of trajectory candidates.

[0015] The selection unit 15 selects a trajectory of the moving object from among multiple trajectory candidates by inputting the prompt into a large-scale language model. The selected trajectory is also called a selected trajectory. The moving object information database 16 stores information about the moving object itself.

[0016] The output unit 17 outputs the selected trajectory to the robot. The result storage unit 18 stores information related to the processing results of the information processing device 1, such as the generated multiple trajectory candidates, the finally selected trajectory, and targets to be avoided.

[0017] Next, the information processing device 1 according to this embodiment will be described with reference to the flowchart in Fig. 2. In the following, a self-propelled robot will be described as an example of a moving object. Therefore, moving object information will also be referred to as robot information.

[0018] In step SA1, the environment recognition unit 10 acquires image information and audio information. For example, an RGB image and point cloud information are acquired as image information by using an RGB-D camera mounted on the robot. The point cloud information includes three-dimensional coordinate values ​​for each point. Image information acquired by an RGB-D camera mounted on the robot is not limited to image information acquired by the RGB-D camera, but may also be used in combination with image information acquired by a surveillance camera or the like that captures the space in which the robot is located. This allows for the use of multi-viewpoint video, such as information on points in the robot's blind spots, to obtain more detailed information. Audio information can be acquired by collecting ambient sound using a microphone.

[0019] In step SA2, the environment recognition unit 10 generates object information about objects around the robot and situation information about the robot's environment. Specifically, based on image information, the environment recognition unit 10 may generate object information by recognizing objects, people, and the like using an object detection algorithm based on a machine learning model, such as AffordanceNet or the Yolo series (Yolo V1 to Yolo V8). Alternatively, the environment recognition unit 10 may generate situation information by using separate algorithms for each output, such as SlowFast, to generate behavior recognition results indicating what the detected person is doing and facial orientation. Furthermore, the environment recognition unit 10 may generate the object information and situation information together using a large-scale language model (LLM). The environment recognition unit 10 may store map information about the location of the robot in advance and estimate the robot's location on the map by matching it with a point cloud. The estimated location information is stored in the mobile object information database 16.

[0020] In step SA3, the object information database 11 stores the recognition results by the environment recognition unit 10. That is, based on the target information and situation information, the object information database 11 stores the positions of recognized objects, the positions of people, the recognition results of people's behavior, etc. as a list. It is assumed that the information from step SA1 to step SA3 is executed at predetermined intervals.

[0021] In step SA4, the interference determination unit 12 determines whether the robot will interfere with an object between the robot's current location and the destination. For example, based on the list stored in the object information database 11 and the robot information, a predetermined discriminant is used to determine whether the robot will interfere with the object when moving to the destination. The interference determination method may include, but is not limited to, a method of repeatedly bringing the robot's position and the position of the object to be avoided closer together and determining the interference position and time, or a method of determining the interference position and time using a numerical solution. If the robot will interfere with the object, the process proceeds to step SA5. On the other hand, if the robot will not interfere with the object, the interference determination unit 12 adds interference information (no interference) to the object list in the object information database 11, and the process proceeds to step SA6.

[0022] In step SA5, the collision detection unit 12 sets the object and person determined to collide in step SA6 as targets to be avoided. The collision detection unit 12 also stores the collision information (the occurrence of collision, the time of collision, and the collision position) in the list of targets to be avoided in the object information database 11 in association with the collision information.

[0023] In step SA6, the collision determination unit 12 determines whether collision has been determined for all objects stored in the object information database 11. If collision has been determined for all objects in the list, the process proceeds to step SA7, and if there are any objects that have not been determined, the process returns to step S4 and the same process is repeated.

[0024] In step SA7, the trajectory generation unit 13 generates a plurality of different trajectory candidates, taking into consideration the interference avoidance area of ​​the avoidance target. For example, if a method of setting a waypoint (relay point) outside the interference avoidance area is used to generate the trajectory candidates, the robot's waypoint may be set at a position that is the sum of the radius of the interference avoidance area and the radius of the robot's interference area. ＊ In a search algorithm, a trajectory candidate can be generated by filling in cells as obstacles from the center position of the avoidance target to the interference avoidance area. Similarly, in a method such as the potential method, a trajectory candidate can be generated without entering the interference avoidance area by providing a gradient equal to the diameter of the interference avoidance area. When generating a plurality of different trajectories, the trajectory generation unit 13 generates a trajectory candidate by, for example, A＊ In trajectory generation methods that use cells for calculations, such as search algorithms, one shortest path is generated in advance, so multiple trajectory candidates can be generated by excluding the generated shortest path and generating a new shortest path.

[0025] In step SA8, the description generation unit 14 generates a prompt by inputting, in accordance with a pre-created template, the distance and travel time of each trajectory candidate, information about the target to be avoided if any, and mobile object information stored in the mobile object information database 16, i.e., information about the robot, the role of the robot, the robot's activity environment, etc. Note that when generating a prompt, past prompts stored in the result storage unit 18 may be referenced and the prompt may be generated based on the past prompt.

[0026] In step SA9, the selection unit 15 selects one trajectory from among the plurality of trajectory candidates as the selected trajectory based on the prompt. The trajectory can be selected, for example, by inputting the prompt into a large-scale language model such as BRET, GPT-4, or Claude3 to obtain an estimation result of the optimal trajectory that the moving object should take.

[0027] In step SA10, the output unit 17 outputs, for example, to the robot, trajectory information relating to the selected trajectory selected by the selection unit 15. As a result, the robot that has received the trajectory information is controlled so as to move along the selected trajectory.

[0028] In step SA11, the result storage unit 18 stores the multiple trajectory candidates generated in step SA7, the prompt generated in step SA8, and the selected trajectory estimated in step SA9. The result storage unit 18 references the position information and speed information of each waypoint (or cell), and if an avoidance target exists, the ID of the avoidance target, from the object information database 11, and holds information about the avoidance target. All of the generated trajectory candidates may be saved in the result storage unit 18.

[0029] Next, an example of a list related to objects stored in the object information database 11 will be described with reference to Fig. 3. As shown in Fig. 3, a list in which an ID, object name, position, speed, speed vector, behavior recognition result, face direction, person attribute, state, interference information, interference avoidance area (avoidance amount), and voice recognition result are associated with each object is stored in the object information database 11. Note that it is not necessary to store one list for each object, and information may be stored in one table as long as each item for each object is associated with each other. Other information may also be stored in the list.

[0030] Next, an example of robot information stored in the mobile object information database 16 will be described with reference to FIG. 4 . As shown in FIG. 4 , the robot information includes the robot's own position information, joint information, role and task content, robot size, default avoidance amount, and default speed. If the robot has multiple joints, the joint information includes information such as the length and range of motion of each arm. The role and task content is information about the robot's assigned role and task, i.e., how it should operate. The robot's size includes the three-dimensional dimensions (length, width, and depth) of the robot's body and hands. The default avoidance amount includes the avoidance amount of each of the robot's body and limbs other than the trunk, such as the robot's hands, arms, and legs, which are set as initial values. The default speed is the robot's movement speed, which is set as an initial value. Note that the robot's own position information and joint information data are continuously collected and updated as the robot operates. Values ​​for other items can be entered and changed at any time by voice, keyboard, or other input before the robot starts or during operation.

[0031] Next, a first example of a prompt generated by the description generating unit 14 and a result of trajectory selection by the selecting unit 15 will be described with reference to FIG. 5 . As shown in FIG. 5 , a prompt 50 is described to instruct the robot on its role and job content and to select a selected trajectory from multiple trajectory candidates. Specifically, the following content is described: "You are a robot working as a clerk at a shopping center. Please prioritize customers when moving. Trajectory 1: Number of avoidance attempts: 1, Movement distance: 15 m, Movement time: 8 seconds, Avoidance target 1: Male, adult, holding a basket, passing in front, ID: 3rd left avoidance, ID: , Object name: , Position: , Speed: , Speed ​​vector: , Behavior recognition result: , Face direction: , Person attribute: , Person / object state: , Impact on person: Minimal, [Other information]. Trajectory 2: Number of avoidance attempts: 1, Movement distance: 15 m, Movement time: 8 seconds, Avoidance target 1: Male, adult, holding a basket, passing behind, ID: 3rd right avoidance, Impact on person: Minimal, [Other information]..." In this way, the prompt 50 describes to the avoidance target 1 a plurality of trajectory candidates (trajectory 1, trajectory 2) generated by the trajectory generation unit 13, such as a descriptive sentence 51 about trajectory 1 passing on the left side and a descriptive sentence 52 about trajectory 2 passing on the right side. By inputting the prompt 50 into the large-scale language model, the more suitable trajectory is determined as the selected trajectory. Note that the "other information" may simply be a list of the contents of the object information database 11.

[0032] As shown in FIG. 5 , for objects other than those to be avoided specified in the prompt 50, information about the object may be extracted from the list stored in the object information database 11 and entered as supplemental information, such as [Other Information]. The "impact on people" may be determined based on the situation, such as whether people are crowded around the trajectory candidate. For example, rankings from minimum to maximum may be performed based on the number of people within a 2-meter radius of a waypoint included in the trajectory candidate. The radius value and the ranking of the impact may be parameters that may be appropriately determined based on the situation.

[0033] The template for the prompt 50 is assumed to have the first half describing the role played by the robot and the location where the robot works, as described in the mobile object information database 16, and the second half describing the target to be avoided and other information about surrounding objects and people, as described in the object information database 11. However, the description is not limited to this, and any description that can be similarly interpreted by a large-scale language model may be used. Furthermore, the description is not limited to natural language, and may be written in a data description language such as XML (extensible Markup Language).

[0034] 5, the selection result 55 of the selected trajectory in response to the prompt 50 by the selection unit 15 is assumed to be "Because it is preferable to pass behind the person rather than in front of them, select 'Trajectory 2'." In this way, in determining whether to pass in front of the person or behind them, there is no need to pass in front of the person unless there is a special reason, so 'Trajectory 2', which is a path that passes behind them, is selected as the selected trajectory.

[0035] Next, a second example of a prompt generated by the description generating unit 14 and the result of trajectory selection by the selecting unit 15 will be described with reference to Fig. 6. Prompt 60 in Fig. 6 is an example in which there are differences in distance and impact on the surrounding environment between the trajectories, as can be seen from description sentence 61 for trajectory 1 and description sentence 62 for trajectory 2, which are trajectory candidates. Specifically, the content is described as follows: "You are a robot working as a clerk at a shopping center. Please move with priority given to customers. Trajectory 1: Number of evasive attempts: 3, 3 contacts, travel distance: 6 m, travel time: 9 seconds, impact on people: large, person to be evaded 1: male, adult, contact with left hand and then pass to the right, id 2nd left evasion, (object information), person to be evaded 2: female, adult, contact with right hand and then pass to the left, id 5th left evasion, (object information), person to be evaded 3: male, adult, contact with left hand and then pass to the right, id 6th right evasion: (object information), Trajectory 2: Number of evasive attempts: 0, detour route, travel distance: 13 m, travel time: 18 seconds, impact on people: minimal [Other information]..." In the following, for the sake of convenience, items stored in the list of the object information database 11, such as "ID:, object name:, position:, speed:, speed vector:, behavior recognition result:, face direction:, person attribute:, person / object state:...", will be described in the "(object information)" section.

[0036] In the example of FIG. 6 , the selection result 65 of the selected trajectory by the selection unit 15 in response to the prompt 60 is assumed to be, "Because priority is given to customers, select 'Trajectory 2,' which is a detour route that avoids areas where people are crowded." The situation shown in FIG. 6 is a situation in which a robot in a shopping center is deciding whether to select a route that is crowded with people or a detour route that takes a slight turn and makes a detour. Trajectory 1 is significantly shorter in travel distance than trajectory 2, and if distance alone were considered as cost, trajectory 1 would likely be selected. However, trajectory 1 involves multiple contacts with the target. Therefore, if trajectory 1 were selected, it would cause inconvenience to people and would likely have a widespread impact on the contacted person and those around them. Therefore, trajectory 1 is deemed to be a significantly high-cost trajectory when considering the surrounding environment. On the other hand, trajectory 2 has the opposite characteristics to trajectory 1, and the robot has almost no impact on the surrounding environment. However, the distance to the destination of trajectory 2 is twice that of trajectory 1. The robot's role in the prompt 60 is "to move with priority given to the staff working at the shopping center and customers," so trajectory 2, a detour route, is selected over trajectory 1, which would affect the target and people.

[0037] Next, a third example of a prompt generated by the description generating unit 14 and the result of trajectory selection by the selecting unit 15 will be described with reference to Fig. 7. A prompt 70 in Fig. 7 is similar to Fig. 6 in that a description 71 of trajectory 1 and a description 72 of trajectory 2 are trajectory candidates, but is a prompt that assumes a case where the robot is running in an emergency. Specifically, the message reads, "You are a robot working as a security guard at a shopping center. You have spotted a suspicious person, so please move immediately. Trajectory 1: Number of evasive attempts: 3, 3 contacts, Movement distance: 6 m, Movement time: 9 seconds, Impact on people: Extremely large, Person to be evaded 1: Male, adult, Contact with left hand and then pass to the right, id 2nd left evasion, (object information), Person to be evaded 2: Female, adult, Contact with right hand and then pass to the left, id 5th left evasion, (object information), Person to be evaded 3: Male, adult, Contact with left hand and then pass to the right, id 6th right evasion: (object information), Trajectory 2: Number of evasive attempts: 0, Detour route, Movement distance: 13 m, Movement time: 18 seconds, Impact on people: Extremely small [Other information]..."

[0038] In the example of Fig. 7, the selection result 75 of the selected trajectory in response to the prompt 70 by the selection unit 15 is assumed to be "Because the role of security guard requires the robot to move quickly, select 'Trajectory 1', which has a short travel distance and time." In this way, in Fig. 7, the roles assigned to the robots are different, and because they are tasked with speed, they need to reach their destination (such as the location of a suspicious person or a security center) as quickly as possible. Therefore, 'Trajectory 1' is selected, which prioritizes the role over the impact on the surrounding environment.

[0039] Next, a fourth example of a prompt generated by the description generating unit 14 and the result of the trajectory selection by the selecting unit 15 will be described with reference to Fig. 8. Prompt 80 in Fig. 8 is a prompt assuming a case where a robot is patrolling for security purposes, and a determination is made as to whether to select a description statement 81 of trajectory 1 or a description statement 82 of trajectory 2, which are trajectory candidates, depending on the target situation. Specifically, the content is described as follows: "You are a robot working as a security guard at a venue where poster presentations are being held. Please patrol the venue to provide security. Trajectory 1: Number of evasive attempts: 3, crossed the street during a conversation, traveled distance: 6 m, travel time: 10 seconds, impact on people: large, person to be avoided 1: male, adult, talking to a woman at position [x, y, z], id1: evasive attempt to the left, (object information), person to be avoided 2: female, adult, talking to a man at position [a, b, c], id2: evasive attempt to the left, (object information), Trajectory 2: Number of evasive attempts: 0, detour route, traveled distance: 13 m, travel time: 18 seconds, impact on people: minimal [other information]..."

[0040] In the example of Fig. 8, the selection result 85 of the selected trajectory in response to the prompt 80 by the selection unit 15 is "It is better not to pass between people who are conversing, so select 'Trajectory 2', which is a detour route." In this way, the robot can pass between people who are conversing at the poster venue, but in consideration of general etiquette estimated from the relationship between the two people, it chooses not to pass between the people, and as a result, trajectory 2 is selected. In other words, a trajectory can be selected according to the relationship between people.

[0041] Next, an example of a method for calculating interference information by the interference determination unit 12 will be described. Here, a method for determining whether or not there is interference using a numerical solution will be shown as an example. The position and velocity vectors of the robot and the object are used for interference determination. The environment recognition unit 10 arranges object information and situation information that are added as needed in chronological order, and calculates the velocity vector for each object and person from the difference between the three-dimensional coordinate information of the latest position information and the three-dimensional coordinate information included in the position information obtained in the previous process. The positions and velocities of object A and robot R at a certain time t [seconds] are respectively expressed as x ｔ Ａ→ , v ｔ Ａ→ , x ｔ Ｒ→ , v ｔ Ｒ→ Here, the superscript arrow " → " indicates that the value is a vector. In this embodiment, in order to handle the three-dimensional movement of the robot R, the vectors of the three axes are expressed as x １ , x ２ , x ３ This is expressed as equation (1).

[0042]

[0043] Next, the relative position x between the object A and the robot R ｔ （Ａ－Ｒ）→ , and the relative velocity between object A and robot R is v ｔ （Ａ－Ｒ）→ Let the current time be t ０ [seconds], the relative position x at any time t [seconds] ｔ （Ａ－Ｒ）→ can be expressed by equation (2).

[0044]

[0045] By solving equation (2), equation (3) is obtained.

[0046]

[0047] Therefore, the relative position x ｔ （Ａ－Ｒ）→is compared with the sum of the radius of the interference area included in the object information and the radius of the interference area of ​​the hand included in the robot information, and interference is determined. If the value of the square root of equation (3) is positive, it is determined that interference occurs, and if it is negative, it is determined that no interference occurs. If interference is determined, the interference time t １ In addition, since the calculation is based on the assumption that the robot R moves at a constant speed in a straight line, the current position, relative speed, and collision time t １ From the above, the interference position x ｔ１ Ｒ→ The interference determination unit 12 performs the above-mentioned calculation for determining interference, and calculates the presence or absence of the calculated interference and the interference time t １ and interference position x ｔ１ Ｒ→ The information relating to the above is treated as interference information, and the interference information is added to the list in the object information database 11. The interference determination unit 12 executes interference determination for all objects listed in the object information database 11.

[0048] Next, a first example of a trajectory generation method of the trajectory generation unit 13 will be described with reference to Fig. 9. The first example shown in Fig. 9 is an example in which a waypoint is set as an avoidance trajectory, and the robot R avoids the target OBJ by moving via the waypoint.

[0049] The trajectory generation unit 13 connects the current position of the robot R to the destination point with a straight line as the shortest path, and calculates the collision time t １ Position x of the target OBJ ｔ１ ＯＢＪ→ , a plurality of waypoints 150 are set in a direction perpendicular to the shortest route. ｔ１ ＯＢＪ→ In a direction perpendicular to the shortest path, the interference avoidance area IA of the target OBJ is Ｏ and the interference avoidance area IA of the robot R Ｒ The position corresponding to the sum of the distances is set as Waypoint 150. ＲThe default avoidance amount of the robot R stored in the mobile body information database 16 can be used for the interference avoidance amount. As shown in FIG. 9, in the method of setting the waypoint 150, basically, multiple trajectories can be generated, so all the generated trajectories can be set as multiple trajectory candidates. Also, even if only one waypoint 150 is generated, the interference avoidance area IA can be set as the multiple trajectory candidates. Ｏ By setting multiple sizes, multiple Waypoints can be generated.

[0050] Next, a second example of the trajectory generation method of the trajectory generation unit 13 will be described with reference to Fig. 10. In the second example shown in Fig. 10, the positional relationship between the target OBJ and the robot R is arranged on a grid. The trajectory generation unit 13 calculates the interference avoidance area IA of the target OBJ. Ｏ and the interference avoidance area IA of robot R Ｒ The trajectory generation unit 13 searches the grid so that the interference avoidance area IA does not overlap with the interference avoidance area IA, and searches for an avoidance trajectory until the robot R reaches the destination. Ｏ and interference avoidance area IA Ｒ For example, the RRT (Rapidly-exploring Random Tree) method, A ＊ Law, RRT ＊ In this case, when the robot R moves three squares on the grid toward the destination, the target OBJ's interference avoidance area IA Ｏ and the interference avoidance area IA of robot R Ｒ Since these overlap, a route is searched for to avoid them to the right and reach the destination via the shortest route.

[0051] In addition, RRT and A ＊are trajectory generation methods that generate only one trajectory, so it is necessary to generate trajectories repeatedly to generate multiple trajectory candidates. Therefore, for example, a condition for repeatedly generating 10 trajectories is set that the trajectory passes through a cell or relay point different from the trajectories already generated. In the example of FIG. 10 , assuming that a trajectory candidate 160 is generated in the first trajectory generation process, a trajectory candidate 161 is generated in the second trajectory generation process by setting a constraint to generate a trajectory other than trajectory candidate 160. Furthermore, in the third trajectory generation process, a constraint is set to prevent the trajectory candidate 161 and trajectory candidate 162 from being generated, and trajectory candidate 162 is generated. By performing such a process multiple times, the trajectory generation process is repeated, and the RRT and A ＊ Even if the

[0052] Next, an example of the hardware configuration of the information processing device 1 according to this embodiment will be described with reference to the block diagram shown in Fig. 11. As shown in Fig. 11, the information processing device 1 includes, for example, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a display 104, a communication interface 105, and a storage 106.

[0053] The CPU 101 is an integrated circuit capable of executing various programs and controls the overall operation of the information processing device 1. The ROM 102 is, for example, a non-volatile semiconductor memory and stores programs and control data for controlling the information processing device 1. The RAM 103 is, for example, a volatile semiconductor memory and is used as a work area for the CPU 101. The display 104 is, for example, a liquid crystal display or a touch panel display and displays information. The communication interface 105 is, for example, an input / output component for exchanging information with the outside, such as a USB, an HDMI (registered trademark), or a network interface. The storage 106 is a non-volatile storage device. The storage 106 stores system software and the like for the information processing device 1. The CPU 101 may also be called a "processor."

[0054] In the above embodiments, the CPU 101 of the information processing device 1 may be another circuit (or processor). For example, the information processing device 1 may include a GPU (Graphics Processing Unit), an NPU (Neural Network Processing Unit), an MPU (Micro Processing Unit), or the like instead of a CPU. Each of the processes described in each embodiment may be realized by dedicated hardware. The processes of the information processing device 1 may be a mixture of processes executed by software and processes executed by hardware, or may be only one of them.

[0055] According to the present embodiment described above, the environment recognition unit generates target information and situation information. The trajectory generation unit generates a plurality of different trajectory candidates for the movement of the moving object based on moving object information, target information, and situation information related to the moving object. The description generation unit generates a prompt, which is a description related to behavior control of the moving object, based on the moving object information, target information, situation information, and the plurality of trajectory candidates. The selection unit selects a trajectory of the moving object from the plurality of trajectory candidates by inputting the prompt into a large-scale language model.

[0056] This makes it possible to select an appropriate route based on the robot's task, the surrounding human environment, and the context, which are difficult to express mathematically. In other words, by inputting various information, it becomes possible to select a route similar to that of a human. Furthermore, even if the task content suddenly changes, it is possible to select a trajectory appropriate for that task, eliminating the need for humans to manually change parameters or set them up in advance. This therefore enables mobility that is highly socially acceptable. As a result, the movement route of a moving object can be determined appropriately and efficiently.

[0057] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0058] 1... Information processing device 10... Environment recognition unit 11... Object information database 12... Interference determination unit 13... Trajectory generation unit 14... Description generation unit 15... Selection unit 16... Mobile body information database 17... Output unit 18... Result storage unit 51, 52, 61, 62, 71, 72, 81, 82... Description sentence 50, 60, 70, 80... Prompt 55, 65, 75, 85... Selection result 104... Display 105... Communication interface 106... Storage 150... Waypoint 160, 161, 162... Trajectory candidate IA Ｏ ...Interference avoidance area IA Ｒ …Interference avoidance area

Claims

1. An information processing device comprising: a recognition unit that generates object information regarding objects existing around a moving object and situation information regarding the situation in which the moving object is placed; a trajectory generation unit that generates a plurality of different trajectory candidates for the movement of the moving object based on the moving object information, the object information, and the situation information regarding the moving object; a description generation unit that generates a prompt, which is a description regarding behavioral control of the moving object, based on the moving object information, the object information, the situation information, and the plurality of trajectory candidates; and a selection unit that selects a trajectory of the moving object from the plurality of trajectory candidates by inputting the prompt into a large-scale language model.

2. The information processing device of claim 1, further comprising a determination unit that determines an object that may interfere with the moving object as an object to be avoided based on the moving object information, the target information, and the situation information, and the description generation unit generates a prompt including an amount of avoidance related to the object to be avoided.

3. The information processing device according to claim 1, wherein when the trajectory generation unit uses a trajectory generation method that generates the shortest path, the trajectory generation unit generates the multiple trajectory candidates by executing the trajectory generation method multiple times, excluding the shortest path generated immediately before.

4. An information processing method comprising: generating object information relating to objects existing around a moving body and situation information relating to the situation in which the moving body is placed; generating a plurality of different trajectory candidates for the movement of the moving body based on the moving body information, the object information, and the situation information relating to the moving body; generating a prompt that is a description of behavioral control of the moving body based on the moving body information, the object information, the situation information, and the plurality of trajectory candidates; and inputting the prompt into a large-scale language model to select a trajectory for the moving body from the plurality of trajectory candidates.

5. An information processing program for causing a computer to function as each part of the information processing device according to claim 1.

Images