Information processing device, information processing method, and information processing program

Abstract

An information processing device according to an embodiment comprises a recognition unit, a generation unit, an estimation unit, and a determination unit. The recognition unit generates object information relating to an object present near a moving body and situation information relating to the situation in which the moving body is placed. The generation unit generates a prompt, which is a description regarding behavior control of the moving body, on the basis of moving body information relating to the moving body, the object information, and the situation information. The estimation unit inputs the prompt into a large-scale language model and estimates an avoidance amount for the moving body to avoid the object. The determination unit determines an avoidance trajectory for the moving body on the basis of the avoidance amount.

Description

Information processing device, information processing method, and information processing program

[0001] The embodiments relate to an information processing device, an information processing method, and an information processing program.

[0002] Many technologies have been studied for generating a trajectory to a target location, and generating an avoidance trajectory to avoid an object to be avoided, such as a person or an object that interferes with the trajectory. For example, there is a method of determining an object to be avoided and then placing relay points called waypoints, and a method of dividing the movement space into grids and calculating which grid the vehicle should pass through (see, for example, Non-Patent Documents 1 and 2).

[0003] Erke Shang et al., “An improved A-Star based path planning algorithm for autonomous land vehicles”, International Journal of Advanced Robotic Systems. 2020;17(5)Brono Brito et al., “Model Predictive Contouring Control for Collision Avoidance in Unstructured Dynamic Environments”, IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 4459-4466, Oct. 2019

[0004] However, in conventional avoidance trajectory generation, a fixed avoidance amount and movement speed are set as parameters. Therefore, for example, regardless of the state or role of the robot, or regardless of changes in the surrounding situation, the robot will perform avoidance movement only with the predetermined avoidance amount. Therefore, even in situations where robot movement should be prioritized, the robot will perform avoidance movement, resulting in a problem of poor movement efficiency.

[0005] The present invention has been made in light of the above circumstances, and its object is to provide an information processing device, an information processing method, and an information processing program that can appropriately and efficiently determine the movement route of a moving object.

[0006] An information processing device according to an embodiment includes a recognition unit, a generation unit, an estimation unit, and a determination unit. The recognition unit generates object information relating to objects present around a moving object and situation information relating to a situation in which the moving object is placed. The generation unit generates a prompt, which is a description of behavioral control of the moving object, based on moving object information relating to the moving object, the object information, and the situation information. The estimation unit inputs the prompt into a large-scale language model and estimates an amount of avoidance of the moving object to avoid the object. The determination unit determines an avoidance trajectory for the moving object based on the amount of avoidance.

[0007] According to the embodiments, it is possible to provide an information processing device, an information processing method, and an information processing program that can appropriately and efficiently determine a movement route of a moving object.

[0008] FIG. 1 is a block diagram illustrating an information processing device according to this embodiment. FIG. 2 is a flowchart illustrating an example of the operation of the information processing device according to this embodiment. FIG. 3 is a diagram illustrating an example of a list related to objects stored in an object information database. FIG. 4 is a diagram illustrating an example of mobile object information stored in a mobile object information database. FIG. 5 is a diagram illustrating a first example of a prompt generated by a description generation unit. FIG. 6 is a conceptual diagram illustrating an estimation result of an avoidance amount for the prompt shown in FIG. 5. FIG. 7 is a diagram illustrating a second example of a prompt generated by a description generation unit. FIG. 8 is a conceptual diagram illustrating an estimation result of an avoidance amount for the prompt shown in FIG. 7. FIG. 9 is a diagram illustrating a third example of a prompt generated by a description generation unit. FIG. 10 is a conceptual diagram illustrating an estimation result of an avoidance amount for the prompt shown in FIG. 9. FIG. 11 is a diagram illustrating a fourth example of a prompt generated by a description generation unit. FIG. 12 is a conceptual diagram illustrating an estimation result of an avoidance amount for the prompt shown in FIG. 11. FIG. 13 is a diagram illustrating a fifth example of a prompt generated by a description generation unit. FIG. 14 is a conceptual diagram illustrating an estimation result of an avoidance amount for the prompt shown in FIG. 13. FIG. 15 is a diagram illustrating a first example of a trajectory generation method performed by a trajectory determination unit. Fig. 16 is a diagram showing a second example of a trajectory generation method of the trajectory determination unit Fig. 17 is a block diagram showing the hardware configuration of an information processing device according to this embodiment.

[0009] Each embodiment will be described below with reference to the drawings. Each embodiment illustrates an apparatus or method for embodying the technical idea of ​​the invention. The drawings are schematic or conceptual. Hereinafter, the same reference numerals are used to designate components having substantially the same functions and configurations. The numbers following the letters that make up the reference numerals are used to distinguish between elements that are referred to by the reference numerals containing the same letters and that have similar configurations. When there is no need to distinguish between elements indicated by reference numerals containing the same letters or numbers, these elements will be referred to by reference numerals containing only letters or numbers.

[0010] An information processing device 1 according to this embodiment will be described with reference to the block diagram of FIG. 1 . The information processing device 1 controls the movement of a mobile object. The information processing device 1 may be mounted on the mobile object, or may be incorporated in a location remote from the mobile object, such as an external server, and may control the movement of the mobile object by wirelessly communicating data between the mobile object and the external server. The mobile object may be a self-propelled robot, a robot driven only by an arm such as an industrial robot, or an aerial vehicle such as a drone, as long as it is a device whose drive can be controlled electronically.

[0011] The information processing device 1 according to this embodiment includes an environment recognition unit 10, an object information database 11, a description generation unit 12, an estimation unit 13, an interference determination unit 14, an orbit determination unit 15, a moving body information database 16, an output unit 17, and a result storage unit 18.

[0012] The environment recognition unit 10 receives signals captured by an external camera and microphone, and generates target information about targets present around the mobile unit and situation information about the situation the mobile unit is in. Targets here include objects, people, and animals that may become obstacles to the movement of the mobile unit.

[0013] The object information database 11 stores the target information and situation information recognized by the environment recognition unit 10. The description generation unit 12 generates a prompt, which is a description regarding behavior control of the mobile object, based on the mobile object information, target information, and situation information regarding the mobile object.

[0014] The estimation unit 13 inputs the prompt into a large-scale language model and estimates the amount of avoidance of the mobile body to avoid the object. The interference determination unit 14 determines whether or not there will be interference with the object based on the amount of avoidance. The trajectory determination unit 15 determines the avoidance trajectory of the mobile body based on the amount of avoidance.

[0015] The mobile object information database 16 stores information about the mobile object itself. The output unit 17 outputs the avoidance trajectory to the robot. The result storage unit 18 stores the processing results such as the generated prompt, the amount of avoidance of the target, and the avoidance trajectory of the mobile object.

[0016] Next, an example of the operation of the information processing device 1 according to this embodiment will be described with reference to the flowchart in Fig. 2. Note that the following description will be given taking a self-propelled robot as an example of a moving object. Therefore, moving object information will also be referred to as robot information.

[0017] In step SA1, the environment recognition unit 10 acquires image information and audio information. For example, an RGB image and point cloud information are acquired as image information by using an RGB-D camera mounted on the robot. The point cloud information includes three-dimensional coordinate values ​​for each point. Image information acquired by an RGB-D camera mounted on the robot is not limited to image information acquired by the RGB-D camera, but may also be used in combination with image information acquired by a surveillance camera or the like that captures the space in which the robot is located. This allows for the use of multi-viewpoint video, such as information on points in the robot's blind spots, to obtain more detailed information. Audio information can be acquired by collecting ambient sound using a microphone.

[0018] In step SA2, the environment recognition unit 10 generates object information about objects around the robot and situation information about the situation in which the robot is placed. Specifically, based on image information, an object detection algorithm using a machine learning model such as AffordanceNet or the Yolo series (Yolo V1 to Yolo V8) may be used to recognize objects, people, etc., and generate the object information. Alternatively, using SlowFast or another algorithm, separate output results such as behavior recognition results indicating what the detected person is doing and facial orientation may be generated as situation information. Furthermore, a large-scale language model (LLM) may be used to generate the above object information and situation information together. The environment recognition unit 10 may also perform speech recognition processing on audio information to generate character strings that serve as the speech recognition results.

[0019] In step SA3, the object information database 11 stores the recognition results obtained by the environment recognition unit 10. That is, based on the target information and situation information, the object information database 11 stores the positions of recognized objects, the positions of people, the results of person behavior recognition, character strings of voice recognition results, etc. as lists. It is assumed that the information from step SA1 to step SA3 is executed at predetermined intervals.

[0020] In step SA4, the description generation unit 12 generates a prompt by inputting the target information stored in the object information database 11 and the mobile object information stored in the mobile object information database 16, i.e., information such as robot information, the robot's role, and the robot's activity environment, in accordance with a pre-created template. When generating the prompt, past prompts stored in the result storage unit 18 may be referenced and the prompt may be generated based on the past prompt. The prompt may be generated when a role is assigned to the robot, at a predetermined timing, or when a person or object is added as target information while the robot is traveling. The prompt may be automatically generated by the description generation unit 12 based on the target information and situation information, or may be input by the user.

[0021] In step SA5, the estimation unit 13 estimates the avoidance amount based on the prompt. As the avoidance amount, for example, an estimation result of an interference avoidance area, which is an area where a moving object should avoid passing, may be obtained by inputting the prompt into a large-scale language model represented by BRET, GPT-4, Claude3, etc. The interference avoidance area is estimated for each object in the list stored in the object information database 11, and the estimated interference avoidance area is added to the list in the object information database 11.

[0022] In step SA6, the interference determination unit 14 determines whether the robot will interfere with an object between the robot's current location and the destination. For example, based on the list stored in the object information database 11 and the robot information, a predetermined discriminant is used to determine whether the robot will interfere with the object when moving to the destination. The interference determination method may include, but is not limited to, a method of repeatedly bringing the robot's position and the position of the object to be avoided closer together and determining the interference position and time, or a method of determining the interference position and time using a numerical solution. If the robot will interfere with the object, the process proceeds to step SA7. On the other hand, if the robot will not interfere with the object, the interference determination unit 14 adds interference information (no interference) to the object list in the object information database 11, and the process proceeds to step SA8.

[0023] In step SA7, the collision detection unit 14 sets the object and person determined to collide in step SA6 as targets to be avoided. The collision detection unit 14 also stores the collision information (the occurrence of collision, the time of collision, and the collision position) in the list of targets to be avoided in the object information database 11 in association with the collision information.

[0024] In step SA8, the collision determination unit 14 determines whether collision has been determined for all objects stored in the object information database 11. If collision has been determined for all objects in the list, the process proceeds to step SA9, and if there are any objects that have not been determined, the process returns to step S6 and the same process is repeated.

[0025] In step SA9, the trajectory determination unit 15 generates an avoidance trajectory based on the estimated avoidance amount, taking into account the interference avoidance area of ​​the avoidance target. For example, if a method of setting a waypoint (relay point) outside the interference avoidance area is used to generate the avoidance trajectory, the robot's waypoint may be set at a position that is the sum of the radius of the interference avoidance area and the radius of the robot's interference area. ＊ In search algorithms, a trajectory can be generated by filling a grid from the center position of the avoidance target to the interference avoidance area as an obstacle. Similarly, in methods such as the potential method, a trajectory can be generated without entering the interference avoidance area by providing a gradient equal to the diameter of the interference avoidance area. In this way, any method can be used as long as it can generate an avoidance trajectory using an estimated avoidance amount.

[0026] In step SA10, the output unit 17 outputs, for example, to a robot, trajectory information regarding the avoidance trajectory generated by the trajectory determination unit 15. As a result, the robot that has received the trajectory information is controlled so as to move along the avoidance trajectory based on the trajectory information.

[0027] In step SA11, the result storage unit 18 stores the prompt generated in step SA4, the interference avoidance area and avoidance amount estimated in step SA5, and the avoidance trajectory generated in step SA9.

[0028] Next, an example of a list related to objects stored in the object information database 11 will be described with reference to Fig. 3. As shown in Fig. 3, a list in which an ID, object name, position, speed, speed vector, behavior recognition result, face direction, person attribute, state, interference information, interference avoidance area (avoidance amount), and voice recognition result are associated with each object is stored in the object information database 11. Note that it is not necessary to store one list for each object, and information may be stored in one table as long as each item for each object is associated with each other. Other information may also be stored in the list.

[0029] Next, an example of mobile object information (robot information) stored in the mobile object information database 16 will be described with reference to FIG. 4 . As shown in FIG. 4 , the robot information stores the robot's own position information, joint information, role and task content, robot size, default avoidance amount, and default speed. If the robot has multiple joints, the joint information includes information such as the length and range of motion of each arm. The role and task content is information about the robot's assigned role and task, i.e., how it should operate. The robot's size includes the three-dimensional dimensions (length, width, and depth) of the robot's body and hands. The default avoidance amount includes the avoidance amount of each of the robot's body and limbs other than the trunk, such as the robot's hands, arms, and legs, which are set as initial values. The default speed is the robot's movement speed, which is set as an initial value. Note that the robot's own position information and joint information data are continuously collected and updated as the robot operates. Values ​​for other items can be entered and changed at any time by voice, keyboard, or other input before the robot starts or during operation.

[0030] Next, a first example of a prompt generated by the description generation unit 12 will be described with reference to FIG. 5 . As shown in FIG. 5 , the prompt is written to instruct the robot on its role and work content, and to obtain the amount of avoidance (interference avoidance area) for the object and, if necessary, the avoidance speed. Furthermore, information about the object is extracted from the list in the object information database 11 and described in the prompt as avoidance target information. Specifically, a prompt such as "You are a robot working as a clerk in a shopping center. Please prioritize customers as you move. There is a chair on the way to the target position, so you need to avoid it. The default interference avoidance area is 1 m. Please consider how much interference you need to avoid based on the following information. Please also change the speed as necessary. [Avoidance target information] Object name: chair, Object position: [x, y, z], Object speed: 0.0 m / s, Object state: stationary..." is generated.

[0031] As shown in FIG. 5 , information about targets other than those specified in the prompt may be extracted from a list stored in the object information database 11 and described as supplemental information such as [Other Information]. That is, it is assumed that the first half of the prompt template describes the role played by the robot and the location where the robot works, as described in the robot information database, and the second half describes the contents of the object information database 11, such as the target to be avoided and information about other surrounding objects and people. However, the description is not limited to this, and may be one that can be similarly interpreted by a large-scale language model. Furthermore, the description is not limited to natural language, and may be written in a data description language such as XML (extensible markup language).

[0032] Next, the result of estimating the avoidance amount in response to the prompt shown in Fig. 5 is shown in the conceptual diagram of Fig. 6. Fig. 6 is a conceptual diagram showing the target avoidance amount 60 estimated by the estimation unit 13 based on the prompt shown in Fig. 5. Here, the robot R is to avoid a chair 61 placed as furniture, and it is assumed that the estimation unit 13 has estimated from the prompt shown in Fig. 5 that this is a normal environment and that normal avoidance is sufficient. Therefore, in accordance with the default avoidance amount, the avoidance amount 60 is estimated to be, for example, "interference avoidance area: 1 m, speed: 0.5 m / s".

[0033] Next, a second example of a prompt generated by the description generation unit 12 will be described with reference to FIG. 7 . The prompt in FIG. 7 is a prompt for avoiding a dangerous object. Specifically, the following prompt may be generated: "You are a robot working as an office worker. Move with the same sense as the people around you. There is a red plastic container on the way to the target position, which must be avoided. The default interference avoidance area is 1 meter. Consider how far you need to avoid it based on the following information. Change the speed as necessary. [Avoidance target information] Object name: red plastic container, object position: [x, y, z], object speed: 0.0 m / s, object state: stationary, standing, filled..."

[0034] Next, Fig. 8 shows the result of estimating the avoidance amount in response to the prompt shown in Fig. 7. Fig. 8 is a conceptual diagram showing an avoidance amount 70 of an object estimated by the estimation unit 13 based on the prompt shown in Fig. 7. In this case, the robot R is to avoid a plastic container 71, and since the object to be avoided is the plastic container 71, which is a dangerous object, based on the prompt shown in Fig. 7, it is estimated that the avoidance amount will be larger than the reference, which is the default avoidance amount (the avoidance trajectory indicated by the dashed line in Fig. 8 based on the avoidance amount 60). Therefore, for example, "interference avoidance area: 1.5 m, speed: 0.5 m / s" is obtained as the avoidance amount 70.

[0035] Next, a third example of a prompt generated by the description generation unit 12 will be described with reference to FIG. 9 . The prompt in FIG. 9 is a prompt assuming a case where a robot moves in an emergency. Specifically, suppose a prompt such as the following is generated: "You are a robot working as security at a shopping center. An urgent job has come in, so please generate an avoidance trajectory with yourself as your priority. There is a person standing before the target position, so you need to avoid the person. The default interference avoidance area is 1 m. Please consider how far you need to avoid the person based on the following information. Please also change the speed as necessary. [Avoidance target information], object name: person, person position: [x, y, z], person speed: 0.5 m / s, person behavior recognition result: walking, person attribute: customer, face direction: recognizes the robot..."

[0036] Next, FIG. 10 shows the estimated avoidance amount for the prompt shown in FIG. 9 . FIG. 10 is a conceptual diagram illustrating the target avoidance amount 80 estimated by the estimation unit 13 based on the prompt shown in FIG. 9 . In this example, the robot R's role is a security robot, and the robot has priority in the given situation. Furthermore, although the robot R needs to avoid a person 81, because the other party recognizes the robot, it is estimated that the person 81 will be far away and the robot R will only need to perform minimal avoidance. Therefore, the avoidance amount 80 is estimated to be smaller than the default avoidance amount (the avoidance trajectory indicated by the dashed line in FIG. 10 based on the avoidance amount 60). Furthermore, the parameters are changed to move faster than the default speed. Thus, for example, the avoidance amount 80 is obtained as "interference avoidance area: 0.3 m, speed: 0.7 m / s." Note that because movement based on this avoidance amount 80 is an emergency, a sound such as a siren and a light such as a warning light may be output along with the robot's movement to notify those around the robot that the robot is moving in an emergency.

[0037] Next, a fourth example of a prompt generated by the description generation unit 12 will be described with reference to FIG. 11. The prompt shown in FIG. 11 is a prompt assuming a case where a robot adjusts its movement parameters depending on the situation. Specifically, a prompt such as "You are a robot working as a security guard at a shopping center. An urgent job has come up, so please generate an avoidance trajectory that gives priority to yourself. There was a voice about the robot. Please adjust the parameters based on the following information. Voice: "Isn't that robot getting too close to the wall? It's scary" is generated.

[0038] Next, the estimation result of the avoidance amount in response to the prompt shown in FIG. 11 is shown in FIG. 12. FIG. 12 is a conceptual diagram showing the target avoidance amount 80 estimated by the estimation unit 13 based on the prompt shown in FIG. 11. Here, the robot changes parameters based on the content of the voice including instructions on the robot's movement state. The pre-set no-entry area 91 from the wall, "50 cm," is changed to the no-entry area 92, "80 cm." By generating parameter change values ​​flexibly in this way, it becomes possible to adjust the parameters.

[0039] Note that the examples in Figures 5 to 12 show examples in which the robot moves on its own, but this is not limiting. For example, the avoidance amount may be calculated in the case of an industrial robot or the like that moves within a relatively small predetermined area.

[0040] Next, a fifth example of a prompt generated by the description generation unit 12, which assumes trajectory generation for an industrial robot, will be described with reference to Fig. 13. The prompt shown in Fig. 13 generates a prompt such as "You are a robot working as an office worker. Generate a grip trajectory with the same feeling as the people around you. There is a powered-on soldering iron between the target position and you need to avoid it. The default interference avoidance area is 0.15 m. Consider how far you need to avoid it based on the information below, and change the speed as necessary. [Avoidance target information] Object name: soldering iron, object position: [x, y, z], object speed: 0.0 m / s, object state: powered on..."

[0041] Next, Fig. 14 shows the estimation result of the avoidance amount for the prompt shown in Fig. 13. Fig. 12 is a conceptual diagram showing the avoidance amount 1000 of the object estimated by the estimation unit 13 based on the prompt shown in Fig. 11. In this case, since the soldering iron that is turned on is also a dangerous object when the robot arm 1001 moves, it is estimated that a large amount of avoidance is required when generating the trajectory of the robot arm 1001. Therefore, for example, "interference avoidance area: 0.25 m, speed: 0.3 m / s" is obtained as the avoidance amount 1000.

[0042] Next, an example of a method for calculating interference information by the interference determination unit 14 will be described. Here, a method for determining whether or not there is interference using a numerical solution will be shown as an example. The position and velocity vectors of the robot and the object are used for interference determination. The environment recognition unit 10 arranges object information and situation information that are added as needed in chronological order, and calculates the velocity vector for each object and person from the difference between the three-dimensional coordinate information of the latest position information and the three-dimensional coordinate information included in the position information obtained in the previous process. The positions and velocities of object A and robot R at a certain time t [seconds] are respectively expressed as x ｔ Ａ→ , v ｔ Ａ→ , x ｔ Ｒ→ , v ｔ Ｒ→ Here, the superscript arrow " → " indicates that the value is a vector. In this embodiment, in order to handle the three-dimensional movement of the robot R, the vectors of the three axes are expressed as x １ , x ２ , x ３ This is expressed as equation (1).

[0043]

[0044] Next, the relative position x between the object A and the robot R ｔ （Ａ－Ｒ）→ , and the relative velocity between object A and robot R is v ｔ （Ａ－Ｒ）→ Let the current time be t ０ [seconds], the relative position x at any time t [seconds] ｔ （Ａ－Ｒ）→can be expressed by equation (2).

[0045]

[0046] By solving equation (2), equation (3) is obtained.

[0047]

[0048] Therefore, the relative position x ｔ （Ａ－Ｒ）→ is compared with the sum of the radius of the interference area included in the object information and the radius of the interference area of ​​the hand included in the robot information, and interference is determined. If the value of the square root of equation (3) is positive, it is determined that interference occurs, and if it is negative, it is determined that no interference occurs. If interference is determined, the interference time t １ In addition, since the calculation is based on the assumption that the robot R moves at a constant speed in a straight line, the current position, relative speed, and collision time t １ From the above, the interference position x ｔ１ Ｒ→ The interference determination unit 14 performs the above-mentioned calculation for determining interference, and calculates the presence or absence of the calculated interference and the interference time t １ and interference position x ｔ１ Ｒ→ The information relating to the above is treated as interference information, and the interference information is added to the list in the object information database 11. The interference determination unit 14 executes interference determination for all objects listed in the object information database 11.

[0049] Next, a first example of a trajectory generation method of the trajectory determination unit 15 will be described with reference to Fig. 15. In the first example shown in Fig. 15, a waypoint 150 is set as an avoidance trajectory, and the robot R avoids the target OBJ by moving via the waypoint.

[0050] The trajectory determination unit 15 connects the current position of the robot R to the destination point with a straight line as the shortest path, and calculates the collision time t １ Position x of the target OBJ ｔ１ ＯＢＪ→ , the waypoint 150 is set in a direction perpendicular to the shortest route. ｔ１ＯＢＪ→ The interference avoidance area IA of the target OBJ calculated by the estimation unit 13 is calculated in a direction perpendicular to the shortest path from Ｏ and the interference avoidance area IA of the robot R Ｒ The position corresponding to the sum of the radius of the robot R and the radius of the robot R is set as Waypoint 150. Ｒ The default avoidance amount of the robot R stored in the mobile body information database 16 may be used.

[0051] Next, a second example of the trajectory generation method of the trajectory determination unit 15 will be described with reference to Fig. 16. In the second example shown in Fig. 16, the positional relationship between the target OBJ and the robot R is arranged on a grid. The trajectory determination unit 15 calculates the interference avoidance area IA of the target OBJ. Ｏ and the interference avoidance area IA of robot R Ｒ The trajectory determination unit 15 searches the grid so that the shortest avoidance trajectory 160 that the robot R can use to reach the destination does not overlap with the interference avoidance area IA. In the example of FIG. 16, the shortest avoidance trajectory 160 that has been searched for is represented by a diagonally shaded block. Specifically, the trajectory determination unit 15 Ｏ and interference avoidance area IA Ｒ For example, the RRT (Rapidly-exploring Random Tree) method, A ＊ algorithm, Dijkstra algorithm, RRT ＊ In this case, when the robot R moves three squares on the grid toward the destination, the next square is the interference avoidance area IA of the target OBJ. Ｏ and the interference avoidance area IA of robot R Ｒ Since these overlap, a route is searched for to avoid them to the right and reach the destination via the shortest route.

[0052] Next, an example of the hardware configuration of the information processing device 1 according to this embodiment will be described with reference to the block diagram shown in Fig. 17. As shown in Fig. 17, the information processing device 1 includes, for example, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a display 104, a communication interface 105, and a storage 106.

[0053] The CPU 101 is an integrated circuit capable of executing various programs and controls the overall operation of the information processing device 1. The ROM 102 is, for example, a non-volatile semiconductor memory and stores programs and control data for controlling the information processing device 1. The RAM 103 is, for example, a volatile semiconductor memory and is used as a work area for the CPU 101. The display 104 is, for example, a liquid crystal display or a touch panel display and displays information. The communication interface 105 is, for example, an input / output component for exchanging information with the outside, such as a USB, an HDMI (registered trademark), or a network interface. The storage 106 is a non-volatile storage device. The storage 106 stores system software and the like for the information processing device 1. The CPU 101 may also be called a "processor."

[0054] In the above embodiments, the CPU 101 of the information processing device 1 may be another circuit (or processor). For example, the information processing device 1 may include a GPU (Graphics Processing Unit), an NPU (Neural Network Processing Unit), an MPU (Micro Processing Unit), or the like instead of a CPU. Each of the processes described in each embodiment may be realized by dedicated hardware. The processes of the information processing device 1 may be a mixture of processes executed by software and processes executed by hardware, or may be only one of them.

[0055] According to the present embodiment described above, the environment recognition unit generates object information and situation information. The description generation unit references the object information, situation information, and information related to the moving object to generate a prompt including the role of the moving object, etc. The estimation unit inputs the prompt into a large-scale language model and estimates the amount of avoidance of the moving object. The determination unit determines the avoidance trajectory of the moving object based on the amount of avoidance.

[0056] This allows the user to flexibly determine and change the amount of avoidance based on the role required of the user, the situation, and the target of avoidance.In other words, the user can determine in real time whether to make a large avoidance move or whether it is okay to move straight ahead, and the vehicle's path can be determined appropriately and efficiently.

[0057] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0058] DESCRIPTION OF SYMBOLS 1... Information processing device 10... Environment recognition unit 11... Object information database 12... Description generation unit 13... Estimation unit 14... Interference determination unit 15... Trajectory determination unit 16... Mobile body information database 17... Output unit 18... Result storage unit 60, 70, 80, 1000... Avoidance amount 61... Chair 71... Polyethylene tank 81... Person 90... Wall 91, 92... No entry area 101... CPU 102... ROM 103... RAM 104... Display 105... Communication interface 106... Storage 150... Waypoint 1001... Robot arm R... Robot OBJ... Object IA Ｒ , I.A. Ｏ …Interference avoidance area

Claims

1. An information processing device comprising: a recognition unit that generates object information regarding objects existing around a moving object and situation information regarding a situation in which the moving object is placed; a generation unit that generates a prompt, which is a description regarding behavioral control of the moving object, based on moving object information regarding the moving object, the object information, and the situation information; an estimation unit that inputs the prompt into a large-scale language model and estimates an amount of avoidance of the moving object to avoid the object; and a determination unit that determines an avoidance trajectory of the moving object based on the amount of avoidance.

2. The information processing device according to claim 1, wherein the estimation unit estimates the amount of avoidance to be greater than a reference amount when the target is a dangerous object.

3. The information processing device according to claim 1, wherein the estimation unit estimates the amount of avoidance to be smaller than a reference amount when the moving body is in an emergency and needs to move and the target is a person.

4. The information processing device according to claim 1, wherein the estimation unit further estimates a moving speed of the moving object.

5. The information processing device described in claim 1, wherein the generation unit generates a prompt including an indication regarding the moving state of the moving object, and the estimation unit generates a change value for the parameter of the moving object by inputting the prompt including the indication into the large-scale language model.

6. An information processing method comprising: generating object information relating to objects present around a moving body and situation information relating to the situation in which the moving body is placed; generating a prompt that is a description of behavioral control of the moving body based on the moving body information, the object information and the situation information relating to the moving body; inputting the prompt into a large-scale language model; estimating the amount of avoidance of the moving body to avoid the object; and determining an avoidance trajectory of the moving body based on the amount of avoidance.

7. An information processing program for causing a computer to function as each part of the information processing device according to claim 1.

Images