Information conversion system, information conversion method, and program product

Abstract

The present disclosure provides an information conversion system, an information conversion method, and a program product. In the information conversion system according to the present disclosure, an input attribute that is an attribute of a path point described in a natural language is input to a learned model in which a large language model (LLM) is incorporated, with respect to a path point that is a candidate for a passing point of an autonomous mobile robot traveling within a facility. In the information conversion system, the learned model recognizes attribute information indicated by the input attribute among a plurality of kinds of attribute information that represents attributes of the path point, and outputs an action of the autonomous mobile robot from the path point to a path point adjacent to the path point, corresponding to the recognized attribute information.

Description

Technical Field

[0001] This disclosure relates to an information conversion system, an information conversion method, and a program product. Background Technology

[0002] Japanese Patent Application Publication No. 2023-064983 discloses a technique for setting waypoints based on user instructions in a virtual space of a virtual model of a facility to be traversed by an autonomous mobile vehicle. In this technique, when generating the traversal route of the autonomous mobile vehicle, instructions representing the work or processing content to be performed by the autonomous mobile vehicle are assigned to each waypoint to be traversed. Summary of the Invention

[0003] The goal is to develop a technique that allows for the specification of actions at a pathpoint even when the pathpoint's attributes are described and input in a manner that varies according to each user. However, the technique described in Japanese Patent Application Publication No. 2023-064983 does not enable such specification.

[0004] In the information conversion system involved in this disclosure, For candidate waypoints that are locations traversed by autonomous mobile robots operating within the facility, input attributes, described in natural language, of these waypoints are input into a learned model loaded with a large-scale language model. The learned model identifies the attribute information shown by the input attribute among multiple types of attribute information representing the attributes of the path point, and outputs the action of the autonomous mobile robot corresponding to the identified attribute information, from the path point to the path point adjacent to the path point.

[0005] In the information conversion methods disclosed herein, The computer performs the following operations: For candidate waypoints that are locations traversed by autonomous mobile robots operating within the facility, input attributes, described in natural language, of these waypoints are input into a learned model loaded with a large-scale language model. The learned model identifies the attribute information shown by the input attribute among multiple types of attribute information representing the attributes of the path point, and obtains the action of the autonomous mobile robot corresponding to the identified attribute information, from the path point to the path point adjacent to the path point.

[0006] The program products involved in this disclosure include programs, wherein, The program causes the computer to perform the following processes: For candidate waypoints that are locations traversed by autonomous mobile robots operating within the facility, input attributes, described in natural language, of these waypoints are input into a learned model loaded with a large-scale language model. The learned model identifies the attribute information shown by the input attribute among multiple types of attribute information representing the attributes of the path point, and obtains the action of the autonomous mobile robot corresponding to the identified attribute information, from the path point to the path point adjacent to the path point.

[0007] According to this disclosure, even when the attributes of a waypoint are described and input in a manner that varies according to each user, it is possible to specify the action at that waypoint. Attached Figure Description

[0008] The features, advantages, technical and industrial significance of representative embodiments of the present invention will be depicted in the following drawings for reference, wherein like symbols denote like elements.

[0009] Figure 1 This is a schematic diagram illustrating an example structure of a management system that uses map data generated in the information conversion system involved in the implementation.

[0010] Figure 2 This is a block diagram illustrating a structural example of an information conversion system involved in an implementation method.

[0011] Figure 3 This is a diagram illustrating an example of a table showing the correspondence between pathpoint attributes and actions.

[0012] Figure 4 To indicate to Figure 2 A schematic diagram of an example of single-level map data of a facility input into an information conversion system.

[0013] Figure 5 To indicate in Figure 2 Information conversion system to Figure 4 An example illustration of waypoints configured with accompanying actions in the map data.

[0014] Figure 6 This is a flowchart illustrating an example of the information conversion method involved in the implementation. Detailed Implementation

[0015] Although the invention is described below by way of embodiments, the invention as defined in the claims is not limited to the embodiments described below. Furthermore, the structures described in the embodiments are not necessarily necessary as means to solve the problem.

[0016] Implementation The information conversion system (hereinafter, this system) according to this embodiment is a system that outputs actions related to waypoints set in map data. In this system, it is also possible to output corrected map data, which is map data after the actions have been set at the waypoints, for the map data. That is, this system can be incorporated into a map correction system that corrects map data. This corrected map data can be used, for example, in a management system for managing autonomous moving bodies, or in other words, autonomous mobile robots, as described below.

[0017] Example of the general structure of a management system Figure 1 This is a schematic diagram illustrating the structure of management system 1. Management system 1 includes a management device 100, robots 200, cameras 500, a network 600, user terminals 400, and auxiliary units 700. Management system 1 is a system for managing one or more robots 200. The management device 100 manages the movement and tasks of the multiple robots 200.

[0018] Robot 200 is an autonomous mobile body that performs tasks such as transport. Robot 200 moves autonomously within medical and welfare facilities such as hospitals, rehabilitation centers, nursing facilities, and elderly care facilities. Robot 200 is used to transport medicines, medical equipment, meals, tableware, medical records, spare supplies, specimens, linen products, and people. The transported items can also be patients. Furthermore, the management system 1 can also be used in commercial facilities such as shopping malls. Robot 200 has wheels, a chassis, motors, sensors, batteries, and controllers. At least one of Robot 200 can be a different type of robot. Robot 200 can also all be of the same type. Each of Robot 200 is assigned a unique identification number (ID). Although in Figure 1 The middle image shows three robots 200, but the number of robots is not specifically limited; it only needs to be one or more.

[0019] Furthermore, at least one of the robots 200 can also perform tasks other than the delivery task. These other tasks include cleaning, security, and guidance. The robot 200 can perform multiple tasks such as cleaning, security, and guidance using the auxiliary unit 700, or it can perform tasks independently. The robot 200 can perform various tasks, for example, by combining the auxiliary unit 700 with itself. The robot 200 can also prepare different auxiliary units depending on the task. By replacing the auxiliary unit 700, the robot 200 can become a multi-tasking robot capable of performing multiple tasks.

[0020] When performing a transport task, the auxiliary unit 700 becomes a wheeled trolley or delivery vehicle for carrying the transported items. When performing a cleaning task, the auxiliary unit 700 differs from the illustrated case but includes a vacuum cleaner for sucking up trash and other debris. When performing a security task, the auxiliary unit 700 differs from the illustrated case but includes sensors such as LiDAR (registered trademark, hereinafter the same), and a camera. The following description primarily focuses on the scenario where robot 200 performs a transport task.

[0021] User U1 or User U2 can use User Terminal 400 to perform tasks such as entrusting the delivery of goods. For example, User Terminal 400 is a tablet computer or a smartphone. User Terminal 400 only needs to be an information processing device capable of wireless or wired communication.

[0022] Robot 200 and user terminal 400 are connected to management device 100 via network 600. Network 600 is a wired or wireless Local Area Network (LAN) or Wide Area Network (WAN). Management device 100 is connected to network 600 via wired or wireless means. Communication between devices can be achieved, for example, using common communication standards such as Wi-Fi (registered trademark).

[0023] Various signals sent from user terminals 400 of users U1 and U2 are transmitted to management device 100 via network 600 and then forwarded from management device 100 to robot 200. Similarly, various signals sent from robot 200 are transmitted to management device 100 via network 600 and then forwarded from management device 100 to user terminals 400. Management device 100 is a server connected to each device, which collects data from each device. Furthermore, management device 100 is not limited to a single physical device and may have multiple devices implementing distributed processing. In addition, management device 100 may also be configured to be distributed among edge devices such as robot 200. For example, part or all of management system 1 may be mounted on robot 200.

[0024] The robot 200 includes a drive motor, wheels, and a battery. It also includes a camera, LiDAR sensors, and a processing unit. The robot 200 infers its own position based on sensor detection results. Based on its position, the robot 200 autonomously moves along a route on a map from its starting point to its destination. The starting point becomes the robot's current position, and the destination becomes the target for transporting the goods. Alternatively, the source of the goods can be used as a route point for pathfinding. Furthermore, the route search can be performed either by the management device 100 or by the robot 200 itself.

[0025] User terminal 400 and robot 200 can also transmit and receive signals without going through management device 100. For example, user terminal 400 and robot 200 can also directly transmit and receive signals via wireless communication. Furthermore, management device 100 can also collect data from camera 500. Camera 500 is a surveillance camera or a burglar alarm camera, etc. Additionally, management device 100 can also collect data from communication devices or sensors (not shown).

[0026] The facility is designed to utilize multiple types of robots 200. A management device 100 assigns tasks to each robot 200. Each robot 200 can also mount an auxiliary unit 700 corresponding to its assigned task and perform that task. The task performed by each of the three robots 200 can be input by user U1 or user U2, or it can be pre-scheduled. For example, user U1 or others can delegate tasks by operating a user terminal 400. User U1 or others can input the type of task to be performed. User U1 or others can also input the area and time period for the task. The management device 100 creates a schedule for the robots 200 to perform tasks efficiently.

[0027] Users U1 or U2 can also operate the user terminal 400 to delegate delivery tasks. In this case, users U1 or U2 input information related to the delivery item. Furthermore, users U1 or U2 can also input predetermined arrival information indicating the scheduled arrival time of the delivery item. The management device 100 allocates robots to perform delivery tasks based on the predetermined arrival information. Moreover, the management device 100 sends control signals for the robots to perform tasks. These control signals may include delivery item information such as the route to the destination and the delivery item itself.

[0028] In such an overall structure, the management system 1 can be constructed as a whole by distributing the various elements of the management system 1 among the robot 200, the user terminal 400, and the management device 100. Furthermore, it can also be constructed by concentrating the essential elements for transporting the goods into a single device.

[0029] The management device 100 includes a server computer and the like to perform calculations for controlling and managing the robots 200. The management device 100 can be installed, for example, as a device equipped with a Central Processing Unit (CPU) or similar processor and memory, capable of executing programs. Furthermore, the functions described later can also be implemented through programs. The management device 100 manages each robot 200 based on internally stored map data, the transport object ID, and the robot ID of the robot 200. Since this map data refers to the revised map data 12c described later, it will be referred to as revised map data below.

[0030] For example, the management device 100 manages the schedules of multiple robots 200 in a way that enables the robot 200 to perform tasks efficiently. For example, when the management device 100 receives a task assignment from a user terminal 400, it selects one robot 200 from the multiple robots 200 and issues instructions to the robot 200 to perform the task. Alternatively, the management device 100 instructs the robot 200 to use an auxiliary unit 700.

[0031] The modified map data used in management system 1 includes, for example, waypoints that are associated with locations or areas on the map. Attributes can be set for these waypoints. For the modified map data, or at least a portion of the waypoints, actions to be taken by the moving vehicle are set based on the waypoints or their attributes; however, passage classification information that sets the passage or priority of the moving vehicle can also be set.

[0032] Examples of waypoints and actions will be described later.

[0033] The revised map data is data representing a floor map of the facility (also simply called a map). This revised map data may also include information related to driving restriction areas and their associated attributes, in addition to waypoint-related information. Furthermore, the revised map data may not be a floor map of the entire facility, but rather data representing a localized area where services are to be performed. In this case, it is sufficient to ensure that the original map data also contains data representing that localized area. Each robot 200 autonomously travels to its destination by referring to the revised map data.

[0034] The corrected map data is data that has been corrected using a map correction system incorporated into this system. The original map data may be generated based on building drawings of a facility, image data obtained from cameras installed inside the facility, distance sensor measurement data, or other building data, or based on a combination of several of these. Furthermore, the corrected map data may be in the same format as the original map data, but may also be converted to a different format.

[0035] The architectural drawing data mentioned above can be image data of architectural drawings obtained by reading paper media, or it can be CAD data, BIM data, or image data that has been converted into PDF (registered trademark. The same applies hereinafter). CAD is an abbreviation for Computer-Aided Design. BIM is an abbreviation for Building Information Modeling. PDF is an abbreviation for Portable Document Format. Architectural drawing data can be, for example, two-dimensional architectural drawing data, but it can also be three-dimensional architectural drawing data.

[0036] The ranging sensor measurement results described above are examples of image data obtained by a robot 200 equipped with the sensor, or other robots or personnel, moving inside a facility, i.e., inside a building. The ranging sensor can be, for example, a LiDAR sensor, a depth sensor, a stereo camera, radar, or a combination thereof. The method of integrating data from the various sensors included in the ranging sensor is not limited; it is sufficient to generate map data 12a based on the data from the various sensors and using existing technology. The uncorrected map data 12a is not limited to two-dimensional map data; it can also be three-dimensional map data.

[0037] The measurement result data, which forms the basis of the map data before correction, refers to point group data obtained by moving a mobile body equipped with a LiDAR sensor within the facility and measuring the area within the facility. This is either two-dimensional or three-dimensional point group data obtained using LiDAR. The mobile body can be an autonomous mobile body, such as a robot 200 that is managed in the management device 100, or other types of autonomous mobile bodies, or a mobile body that moves by manual pushing or remote operation. Furthermore, regardless of whether the mobile body is autonomous or manually propelled, as long as gears or sensors for detecting its direction are pre-installed on the tires of the mobile body, the movement distance and direction can be monitored, thus allowing for the determination of the actual position of each measured point. Additionally, the LiDAR sensor is, for example, a two-dimensional LiDAR sensor. In this case, the map data 12a is obtained as two-dimensional point group data, that is, as point group data on the measurement plane. However, as mentioned above, map data 12a can be three-dimensional map data, and thus, for example, it can be point group data in three-dimensional space obtained by a three-dimensional LiDAR sensor. Furthermore, the aforementioned moving body can be configured as a LiDAR sensor, and the sensor is of the same type as the LiDAR sensor possessed by the autonomous moving body such as the robot 200 used within the facility, with the same setup conditions as that autonomous moving robot. The same setup conditions could, for example, refer to the same height, the same direction, and the same number of sensors. Thus, it is possible to obtain corrected map data 12c, where map data 12a is consistent with the distance measurement data during use in the autonomous moving body like the robot 200.

[0038] Example of the system structure For a system like the one described above, use Figure 2 Let me explain. Figure 2 This is a block diagram illustrating an information conversion system 10, which is a structural example of this system.

[0039] The information conversion system 10 can be configured using a computer. For example, the information conversion system 10 can include an arithmetic processing unit 11 composed of a processor such as a CPU and memory, a storage unit 12 composed of a storage device, a communication unit 13 for communicating with external devices, an operation unit 14 for receiving user operations, and a display unit 15 for displaying information. The arithmetic processing unit 11 can execute programs to implement functions related to information conversion and map correction, as described later. The communication unit 13 has a communication interface. As user operations, the operation unit 14 accepts input operations such as input attributes, specifying map data 12a, and correction instruction operations, as described later. The information input operation and the correction instruction operation are equivalent to input request operations. Furthermore, the information conversion system 10 can also be configured as a distributed system where some functions are distributed across multiple devices.

[0040] The storage unit 12 stores the learned model 12b in a state accessible from the processing unit 11. The learned model 12b is an example of a learned model that contains a Large Language Model (LLM) 12ba. The LLM 12ba can also generate AI (Artificial Intelligence) such as Chat-GPT (registered trademark), LaMDA, and BERT, but is not limited to these, and the type of LLM 12ba is not limited. Furthermore, LaMDA is an abbreviation for Language Model for Dialogue Applications, and BERT is an abbreviation for Bidirectional Encoder Representations from Transformers. When the learned model 12b contains a machine learning model other than LLM 12ba, this model can, for example, be a model implementing post-processing of LLM 12ba, and its algorithm, etc., only needs to be able to work in conjunction with LLM 12ba to perform the requested processing.

[0041] In the information conversion system 10, the processing unit 11 inputs input attributes—which are described in natural language as attributes of the path points—to the learned model 12b loaded with LLM 12ba. Path points are candidate locations for the autonomous mobile robot to travel through the facility. Input attributes are information input as part or all of a request. Although they are input by the user using the operation unit 14, they can also be selected by the user from a variety of pre-prepared requests.

[0042] Input attributes can also be input by specifying one or more waypoints on map data 12a stored in storage unit 12. Alternatively, input attributes can be input by specifying each feature of the waypoint's location. Location features can be input, for example, by specifying structures other than the main structural components of the facility. Here, the main structural components can refer to, for example, walls, columns, floors, beams, roofs, stairs, etc., and refer to the main part of the facility's internal structure. Structures other than the main structural components refer to building materials and installations such as room doors and emergency doors. However, structures other than the main structural components are not limited to the structures shown, as long as they are predefined. In addition, structures are not limited to part of the facility's internal structure, and can also include objects that are installed after the facility's construction, that is, objects that are installed during the facility's operation. Such objects include, for example, furniture, handrails, cushions, etc.

[0043] Furthermore, the aforementioned natural language can encompass multiple individual languages. In other words, LLM12ba can handle multiple individual languages. Additionally, robot 200 will be used as an example of an autonomous mobile robot for illustration below.

[0044] Next, in the information conversion system 10, the processing unit 11 identifies the attribute information shown by the input attribute among multiple types of attribute information representing the path point by executing the operations in the learned model 12b, mainly in LLM 12ba. LLM 12ba only needs to learn natural language related to various types of attribute information.

[0045] Furthermore, in the information conversion system 10, the arithmetic processing unit 11 executes the calculations in the subsequent stage of the learned model 12b, primarily LLM 12ba, to output the actions of the robot 200 corresponding to the identified attribute information, from a path point to a path point adjacent to that path point. Of course, the adjacent path points are also candidates for the robot 200's transit points. Here, when each path point is set in the map data 12a, in order to assign attribute information to each path point, the attribute information can be output in advance by establishing an association with each path point, or it can be output simply by establishing an association between the actions and each path point. Also, although for convenience the calculations in the learned model 12b are divided into calculations in LLM 12ba and its subsequent stage calculations, these calculations can certainly be performed simultaneously.

[0046] Specifically, the computation processing unit 11 can also output the robot 200's movements from a path point to the next path point on the travel path, corresponding to the attribute information identified as input attributes, by executing the calculations in the learned model 12b. In this example, it is advantageous to output movements that match the travel path when the robot 200's travel path is known in advance. In this case, different movements can be set even for the same path points when the travel paths are different.

[0047] Here, use Figure 3 This section will illustrate an example of the correspondence between the attribute information of a pathpoint (hereinafter referred to as WP attribute) and the action. Figure 3 This is a diagram illustrating an example of a table showing the correspondence between WP attributes and actions.

[0048] Actions are output based on the types of attribute information identified as WP attributes. Waypoints can be considered the locations that determine the actions of robot 200. Figure 3 Table 30 shows the WP attributes related to a path point and their corresponding actions taken at that path point during application in the management system 1. To allow reference to the learned model 12b, a table like Table 30 can be stored in advance either by implicitly including the learned model 12b in the storage unit 12 or as additional information. As shown in Table 30, WP attributes can include, for example, general path point, charger, return point, door, EV inside, EV elevator, EV elevator exit, delivery truck loading, delivery truck unloading, delivery truck queue waiting, right-of-way waiting, and right-of-way opening. Furthermore, a path point may correspond to two or more attributes. Specifically, in addition to general WP attributes, there may be one or more other attributes. In other words, attribute information can be considered as information representing attributes associated with one or more actions that can be taken as actions. That is, attribute information can be considered as information that can distinguish actions. Actions, for example, include not only simple continuation of movement, i.e., passing through, but also various other actions such as granting right-of-way, automatic door authentication, and elevator calling.

[0049] General waypoints are path points that represent locations along a movement route. These path points can be either start or end points. Candidate path points include, for example, locations near exits, narrow passages, or elevators, but are not limited to these. General waypoints serve as points along a route in a planned path. Robot 200 passes through general waypoints and moves autonomously toward the next path point. For example, after identifying each path point as a general waypoint, the learned model 12b can identify WP attributes other than those shown in Table 30, thereby enabling the setting of actions corresponding to the WP attributes through overwrite or append methods. WP attributes can also be attributes corresponding to the configuration and type of surrounding rooms; in this case, as input attributes, the user only needs to input the configuration and type of the surrounding rooms.

[0050] The following explains the setting of WP attributes and the adoption of actions. However, the setting of WP attributes refers to the recognition of WP attributes in the learned model 12b, and the actions that can be adopted refer to the output actions.

[0051] When a charger is set as a WP attribute, any of the following actions can be performed: charger connection, charger disconnection, or relative position correction relative to the charger achieved by recognizing a mark on the charger. For example, if the remaining battery level of robot 200 falls below a certain value, robot 200 moves towards the charger's waypoint. When robot 200 reaches the charger's waypoint, it performs relative position correction. For example, a mark is attached to the charger, and relative position correction is performed by photographing the mark using robot 200's camera. Then, robot 200 connects to the charger to begin charging. When charging is complete, robot 200 disconnects from the charger. The following WP attributes will be explained more concisely.

[0052] When a return point is set as a WP attribute, the robot 200 can identify its own position using markers recorded at various locations within the facility. When a door is set as a WP attribute, the robot can open the automatic door. When an elevator (EV) is set as a WP attribute, the robot can switch the destination floor map to the elevator (EV) map. When an elevator (EV) is set as a WP attribute, the robot can call the EV car, detect people and obstacles inside the car, take the elevator, or make a sound indicating that the robot is taking the elevator or temporarily suspended. When an elevator (EV) is set as a WP attribute, the robot can descend the elevator or make a sound indicating that the robot is descending. When a delivery truck is set as a WP attribute, the robot can correct its relative position to the delivery truck by recognizing markers on the delivery truck, crawl under the delivery truck, or lift the delivery truck. When a delivery truck is set as a WP attribute for unloading, the actions that can be performed include obstacle detection at the delivery truck's placement location, movement to the unloading location, or descent of the delivery truck. When a delivery truck is set as a WP attribute for waiting in a queue, the action that can be performed is to wait until an entry permission is received from the management device 100 (which acts as a server) or the advance robot 200. When a right-of-way is set as a WP attribute for waiting, the action that can be performed is to request the right of way to the right-of-way area at a waypoint near the right-of-way area and wait until an entry permission is received from the server. When a right-of-way is set as a WP attribute for granting rights, the action that can be performed is to notify the server that the right-of-way area has been traversed.

[0053] When a request containing input attributes is input, the processing unit 11 inputs it into the LLM 12ba of the learned model 12b, thereby obtaining, as an output or a subsequent output, an action corresponding to the identified attribute information as illustrated in Table 30 from the LLM 12ba.

[0054] Furthermore, the processing unit 11 can also input the direction information representing the robot 200's travel direction as part of a request, or by reading pre-stored information, into the learned model 12b. Thus, the direction information does not necessarily need to be described in natural language; for example, it can simply be read once the robot 200's travel direction is determined. In this case, the learned model 12b can determine adjacent path points based on this direction information. In other words, it is sufficient to have the learned model 12b perform machine learning in advance in a manner capable of making such a decision.

[0055] Alternatively, the processing unit 11 may input path information representing the travel path in the robot 200 as part of a request, or by reading pre-stored information, into the learned model 12b. In this way, the path information may not be described in natural language; for example, it can simply be read once the travel path of the robot 200 is determined. In this case, the learned model 12b can determine adjacent path points based on the path information. In other words, it is only necessary to perform machine learning in advance on the learned model 12b in a manner capable of making such a decision.

[0056] The learned model 12b, by corresponding to either direction information or path information, can determine which path point matches an adjacent path point, i.e., which location the next path point will be, based on either direction information or path information. Furthermore, the learned model 12b can output different actions based on this determination. Thus, the learned model 12b can, for example, produce different actions on the merging and branching sides of a three-way intersection, and different actions when heading in a blocked direction or leaving a blocked area.

[0057] Next, an example will be described in which the information conversion system 10 further modifies the map data 12a stored in the storage unit 12 based on the actions obtained in this manner.

[0058] In the information conversion system 10, the processing unit 11 inputs map data 12a, which corresponds to the previously modified map data, into the learned model 12b. Hereinafter, the description assumes that the processing unit 11 reads the map data 12a stored in the storage unit 12 as the input object. Therefore, the processing unit 11 can obtain the map data 12a in advance from an external server or other device via the communication unit 13 and store it in the storage unit 12, or it can obtain the map data 12a from an external storage device via an interface not shown and store it in the storage unit 12. The storage unit 12 only needs to store the map data 12a in advance in a state accessible from the processing unit 11. Furthermore, the storage of the map data 12a can also be temporary. In this case, the input of the map data 12a essentially means obtaining the map data 12a from an external source.

[0059] To correct map data 12a, as an example of a machine learning model other than LLM12ba, the learned model 12b can include a Vision-Language Model (VLM). A VLM is a model that takes image data and language data as input and outputs at least one of them. A VLM can also generate AI for Chat-GPT (registered trademark), but is not limited to this, and the type of VLM is not limited. When the learned model 12b includes a machine learning model other than LLM12ba and VLM, this model can, for example, be configured as an intermediate or post-processing model implementing the VLM, and its algorithm only needs to be able to work collaboratively with LLM12ba and VLM to perform the requested processing.

[0060] As described above, the processing unit 11 of the information conversion system 10 not only inputs the input attributes into the learned model 12b, but also inputs the facility map data 12a into the learned model 12b.

[0061] The input map data 12a can contain information indicating that the area shown by the map data 12a will be divided into multiple segmented regions based on structures other than the main structure of the facility. Thus, the map data 12a can be either already segmented into multiple regions or data before segmentation. The learned model 12b only needs to be used with the input map data 12a to perform machine learning and be generated.

[0062] The learned model 12b sets a path point corresponding to the attribute information shown in the input attributes for at least one of the multiple segmented regions of the map data 12a, and sets an action corresponding to the attribute information at that path point. Alternatively, attribute information can be assigned to the path point, and the action can be pre-associated with that attribute information. The attribute of the path point can also be called the path point category.

[0063] The processing unit 11 performs the calculations in the learned model 12b in this manner, thereby outputting the corrected map data 12c, with path points and actions set, to the output target. Furthermore, the corrected map data 12c may differ in form from the map data 12a. For example, if the map data 12a is point cluster data, the corrected map data 12c may retain that form, or it may be line drawing data obtained by smoothing the point clusters.

[0064] To correct the output target of map data 12c, simply set it to storage unit 12. Additionally, Figure 2The diagram illustrates the state in which the corrected map data 12c is output and stored in the storage unit 12. The corrected map data 12c can also be temporarily stored in the storage unit 12 for external transmission. Furthermore, the processing unit 11 can also display the corrected map data 12c on the display unit 15, in which case the display unit 15 becomes the output target.

[0065] Next, use Figure 4 as well as Figure 5 The following describes a correction example of map data 12a in the information conversion system 10, namely, a generation example of corrected map data 12c. Figure 4 This is a schematic diagram illustrating an example of single-layer map data within a facility input into the information conversion system 10. Figure 5 To indicate in Figure 4 This is an example illustration of waypoints configured with accompanying actions within the map data. Figure 4 The map data shown is 1000a and Figure 5 In the corrected map data 1000c shown, the diagonal area indicated by region NR refers to the main structure or interior space, specifically the space or concrete structure where the movement of robot 200 is not anticipated. Furthermore, since the facility has multiple floors, only each floor needs to be processed and corrected map data 12c output; therefore, the following explanation focuses on only one floor.

[0066] Figure 4 Map data 1000a shown is map data representing a floor within a hospital as an example of a facility, and is an example of map data 12a. As shown in map data 1000a, in addition to staff workstations and rooms such as wards, this floor also contains structures such as elevators (EVs) and fixtures such as tables or shelves.

[0067] In the learned model 12b, for at least one segmented region among multiple segmented regions of map data 1000a input to the learned model 12b or the VLM contained therein, waypoints and actions are set based on attribute information shown by the input attributes. The segmented region is the area shown by map data 1000a divided into multiple regions based on structures other than the main structural components of the facility. Figure 5 In the corrected map data 1000c, the segmented regions obtained through segmentation are illustrated by different shaded lines. Segmentation can be performed either before inputting into the learned model 12b or by the learned model 12b itself. Waypoints can also be set at predetermined locations in each segmented region based on attribute information, such as the center of the region or the boundary with adjacent regions.

[0068] Here are examples of requests that include input attributes, such as requests to assign general waypoints to the four corners of a corridor or the pillars in the passageway, and requests to assign waypoints to EVs taking the elevator. Furthermore, the learned model 12b can identify locations near the elevator even when no waypoint features are specifically described for EVs taking the elevator.

[0069] Based on the request, the learned model 12b sets path points at the object's location for general path points and establishes associations for actions that are part of the movement path, as shown in Table 30 corresponding to EV elevator rides, such as cage (car) calls. Furthermore, the calculation processing unit 11 sets path points for such incidental actions on the map data 1000a and outputs the revised map data 1000c as a result to the output target.

[0070] In the output corrected map data 1000c, general path points are set at locations indicated by path points WP11, WP12, etc., and an action AC11 indicating movement between adjacent path points is set. Additionally, in Figure 5 In the map data 1000c, path points WP13 and WP14 are set near the elevator, and actions such as cage calling are set as actions for these actions.

[0071] Thus, the map data generated by correcting map data 1000a using map data 1000a and the requested input becomes the illustrated corrected map data 1000c. However, the output corrected map data 1000c can be either the data after removing the information of the segmented areas, or the data after deleting useless information such as redundant lines or redundant text such as employee workstations.

[0072] Here, the region segmentation used to annotate and represent segmented regions will be explained. Region segmentation can be performed using segmentation algorithms such as the Voronoi segmentation algorithm. That is, the segmented region can also be a region segmented by such a segmentation algorithm. In this case, the processing unit 11 performs segmentation into multiple segmented regions by also performing segmentation processing in the learned model 12b, or performs segmentation into multiple segmented regions in a segmentation algorithm set in the preceding stage of the VLM. Furthermore, the segmentation algorithm can also be a learning model obtained by using the results of region segmentation performed on various types of building data as teaching data for machine learning.

[0073] Here, we will briefly supplement the learning process for generating the learned model 12b containing the VLM. The VLM can be generated by processing image data and text data such as map data 12a in the same coordinate space using existing techniques and performing machine learning. Furthermore, the learned model 12b can be pre-generated, for example, by including an extraction model that extracts the main structural elements and other structures from the map data 12a. This extraction model can be implemented by machine learning in a manner that allows extraction of the main structural elements and other structures associated with the requested input attributes from the map data 12a, either after or before region segmentation, and extraction of path points corresponding to the attribute information shown in the input attributes. This extraction model can also include the aforementioned segmentation algorithm to be applicable to the map data 12a before region segmentation.

[0074] Furthermore, this extraction model can be pre-generated to accommodate extraction methods that use names or abbreviations related to the main structural components and other structures for segmented regions. In fact, as... Figure 4 As shown in map data 1000a, map data 12a, when it is architectural drawing data, sometimes contains text and symbols representing internal structures. Even if the existing architectural drawing data does not contain such text or symbols, when used as a dataset for learning, the text or symbols can be added as text data or pre-written as handwritten text. Furthermore, symbols refer to symbols representing, for example, elevators, fire extinguishers, fireproof roller shutters, etc. Therefore, in the learning process of this extraction model, machine learning can be implemented in a specific way by extracting predetermined text or symbols from architectural drawing data to identify structures, their extraction locations, and the types of segmented regions. Additionally, the text data included in the dataset for learning can be essentially only the text or symbols contained in the architectural drawing data, or it can be prepared separately as data describing the various shapes of each structure or the characteristics of the location of each structure. Furthermore, by extracting names or abbreviations, this information can be removed from the input map data 12a. Furthermore, by extracting the types of segmented regions, it is possible to determine the segmented regions to be set for path points corresponding to the attribute information shown in the input attributes.

[0075] Summary of the processing of this system This disclosure, as exemplified by the processing of the information conversion system 10, also includes a method for correcting and outputting the aforementioned map data using a computer. Although it may use... Figure 6 This explanation of the information conversion method is provided briefly, but it can be applied as various application examples illustrated in this system. Figure 6This is a flowchart illustrating an example of the information conversion method involved in this embodiment.

[0076] In this information conversion method, firstly, the computer exemplified in the information conversion system 10 inputs input attributes, which are attributes of path points, and map data 12a into the learned model 12b (S1). Next, as part of the calculations in the learned model 12b, the computer inputs the input attributes into LLM 12ba (S2). Then, the computer obtains the action of the robot 200 at the path point by executing the calculations in LLM 12ba and outputs the action to the next stage (S3). This output can be executed for each path point.

[0077] Next, by executing the subsequent operations of LLM12ba in the learned model 12b, the computer sets the path points and actions in the map data 12a (S4). Then, the computer obtains the map data 12a after the aforementioned settings from the learned model 12b, i.e., corrected map data 12c, and outputs it to the output target (S5), and the process ends. Although the operations in the learned model 12b are divided into operations in LLM12ba and its subsequent operations for convenience, they can certainly be performed simultaneously. Even if the inputs in S1 are not simultaneous, it is sufficient that the attributes are input before S2 and the map data 12a is input before S4.

[0078] Furthermore, this disclosure also includes a method of causing a computer to execute the processing shown in such an information conversion method. Additionally, some or all of the processing in the robot 200, management device 100, etc., described above can also be implemented as a program. These programs can be stored and supplied to a computer using a wide variety of non-transitory computer-readable media (storage media). Non-transitory computer-readable media include a wide variety of types of physical recording media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., floppy disks, magnetic tapes, hard disk drives), optomagnetic recording media (e.g., optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, RAMs (Random Access Memory)). Furthermore, programs can also be supplied to a computer using a wide variety of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. Temporary computer-readable media can deliver programs to a computer via wired communication paths such as electrical wires and optical fibers, or via wireless communication paths.

[0079] Effects of this implementation method According to this embodiment, even when the attributes of waypoints are described and input in a manner that varies according to each user's individual behavior, fluctuations in natural language expression and attributes expressed in multiple individual languages ​​can be identified and converted into actions corresponding to those attributes. In other words, according to this embodiment, even when the attributes of waypoints are described and input in a manner that varies according to each user's individual behavior, actions at those waypoints can be specified. Therefore, according to this embodiment, actions of autonomous mobile robots can be easily specified. Furthermore, according to this embodiment, waypoints for which actions specified in this manner are set can be automatically set in map data. In particular, while setting multiple waypoints in advance during the use of an autonomous mobile robot allows for more precise control, according to this embodiment, compared to setting multiple actions manually for waypoints in advance based on map data, the time and effort required for setting up the waypoints can be reduced, thereby improving the efficiency of the setting up process.

[0080] Other application examples Furthermore, the present invention is not limited to the above-described embodiments, but can be appropriately modified without departing from the spirit of the invention.

[0081] For example, the generated map data can also be used for purposes other than the movement of autonomous mobile bodies like robot 200. Although in Figures 3-5 This text assumes that the input map data and the output corrected map data are two-dimensional data, but as explained regarding map data, this data can also be three-dimensional. Regarding three-dimensional map data, attributes such as ceilings and floors can be set as waypoints. For example, attributes such as ceilings as vents, ceilings below a predetermined height, floors with guide lights, and whether the floor material is a predetermined material can also be set. Therefore, for example, based on the height of robot 200, at a waypoint where the attributes include a ceiling below a predetermined height, actions such as prohibiting passage can be set as actions. Furthermore, at a waypoint where the attributes include a floor with guide lights, actions such as prohibiting parking can be set as actions. Additionally, if it is a robot with an arm, waypoints and their attributes can also be used to control the arm's range of motion. Furthermore, three-dimensional map data can also be used in the operation of autonomous flying bodies that perform flight.

[0082] Furthermore, the learned model 12b, or the LLM12ba or VLM contained therein, can also be a model capable of relearning. For example, as an LLM12ba or VLM, the learned model 12b can include open-source machine learning models such as Retrieval-Augmented Generation (RaG). Thus, the processing unit 11 can update the learned model based on the latest database. This is because the database can be updated in a way that yields more accurate output.

Claims

1. An information conversion system, wherein, For candidate waypoints that are locations traversed by autonomous mobile robots operating within the facility, input attributes, described in natural language, of these waypoints are input into a learned model loaded with a large-scale language model. The learned model identifies the attribute information shown by the input attribute among multiple types of attribute information representing the attributes of the path point, and outputs the action of the autonomous mobile robot corresponding to the identified attribute information, from the path point to the path point adjacent to the path point.

2. The information conversion system as described in claim 1, wherein, The direction information representing the autonomous mobile robot's travel direction is input into the learned model, and the learned model determines the adjacent path points based on the direction information.

3. The information conversion system as described in claim 1, wherein, The path information representing the travel path of the autonomous mobile robot is input into the learned model, and the learned model determines the adjacent path points based on the path information.

4. An information conversion method, wherein, The computer performs the following operations: For candidate waypoints that are locations traversed by autonomous mobile robots operating within the facility, input attributes, described in natural language, of these waypoints are input into a learned model loaded with a large-scale language model. The learned model identifies the attribute information shown by the input attribute among multiple types of attribute information representing the attributes of the path point, and obtains the action of the autonomous mobile robot corresponding to the identified attribute information, from the path point to the path point adjacent to the path point.

5. A program product comprising a program, wherein, The program causes the computer to perform the following processes: For candidate waypoints that are locations traversed by autonomous mobile robots operating within the facility, input attributes, described in natural language, of these waypoints are input into a learned model loaded with a large-scale language model. The learned model identifies the attribute information shown by the input attribute among multiple types of attribute information representing the attributes of the path point, and obtains the action of the autonomous mobile robot corresponding to the identified attribute information, from the path point to the path point adjacent to the path point.

Images