Map generation system

Abstract

This system provides a map generation system that can easily generate map data containing desired information from image data of the internal structure of a building. [Solution] The map generation system 10 according to this disclosure inputs building data, which is image data including at least the internal structure of a building, and a request to a visual language model. The visual language model is a model that takes image data and language data as input and outputs at least one of the image data and language data. The request includes a request to generate first information, which is information about at least one of a part of the internal structure and an existing object included in the building data. The map generation system obtains output information output from the visual language model in response to the request and generates map data related to the internal structure of the building based on the obtained output information.

Description

Technical Field

[0005] ,

[0001] The present disclosure relates to a map generation system.

Background Art

[0002] Patent Document 1 describes a map conversion system that acquires a two-dimensional or three-dimensional map defining a movable path of a moving object. The map conversion system acquires a map by simulation using self-position estimation based on three-dimensional BIM data representing a space in which internal structures and attribute information of structures have been processed in advance and predetermined individual information corresponding to the type of the moving object. Note that BIM is an abbreviation for Building Information Modeling.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the technique described in Patent Document 1, there is a possibility that map information such as an area where an autonomous mobile object cannot move cannot be obtained. Further, in the technique described in Patent Document 1, information about detailed structures and objects not included in the data needs to be obtained by manual input or measurement while making rounds, which requires a huge amount of man-hours. Therefore, development of a technique for easily generating map data including desired information such as information that affects the movement of an autonomous mobile object from image data of the internal structure of a building is desired.

Means for Solving the Problems

[0005] The map generation system described herein inputs building data, which is image data including at least the internal structure of a building, and a request including a request to generate first information, which is information relating to at least one of a part of the internal structure and an existing object included in the building data, to a visual language model, which is a model that takes image data and language data as inputs and outputs at least one of image data and language data. The system then acquires output information output from the visual language model in response to the request and generates map data relating to the internal structure of the building based on the acquired output information. [Effects of the Invention]

[0006] According to this disclosure, map data containing desired information can be easily generated from image data of the internal structure of a building. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic diagram showing one example configuration of a management system that uses map data generated by the map generation system according to the embodiment. [Figure 2] This is a block diagram showing an example configuration of a map generation system according to an embodiment. [Figure 3] Figure 2 is a schematic diagram showing an example of map data for one floor within a facility that is input into the map generation system. [Figure 4] This figure shows an example of a table illustrating the correspondence between waypoint attributes and actions. [Figure 5] This is a schematic diagram showing an example of map data for one floor of a facility, output from the map generation system shown in Figure 2, using the map data in Figure 3 as input. [Figure 6] This is a flowchart illustrating an example of a map generation method according to an embodiment. [Modes for carrying out the invention]

[0008] The present invention will be described below through embodiments of the invention, but the invention as claimed is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential as means for solving the problem.

[0009] (Embodiment) The map data generated by the map generation system according to this embodiment (hereinafter referred to as "this system") can be used, for example, in a management system for managing autonomous mobile vehicles, as described below.

[0010] (Example of a general configuration of a management system) Figure 1 is a schematic diagram showing the configuration of the management system 1. The management system 1 comprises a management device 100, robots 200, cameras 500, a network 600, a user terminal 400, and an accessory unit 700. The management system 1 is a system for managing one or more robots 200. The management device 100 manages the movement and tasks of one or more robots 200.

[0011] Robot 200 is an autonomous mobile unit that performs tasks such as transport. Robot 200 moves autonomously within medical and welfare facilities such as hospitals, rehabilitation centers, nursing homes, and elderly care facilities. Robot 200 is used to transport pharmaceuticals, medical equipment, meals, tableware, medical records, supplies, specimens, linens, and people. The transported object may also be a person, such as a patient. Furthermore, management system 1 can also be used in commercial facilities such as shopping malls. Robot 200 has wheels, a chassis, motors, sensors, a battery, a controller, etc. At least one of the robots 200 is of a different type. All robots 200 may be of the same type. Each robot 200 is assigned a unique identification number (ID). Figure 1 shows three robots 200, but the number of robots is not particularly limited as long as there is one or more.

[0012] Furthermore, at least one of the robots 200 may perform tasks other than transport tasks. These other tasks may include cleaning, security, and guidance tasks. The robot 200 may perform multiple tasks such as cleaning, security, and guidance using the accessory units 700, or it may perform tasks independently. For example, the robot 200 can perform various tasks by using the accessory units 700 in combination with the robot 200. The robot 200 may be equipped with different accessory units depending on the task. By changing the accessory units 700, the robot 200 becomes a multitasking robot capable of performing multiple tasks.

[0013] For transport tasks, the auxiliary unit 700 is a wheeled cart or wagon for carrying the transported items. For cleaning tasks, the auxiliary unit 700, although different from the one shown in the illustration, has a vacuum cleaner for sucking up dirt and debris. For security tasks, the auxiliary unit 700, although different from the one shown in the illustration, has sensors such as LiDAR (registered trademark; the same applies hereinafter) and cameras. In the following description, the robot 200 will be the primary operator for transport tasks.

[0014] User U1 or User U2 can use the user terminal 400 to submit task requests, such as requests for the transport of goods. For example, the user terminal 400 may be a tablet computer or a smartphone. The user terminal 400 can be any information processing device capable of wireless or wired communication.

[0015] The robot 200 and the user terminal 400 are connected to the management device 100 via the network 600. The network 600 is a wired or wireless LAN (Local Area Network) or WAN (Wide Area Network). Furthermore, the management device 100 is connected to the network 600 by wired or wireless connection. Communication between each device can use a general-purpose communication standard such as Wi-Fi (registered trademark).

[0016] Various signals transmitted from the user terminals 400 of users U1 and U2 are first sent to the management device 100 via the network 600 and then transferred from the management device 100 to the target robot 200. Similarly, various signals transmitted from the robot 200 are first sent to the management device 100 via the network 600 and then transferred from the management device 100 to the target user terminal 400. The management device 100 is a server connected to each device and collects data from each device. Also, the management device 100 is not limited to a physically single device and may have a plurality of devices that perform distributed processing. Further, the management device 100 may be distributed and arranged in edge devices such as the robot 200. For example, part or all of the management system 1 may be installed on the robot 200.

[0017] The robot 200 has a drive motor, wheels, a battery, etc. Further, the robot 200 has sensors such as a camera and LiDAR, and an arithmetic processing unit such as a processor. The robot 200 estimates its own position based on the detection results of the sensors. The robot 200 autonomously moves along a route from the starting point to the destination on the map based on its own position. The starting point is the current position of the robot, and the destination is the destination of the transported item. Also, route search may be performed via the place where the transported item originated as a transit point. Note that the management device 100 may perform route search from the starting point to the destination, or the robot 200 may perform route search.

[0018] The user terminal 400 and the robot 200 may transmit and receive signals without going through the management device 100. For example, the user terminal 400 and the robot 200 may directly transmit and receive signals by wireless communication. Also, the management device 100 may collect data from the camera 500. The camera 500 is a surveillance camera, a security camera, etc. Further, the management device 100 may collect data from communication devices and sensors not shown in the figure.

[0019] In a facility, it is assumed that multiple types of robots 200 are being used. The management device 100 assigns tasks to each robot 200. Each robot 200 may carry an accessory unit 700 corresponding to the assigned task and execute the task. The tasks executed by each of the three robots 200 may be input by user U1 or user U2, or may be scheduled in advance. For example, user U1 or the like makes a task request by operating the user terminal 400. User U1 or the like can input the type of task to be executed. User U1 or the like may also input the area or time zone where the task is to be executed. The management device 100 creates a schedule for the robots 200 to efficiently execute tasks.

[0020] User U1 or user U2 may operate the user terminal 400 to request a transportation task. In this case, user U1 or user U2 inputs information regarding the item to be transported. Further, user U1 or user U2 may input arrival schedule information indicating the scheduled arrival of the item to be transported. The management device 100 assigns a robot to execute the transportation task based on the arrival schedule information. Then, the management device 100 transmits a control signal for the robot to execute the task. The control signal may include information such as the route to the destination and the item information indicating the item to be transported.

[0021] In such an overall configuration, each element of the management system 1 can be distributed among the robot 200, the user terminal 400, and the management device 100 to construct the management system 1 as a whole. Also, the substantial elements for realizing the transportation of the item to be transported can be gathered in one device for construction.

[0022] The management device 100 has a server computer or the like and performs calculations to control and manage the robots 200. The management device 100 can be implemented as a device capable of executing programs, such as a computer's central processing unit (CPU). The functions described later can also be realized by program. The management device 100 manages each robot 200 based on map data stored internally, the transported object ID of the transported object, and the robot ID of each robot 200.

[0023] For example, the management device 100 manages the schedules of multiple robots 200 so that the robots 200 can perform tasks efficiently. For example, when the management device 100 receives a task request from a user terminal 400 or the like, it selects one robot 200 from the multiple robots 200 and instructs the robot 200 to perform the task. Alternatively, the management device 100 instructs the robot 200 to use the auxiliary unit 700.

[0024] The map data used in the management system 1 may, for example, include waypoints associated with locations or areas on a map. Waypoints may also have attributes set. This map data, or waypoints, may have at least one of the following set: an action taken by a moving object, or traffic classification information that determines whether or not a moving object can pass or its priority, depending on the waypoint or its attributes.

[0025] The map data is a map showing the floor plan (or simply the map) of the facility. This map data may include information such as restricted areas and waypoints. Furthermore, the map data may not be a floor plan of the entire facility, but rather a map that partially includes the area where the service is to be performed. Each robot 200 autonomously navigates to its destination by referring to the map data.

[0026] Map data is data generated by this system. Map data may be generated based on building data such as architectural drawings of a facility, image data acquired by cameras installed inside the facility, and measurement result data from distance measuring sensors, or based on a combination of several of these. Architectural drawing data may be image data obtained by reading paper architectural drawings, or it may be CAD data, BIM data, or image data such as PDF (registered trademark; the same applies hereinafter) of such data. CAD is an abbreviation for Computer-Aided Design, and PDF is an abbreviation for Portable Document Format. Measurement result data from distance measuring sensors is an example of image data acquired by a sensor-equipped robot 200 or another robot or person moving inside a building. Distance measuring sensors can be, for example, LiDAR (registered trademark; the same applies hereinafter), depth sensors, stereo cameras, radar, or a combination of several of these. Map data is not limited to two-dimensional map data, but may also be three-dimensional map data. Measurement result data that forms the basis of map data refers to, for example, two-dimensional point cloud data or three-dimensional point cloud data acquired using LiDAR when the distance measuring sensor is LiDAR.

[0027] (Example of this system configuration) The system that generates the map data described above will be explained using Figure 2. Figure 2 is a block diagram showing a map generation system 10, which is one example of the configuration of this system.

[0028] The map generation system 10 can be composed of a computer and includes an arithmetic processing unit 11 consisting of a processor and memory, a storage unit 12 consisting of a storage device, a communication unit 13 for communicating with the outside world, an operation unit 14 for receiving user input, and a display unit 15 for displaying information. The communication unit 13 is equipped with a communication interface. The map generation system 10 can also be constructed as a distributed system in which some functions are distributed across multiple devices.

[0029] The memory unit 12 stores the learning model 12a in a state accessible from the arithmetic processing unit 11. The learning model 12a is an example of a trained model that has undergone machine learning, including at least a Vision-Language Model (VLM). The type of VLM is not specified. If the learning model 12a includes machine learning models other than VLM, those models can, for example, perform pre-processing, intermediate processing, or post-processing of the VLM, and their algorithms, etc., only need to be able to work in cooperation with the VLM to perform processing according to requests.

[0030] The map generation system 10 inputs the building data, which is image data including at least the internal structure of the building, along with the request, to the learning model 12a or the VLM included in the learning model 12a.

[0031] A VLM is a model that takes image data and language data as input and outputs at least one of the image data and language data. A VLM may be, but is not limited to, a generative AI (Artificial Intelligence) such as Chat-GPT (registered trademark). Building data may include, for example, the various architectural drawing data mentioned above, image data acquired by cameras installed inside the facility, or a combination of these, and may be referred to as building drawing data. Building data may also include measurement result data from distance measuring sensors.

[0032] A request is a request that can be received from the operation unit 14 and includes a request to generate first information, which is information about at least one of the internal structure and objects included in the building data. The first information may be, for example, a predetermined keyword or a predetermined icon. The request may also be a request that has been previously stored in the storage unit 12, in which case the request is read and input to the VLM. More specifically, a request may be as follows: That is, read the internal structure and objects of the building included in the simultaneously input building data, extract those corresponding to walls, passages, passages of a predetermined width or less, stairs, elevators, and installed objects, and output map data with waypoints of the corresponding attributes set therein. This request may also include the definition of waypoints, and may also request the setting of waypoints that do not fall under these categories. Alternatively, the request may include the definition of waypoints and simply request the setting of general waypoints.

[0033] The calculation processing unit 11 of the map generation system 10 acquires input building data and output information output from VLM in response to a request, and generates map data related to the internal structure of the building based on the acquired output information. The output information only needs to be information that indicates the same type as the first information, and may be information extracted from the building data in accordance with the first information. The output information can be, for example, abbreviations for parts of the internal structure or existing objects, keywords or icons handled by the management system 1, such as type IDs. Parts of the internal structure include walls, elevators, etc., and are hereinafter referred to as structures. Existing objects can refer to objects installed in the building. The output information includes location information indicating the position, such as the coordinates corresponding to the building data.

[0034] The arithmetic processing unit 11 generates and outputs map data in response to the user's request by adding output information to the original building data or image data generated from that building data. The output destination may be an external device via the communication unit 13, the display unit 15, storage in the storage unit 12, or a combination of these.

[0035] Image data generated from building data can be, for example, image data generated according to an input request. A request for image data generation may be a request indicating a method for processing building data, such as deleting extraneous line segments or converting location information into map data in a format used by the management system 1. Alternatively, a request for the generation of other types of data may be made instead of, or in addition to, a request for image data generation. For example, this request may be a request indicating a method for generating information such as generating a database that associates location coordinates with descriptive text describing the structure or object for which the building data is located.

[0036] The generated map data should ideally include keywords or icons related to parts of the internal structure or existing objects, which should be attached to or associated with the original building data or image data generated from that building data based on location information.

[0037] Thus, the output map data is map data that the robot 200 refers to in order to move, and the output information may be data that has been set to correspond to a position on the map data. Here, setting to correspond to a position may be an addition to position information such as coordinate information, or it may be an update of position information set in building data.

[0038] Next, an example of map data generation in the map generation system 10 will be explained using Figures 3 to 5. Figure 3 is a schematic diagram showing an example of map data for one floor of a facility that is input into the map generation system 10. Figure 4 is a diagram showing an example of a table showing the correspondence between waypoint attributes and actions. Figure 5 is a schematic diagram showing an example of map data for one floor of a facility that is output from the map generation system 10 in Figure 2, with the map data in Figure 3 as input.

[0039] As an example of building data input to the learning model 12a or the VLM included therein, we will explain using the architectural drawing data 1000a for one floor of the building exemplified in Figure 3.

[0040] Architectural drawing data 1000a is drawing data used during construction to show a particular floor within a facility. The facility shown as an example is a hospital, and as shown in Figure 3, this floor contains structural elements such as staff stations Ss1, Ss2, Ss3, Ss4, stairs St, and elevators EV, as well as installed elements such as tables or shelves T1, T2, etc.

[0041] The first information included in the request, that is, the information to be obtained as output information, can be information about at least one of structures and objects that affect the movement of the robot 200 (hereinafter referred to as the second information). The second information may include information about one or more of the following: waypoints indicating the route of the robot 200, narrow passages, walls, areas where people are prohibited from passing, areas where people are permitted to pass, areas where the robot 200 is prohibited from passing, areas where the robot 200 is permitted to pass, and obstacles. Note that waypoints may be referred to as relay points.

[0042] Information about waypoints can include definitions and descriptions of waypoints, and for example, it can include definitions and descriptions of waypoint attributes (hereinafter referred to as WP attributes), which are attributes of waypoints.

[0043] Here, we will explain waypoints. Table 40 is a table that describes the correspondence between the WP attributes set for a waypoint and the actions taken at that waypoint when the management system 1 is in operation. Note that a waypoint may have more than one attribute assigned to it. Specifically, in addition to the general WP attribute, one or more other attributes may be assigned.

[0044] A general waypoint is a waypoint that indicates a point along the way during travel. A waypoint can be either a starting point or an ending point. Examples of possible waypoints include, but are not limited to, areas near doors, narrow corridors, and elevators. General waypoints serve as intermediate points in route planning. Robot 200 autonomously moves towards the next waypoint by passing through the general waypoints. For example, after each waypoint is set as a general waypoint in the map data, other WP attributes shown in Table 40 can be set to override or add to it. WP attributes may also be attributes that correspond to the layout and type of surrounding rooms.

[0045] When a charger is set as the WP attribute, one of the following actions can be adopted: connecting to the charger, disconnecting from the charger, or correcting the relative position to the charger by recognizing the marker marked on the charger. For example, when the battery level of robot 200 falls below a certain value, robot 200 moves to the charger's waypoint as its destination. When robot 200 arrives at the charger's waypoint, robot 200 performs relative position correction. For example, the charger has a marker, and relative position correction is performed by the robot 200's camera capturing an image of the marker. Then, robot 200 connects to the charger and starts charging. When charging is complete, robot 200 disconnects from the charger. The following WP attributes will be explained more concisely.

[0046] When the WP attribute is set to "Return Point," an action can be adopted in which robot 200 recognizes its own position using markers placed at various locations within the facility. When the WP attribute is set to "Door," an action can be adopted in which robot 200 requests to open an automatic door. When the WP attribute is set to "Inside an Elevator," an action can be adopted in which robot 200 switches the target floor map to the elevator (EV) interior map. When the WP attribute is set to "Elevator Boarding," one of the following actions can be adopted: calling the EV car, detecting people or obstacles inside the car, boarding, or uttering a statement to indicate the intention to board or be seen off. When the WP attribute is set to "Elevator Disembarking," one of the following actions can be adopted: disembarking action, or uttering a statement to warn about disembarking. When the WP attribute is set to "Wagon Loading," one of the following actions can be adopted: correcting the relative position with the wagon by recognizing markers placed on the wagon, crawling under the wagon, or lifting up the wagon. When the WP attribute is set to "Wagon Unloading," one of the following actions can be adopted: detecting obstacles in the wagon storage area, moving to the wagon unloading position, or lifting down the wagon. When the WP attribute is set to "Waiting for Wagon Lane," the action can be taken to wait until permission to enter is received from the server, which is the management device 100, or the preceding robot 200. When the WP attribute is set to "Waiting for Right," the action can be taken to request the right to pass through the entry area at the waypoint before the right area and wait until permission to enter is received from the server. When the WP attribute is set to "Right Release," the action can be taken to notify the server that the right area has been passed.

[0047] When a request containing information about the waypoints of the robot 200 is input as first information, the arithmetic processing unit 11 inputs it into the learning model 12a. The arithmetic processing unit 11 then obtains the positional information of each location corresponding to the waypoints in the architectural drawing data 1000a, which was also input, as VLM output information, and generates map data with that positional information set.

[0048] The map data generated by inputting architectural drawing data 1000a and a request is, for example, map data 1000b shown in Figure 5. Map data 1000b is generated when a request is input requesting waypoint information defined near narrow passages as the first information. Map data 1000b is assigned waypoint WP information at each location shown by the black circles. Of course, it is also possible to simultaneously include waypoint information for definitions corresponding to each WP attribute exemplified in Table 40 as the first information. Thus, the map generation system 10 may output map data by inputting only a request that includes one type of information as the first information along with the building data, or it may output map data by inputting a request that includes multiple types of information as the first information.

[0049] Furthermore, if information other than waypoint information is set as the primary information, it is desirable that map data with such information attached be generated at the corresponding location. To give a simpler example, if the primary information includes information indicating a wall, the generated map data will include information indicating that each wall is a wall.

[0050] Furthermore, candidate waypoints may be locations that correspond to areas divided by a segmentation algorithm such as the Voronoi partitioning algorithm. In this case, the arithmetic processing unit 11 inputs the output information from the VLM to the segmentation algorithm provided after the VLM in the learning model 12a. The arithmetic processing unit 11 then performs segmentation processing based on the output information, sets waypoints at predetermined locations in each of the divided areas, and outputs the map data after setting. For example, if first information including information indicating walls is input, the VLM may output the wall information as part or all of the output information, and the segmentation algorithm may perform processing using that output information as a keyword. This processing is performed on building data input to the VLM or map data output from the VLM, and the map data can be output to include information on areas divided based on information indicating walls. Through this processing, when setting waypoints, waypoints can be set as intermediate points at predetermined locations in each area, such as the center of the area or the boundary with adjacent areas. Furthermore, by including other types of information besides walls in the first set of data, it becomes possible to define attributes for waypoints in each area, or the actions that can be taken. Additionally, the segmentation algorithm may be a machine learning model that uses the results of area segmentation performed on various building data as training data.

[0051] Furthermore, if rules indicating how area division is performed by a segmentation algorithm for various building data are known, then by including those rules in the request as part or all of the definition and explanation of waypoints as information about waypoints, waypoint information can be obtained as output information from VLM even without including a segmentation algorithm in the learning model 12a.

[0052] (Overview of this system's processing) This disclosure also includes a form of map generation method in which a computer generates the map data described above, as illustrated in the processing of the map generation system 10. This map generation method will be briefly explained using Figure 6, but various application examples, such as those illustrated in this system, can be applied. Figure 6 is a flowchart illustrating an example of the map generation method according to this embodiment.

[0053] In this map generation method, first, the computer in the map generation system 10 inputs building data, which is image data, into the VLM (S1), and also inputs a request into the VLM (S2). The order of S1 and S2 does not matter, and they may be performed simultaneously. Next, the computer performs calculations in the VLM (S3) and obtains output information that is output from the VLM in response to the request (S4). Then, the computer generates map data related to the internal structure of the building based on the output information it has obtained (S5), and the process ends.

[0054] Furthermore, this disclosure also includes the form of a program that causes a computer to perform the processing shown in such a map generation method. Also, some or all of the processing in the robot 200, management device 100, etc., described above can be implemented as a program. Such a program can be stored and supplied to a computer using various types of non-temporary computer-readable media. Non-temporary computer-readable media include various types of tangible recording media. Examples of non-temporary computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, RAMs (Random Access Memory)). Furthermore, programs may be supplied to a computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. Temporary computer-readable media can be supplied to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0055] (Effects of this embodiment) According to this embodiment, map data containing desired information, such as information that affects the movement of an autonomous mobile vehicle, can be easily generated from image data of the building's internal structure without requiring a huge amount of effort. Furthermore, according to this embodiment, because VLM automatically generates information about the building's interior, map data can be generated without the movement of an autonomous mobile vehicle in the simulation environment, that is, without relying on measurement result data. Therefore, according to this embodiment, regardless of whether the building data includes measurement result data or not, situations where map information such as areas where the autonomous mobile vehicle cannot move in the simulation environment cannot be obtained do not occur. Even if the building data consists only of measurement result data, this embodiment can output map data with information such as waypoints set in an area that does not affect the movement of the autonomous mobile vehicle at a minimum. In addition, according to this embodiment, because VLM can automatically generate information about the building's interior, including information that cannot be directly read from pre-prepared data such as BIM, the effort required to acquire additional building interior information can be reduced.

[0056] (Other application examples) It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention.

[0057] For example, the generated map data may be used for purposes other than the movement of an autonomous mobile object such as robot 200. Figures 3 to 5 assume that the input building data and generated map data are two-dimensional, but as explained regarding map data, they may also be three-dimensional data. Three-dimensional map data can have attributes set as waypoints, such as ceilings and floors, and attributes such as ceilings with exhaust vents, ceilings below a predetermined height, floors with guide lights, and whether or not the floor material is a predetermined material can also be set. Therefore, for example, depending on the height of robot 200, a waypoint with the attribute of a ceiling below a predetermined height can have actions such as "no entry" set. Also, a waypoint with the attribute of a floor with guide lights can have actions such as "no parking" set. Furthermore, three-dimensional map data can also be used for the operation of an autonomous flying object, which is an autonomous mobile object that performs flight.

[0058] Furthermore, although the explanation assumed that the learning model 12a or the LVM included therein is a pre-trained model, it may also be a model that can be retrained. For example, the learning model 12a may include an open-source machine learning model such as RaG (Retrieval-Augmented Generation) as a VLM. This allows the processing unit 11 to update the learning model 12a based on the latest database. This is because the database can be updated to obtain more accurate output information. [Explanation of symbols]

[0059] 1 management system, 10 map generation systems, 100 control devices, 200 robots, 500 cameras

Claims

1. A visual language model, which is a model that takes image data and language data as inputs and outputs at least one of the image data and language data, inputs building data, which is image data that includes at least the internal structure of a building, and a request that includes a request to generate first information, which is information about at least one of a part of the internal structure and an existing object included in the building data, In response to the aforementioned request, the output information output from the visual language model is obtained, Based on the acquired output information, map data related to the internal structure of the building is generated. Map generation system.

2. The first information is information relating to at least one of a part of the internal structure and an object that affects the movement of the moving body. The map generation system according to claim 1.

3. The first information includes information relating to the waypoint of the moving object, The map generation system according to claim 2.

4. The aforementioned map data is map data that a moving object refers to in order to move, and the output information is data set to correspond to a position on the aforementioned map data. A map generation system according to claim 1 or 2.

5. The building data is at least one of the following: architectural drawing data of the building, image data acquired by a camera installed inside the building, and image data acquired by an autonomous mobile body equipped with sensors moving inside the building. A map generation system according to claim 1 or 2.

Images