Map generation system, map generation device, map generation method, map generation program, and measurement system

Abstract

Provided are a map generation system, a map generation device, a map generation method, a map generation program, and a measurement system that make it possible to manage, inter alia, generation of map information of a facility at lower costs, the map information being used by a movable body. The map generation system comprises a plurality of markers (19) having a three-dimensional shape, a measurement device (21), and a generation unit. The markers (19) are each temporarily disposed at a corresponding position in a facility. The measurement device (21) is a transportable device provided with a second measurement unit (22) and a detection unit (26). In sections including the positions corresponding to the markers (19), the second measurement unit (22) measures three-dimensional measurement data of the surroundings. The generation unit generates map information of the facility for a plurality of sections by combining, with reference to the markers (19) detected by the detection unit (26), three-dimensional measurement data measured for different sections each including the position corresponding to a certain marker (19).

Description

Map generation system, map generation device, map generation method, map generation program, and measurement system

[0001] The present disclosure relates to a map generation system, a map generation device, a map generation method, a map generation program, and a measurement system.

[0002] Patent Literature 1 discloses an example of a system for generating three-dimensional data of an environment in which a mobile object moves. The measurement device of the system includes an inertial measurement unit and a laser scanner. The measurement device continuously measures the three-dimensional data while moving.

[0003] Japanese Patent Publication No. 2022-515162

[0004] However, in the system of Patent Document 1, measurements are continuously taken within the range in which 3D data is measured, so when measuring data for the entire facility, it is necessary to continuously measure the entire facility. In this case, it may be necessary to restrict access to the facility during measurement, such as by facility users, because this may interfere with the measurement. In this case, for example, restricting access to the facility is time-consuming, and generating map information requires management costs.

[0005] The present disclosure relates to solving such problems, and provides a map generation system, a map generation device, a map generation method, a map generation program, and a measurement system that can generate and manage map information for facilities used by mobile objects at lower cost.

[0006] The map generation system according to the present disclosure is a map generation system that generates map information used to reference the position within a facility including a plurality of sections of a mobile body moving through the facility, and includes: a plurality of markers having three-dimensional shapes and temporarily placed at corresponding positions in the facility; a portable measurement device including a measurement unit that measures three-dimensional measurement data of the surrounding area of ​​the section including the positions corresponding to the markers, and a detection unit that identifies and detects each of the markers; and a generation unit that generates map information of the facility across a plurality of sections including the first section and the second section by combining the different three-dimensional measurement data measured by the measurement unit for each of a first section and a second section that are different sections and each include the positions corresponding to the markers, using the positions of the markers detected by the detection unit as a reference.

[0007] The map generation device according to the present disclosure is a map generation device that generates map information used by a mobile body moving through a facility including a plurality of sections to refer to its position within the facility, and includes an acquisition unit that identifies and detects each of a plurality of markers that have three-dimensional shapes and are temporarily placed at corresponding positions in the facility, and acquires the measured three-dimensional measurement data from a portable measurement device that measures three-dimensional measurement data of the surroundings of each section in the section including the position to which the marker corresponds, and a generation unit that generates map information of the facility across a plurality of sections including the first section and the second section by combining the different three-dimensional measurement data measured by the measurement device for each of first and second sections that are different sections and each include the position to which the marker corresponds, using the positions of the markers detected by the measurement device as a reference.

[0008] A map generation method according to the present disclosure is a map generation method for generating map information used to reference the position within a facility including a plurality of sections of a mobile body moving through the facility, the method comprising the steps of: a computer identifying and detecting each of a plurality of markers having a three-dimensional shape and temporarily placed at corresponding positions in the facility; acquiring the measured three-dimensional measurement data from a portable measurement device that measures three-dimensional measurement data of the surroundings of a section including the position corresponding to the marker; and generating map information of the facility across a plurality of sections including the first section and the second section by combining the different three-dimensional measurement data measured by the measurement device for a first section and a second section that are different sections and each include the position corresponding to the marker, using the positions of the markers detected by the measurement device as a reference.

[0009] A map generation program according to the present disclosure is a map generation program that causes a computer to generate map information used to reference the position within a facility including a plurality of sections of a mobile object moving within the facility, and causes the computer to identify and detect each of a plurality of markers that have three-dimensional shapes and are temporarily placed at corresponding positions in the facility, and acquire the measured three-dimensional measurement data from a portable measurement device that measures three-dimensional measurement data of the surrounding area of ​​the section including the position corresponding to the marker, and generate map information of the facility across a plurality of sections including the first section and the second section by combining the different three-dimensional measurement data measured by the measurement device for each of a first section and a second section that are different sections and each include the position corresponding to the marker, using the positions of the markers detected by the measurement device as a reference.

[0010] The measurement system disclosed herein is a measurement system used to generate map information used to reference the position within a facility including a plurality of sections of a moving object moving within the facility, and comprises a plurality of markers having three-dimensional shapes and temporarily placed at corresponding positions in the facility, and a portable measurement device. The measurement device comprises a measurement unit that measures three-dimensional measurement data of the surrounding area of ​​a section including the position corresponding to the marker, a detection unit that identifies and detects each of the markers, and a communication unit that outputs the three-dimensional measurement data measured by the measurement unit to a map generation device that generates map information of the facility across a plurality of sections including the first section and the second section by combining the different three-dimensional measurement data measured by the measurement unit for each of a first section and a second section that are different sections and each include the position corresponding to the marker, using the position of the marker detected by the detection unit as a reference.

[0011] According to the map generation system, map generation device, map generation method, map generation program, or measurement system disclosed herein, it becomes possible to generate and manage map information for facilities used by mobile bodies at lower cost.

[0012] 1 is a configuration diagram of a mobile body system according to a first embodiment. FIG. 2 is a diagram illustrating the relationship of data used to generate map information in the mobile body system according to the first embodiment. FIG. 3 is a diagram illustrating an example of the structure of facility structure data in the mobile body system according to the first embodiment. FIG. 4 is a diagram illustrating another example of the structure of facility map data in the mobile body system according to the first embodiment. FIG. 5 is a diagram illustrating an example of markers placed at a facility in the mobile body system according to the first embodiment. FIG. 6 is a diagram illustrating an example of a section in which three-dimensional measurement data is measured in the mobile body system according to the first embodiment. FIG. 7 is a diagram illustrating an example of alignment when measuring three-dimensional measurement data in the mobile body system according to the first embodiment. FIG. 8 is a diagram illustrating an example of alignment when measuring three-dimensional measurement data in the mobile body system according to the first embodiment. FIG. 9 is a diagram illustrating an example of a data structure of measurement information that stores three-dimensional measurement data measured in the mobile body system according to the first embodiment. FIG. 10 is a diagram illustrating an example of a data structure of measurement information that stores three-dimensional measurement data measured in the mobile body system according to the first embodiment. FIG. 11 is a diagram illustrating an example of a section in which sub-markers are placed in the mobile body system 1 according to the first embodiment. FIG. 12 is a configuration diagram of a mobile body system according to a second embodiment. FIG. 13 is a block diagram illustrating an example of learning and inference by a learning unit according to the second embodiment. FIG. 14 is an example of a floor plan of a facility to which a mobile body system according to a third embodiment is applied. 11 is an example of a graph structure of facility structure data generated for a facility to which a mobile system according to a third embodiment is applied.

[0013] The following describes embodiments of the subject matter of the present disclosure with reference to the accompanying drawings. In each drawing, identical or corresponding parts are designated by the same reference numerals, and redundant explanations are appropriately simplified or omitted. Note that the subject matter of the present disclosure is not limited to the following embodiments, and any component of the embodiments may be modified or omitted within the scope of the gist of the present disclosure.

[0014] First Embodiment Fig. 1 is a configuration diagram of a mobile system 1 according to a first embodiment.

[0015] The mobile body system 1 is a system including a mobile body 2 that operates in a facility. The mobile body system 1 manages information about the mobile body 2 itself and information about the facility that the mobile body 2 refers to.

[0016] A facility may include, for example, one or more buildings. A facility may be part or all of a building. A facility may include one or both of an outdoor portion and an indoor portion of a building. A facility may be, for example, a commercial facility, an office building, a lodging facility, a residential facility, a public facility, or other facility, or a combination thereof.

[0017] A facility is divided into multiple sections. That is, a facility is composed of multiple sections. A section may be, for example, a corridor within the facility. Adjacent sections within a facility may overlap with each other. In a facility, free movement of facility users is permitted within the sections. In a facility, physical separations such as walls, steps, or partitions may be installed between sections, restricting the movement of users and others. In a facility, a room may contain a corridor or other passageway due to features such as walls or desks within the room. In this case, the passageway may be treated as a single section. Examples of features include furniture and other structures located within the facility that are not part of the facility itself. A facility is equipped with equipment that opens the separation between sections. Examples of equipment in a facility include access control devices such as automatic doors or security gates. Furthermore, a facility is divided into multiple sections vertically. Sections at the same height are considered to be on the same floor, and a facility includes a facility composed of multiple floors. Sections on different floors are separated by floors and ceilings. However, it may also include an atrium that is not separated by floors or ceilings. In a facility, it includes passageways between sections on different floors that are not separated by walls and allow movement in the vertical direction, such as stairs or ramps. In addition, facilities are equipped with equipment that has sections that move in the vertical direction. Equipment in a facility includes, for example, lifts such as elevators or escalators.

[0018] The mobile object 2 is a mobile device that operates to provide services in the facility. In this example, a plurality of mobile objects 2 are operating in the facility. The mobile object 2 is, for example, an autonomous mobile object that moves autonomously. The mobile object 2 may be, for example, a robot, a drone, mobility, or other mobile device. Each mobile object 2 includes a calculation unit 3a, a memory unit 4a, a communication unit 5a, a first measurement unit 6, and a drive unit 7.

[0019] The calculation unit 3a is, for example, a device such as a CPU (Central Processing Unit), an arithmetic device, a microprocessor, or a microcomputer. The storage unit 4a is, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read-Only Memory), a flash memory, an EPROM (Erasable Programmable Read-Only Memory), or an EEPROM (Electrically Erasable Programmable Read-Only Memory), or a device such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD (Digital Versatile Disc). Part or all of the calculation unit 3a and the storage unit 4a may be configured with dedicated processing circuits. The storage unit 4a stores, for example, programs as software or firmware. In the mobile object 2, the calculation unit 3a executes the programs stored in the storage unit 4a to perform pre-set processing, and each function is realized as a result of collaboration between hardware and software. Each function of the mobile object 2 may be realized by a separate processing circuit. Alternatively, part or all of the functions of the mobile object 2 may be realized collectively by a processing circuit. Furthermore, the processing circuit may be realized by, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these.

[0020] The communication unit 5a is a part equipped with a function for communicating information with devices external to the mobile object 2. The communication unit 5a communicates with the external devices, for example, by wireless communication. The communication unit 5a communicates with the external devices, for example, through a communication network 8, such as the Internet, a telephone network, or an optical communication network. The communication network 8 may include a local network such as a LAN (Local Area Network) within a facility, or may include industrial communication such as short-range wireless communication. The communication network 8 may include a wired or wireless intranet, for example.

[0021] The first measurement unit 6 is a part including a device for measuring information necessary for the operation of the mobile object 2, such as the environment around the mobile object 2. The first measurement unit 6 may be, for example, a LiDAR (Light Detection and Ranging), an IMU (Inertial Measurement Unit), an ultrasonic sensor, an RGBD camera (RGBD: Red-Green-Blue-Depth), a stereo camera, a satellite positioning system such as a GPS (Global Positioning System), an indoor positioning system, other sensors, or modules of other measurement systems. The information measured by the first measurement unit 6 is processed in the calculation unit 3a or the like.

[0022] The drive unit 7 is a part that includes a device that generates a drive force to move the mobile object 2. The drive unit 7 may include a drive source such as a motor, wheels, crawlers, a quadrupedal drive device, a bipedal drive device, or an inverted pendulum drive device. The drive unit 7 acquires a route from the calculation unit 3a and controls the drive source to follow the route. The drive unit 7 acquires the state of the mobile object 2 from the first measurement unit 6 and controls or suspends the drive source to maintain safety. The drive unit 7 acquires the location of surrounding obstacles from the first measurement unit 6 and controls or suspends the drive source to avoid the obstacles. The drive unit 7 also includes a function to manage the drive source to execute the services provided by the mobile object 2.

[0023] The calculation unit 3a includes a behavior control unit 9. The behavior control unit 9 is a part that controls the behavior of the mobile object 2 in a facility. The behavior of the mobile object 2 in a facility includes the movement of the mobile object 2 in the facility and the execution of tasks for services provided by the mobile object 2 in the facility. The behavior control unit 9 controls the behavior of the mobile object 2, for example, by outputting a control signal to the driving unit 7 based on information measured by the first measurement unit 6. The behavior control unit 9 operates according to a program that registers and manages the behavior or state of the mobile object 2. When the mobile object 2 can execute behaviors based on multiple states, it switches the control commands set in the driving unit 7 based on information obtained from the communication unit 5a and the first measurement unit 6. The mobile object 2 has a function to autonomously switch states depending on the status of the driving unit 7. The behavior control unit 9 manages the timing of state transitions and the synchronization of states with other devices. The behavior control unit 9 operates according to a program based on a finite state machine, for example. The behavior control unit 9 operates according to a program based on a state transition diagram, for example. The behavior control unit 9 operates, for example, according to a program based on sequence control. When controlling the behavior of the mobile object 2 within a facility, the behavior control unit 9 uses map information to refer to the position of the mobile object 2 within the facility. The map information of the facility is, for example, information that represents the three-dimensional arrangement of structures and the like within the area in which the mobile object 2 moves within the facility. The map information used by the mobile object 2 is represented, for example, in the form of point cloud data, an occupancy grid map, CAD data (CAD: Computer-Aided Design), or some other format.

[0024] The mobile system 1 includes a mobile server 10. The mobile server 10 controls the operation of the mobile object 2. The operation of the mobile object 2 includes information processing in the mobile object 2 and the behavior of the mobile object 2 in facilities. The mobile server 10 may control the operation of multiple mobile objects 2. The computing unit 3a installed in each mobile object 2 may have performance constraints due to the power supply capacity of the mobile object 2's battery, etc. In such cases, the mobile server 10 can handle part or all of the processing related to the control of the operation of the mobile object 2. On the other hand, for example, when the computing unit 3a installed in each mobile object 2 has fewer performance constraints, the mobile object 2 may handle part or all of the processing of the mobile server 10. The mobile system 1 may include multiple mobile servers 10. Each mobile server 10 may be managed by a different administrator, for example. The administrator of the mobile server 10 may be, for example, the manufacturer or management company of the mobile object 2. The same administrator may manage multiple mobile servers 10. Each mobile server 10 is, for example, a server device consisting of one or more server computers. The multiple server devices constituting the mobile server 10 may be located in different locations. In this case, the multiple server devices communicate information with each other, for example, via a communication network 8. Each mobile server 10 includes a calculation unit 3b, a storage unit 4b, and a communication unit 5b.

[0025] The calculation unit 3b is a device such as a CPU, an arithmetic unit, a microprocessor, or a microcomputer. The storage unit 4b is a device such as a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, an EPROM, or an EEPROM, or a device such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD. Part or all of the calculation unit 3b and the storage unit 4b may be configured with a dedicated processing circuit. The storage unit 4b stores, for example, programs such as software or firmware. In the mobile server 10, the calculation unit 3b executes programs stored in the storage unit 4b to perform pre-set processing, and each function is realized as a result of collaboration between hardware and software. Each function of the mobile server 10 may be realized by a separate processing circuit. Alternatively, some or all of the functions of the mobile server 10 may be realized collectively by a processing circuit. The processing circuitry may be implemented, for example, as a single circuit, multiple circuits, a programmed processor, a parallel programmed processor, an ASIC, or an FPGA, or a combination thereof. Some or all of the functions of the mobile server 10 may be implemented, for example, by processing or storage resources on a cloud service.

[0026] The communication unit 5b is a part equipped with a function for communicating information with devices external to the mobile server 10. The communication unit 5b communicates with external devices via the communication network 8, for example, by wired communication or wireless communication. The communication unit 5b communicates information with one or more mobile bodies 2, the operation of which is to be controlled, as external devices.

[0027] The calculation unit 3b includes a behavior planning unit 11 and a first generation unit 12. The behavior planning unit 11 plans the behavior of the mobile object 2 in a facility and manages necessary information. Here, information management includes, for example, operations such as retaining, adding, changing, and deleting the information. The information managed by the behavior planning unit 11 includes information about the facility related to the movement of the mobile object 2. The behavior planning unit 11 controls the behavior of the mobile object 2 in the facility using, for example, facility map data. The facility map data includes, for example, information about the physical structure, such as the layout of structures in the facility, and information about the logical structure, such as the connection relationships between each location in the facility. The connection relationships between each location indicate, for example, the passability or pass weight of users or the mobile object 2 between the locations. The first generation unit 12 performs processing related to the generation of map information used to reference the location of the mobile object 2 in the facility. The first generation unit 12 generates map information to be distributed to the mobile object 2 so that the behavior control unit 9 of each mobile object 2 can refer to it. The first generator 12 generates map information to be distributed to the mobile object 2 based on, for example, facility map data.

[0028] The mobile object system 1 includes a facility server 13. The facility server 13 is a part that manages facility information, etc. The facility information includes, for example, information on whether users or mobile objects 2 can pass between each location in the facility. The facility server 13 is, for example, a server device consisting of one or more server computers. The multiple server devices that make up the facility server 13 may be located in different locations. In this case, the multiple server devices communicate information with each other, for example, via a communication network 8. The facility server 13 includes a calculation unit 3c, a memory unit 4c, a communication unit 5c, an input unit 14, and an output unit 15.

[0029] The calculation unit 3c is a device such as a CPU, an arithmetic device, a microprocessor, or a microcomputer. The storage unit 4c is a device such as a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, an EPROM, or an EEPROM, or a device such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD. The calculation unit 3c and part or all of the storage unit 4c may be configured with a dedicated processing circuit. The storage unit 4c stores, for example, programs as software or firmware. In the facility server 13, the calculation unit 3c executes the programs stored in the storage unit 4c to perform pre-set processing, and each function is realized as a result of collaboration between hardware and software. Each function of the facility server 13 may be realized by a separate processing circuit. Alternatively, some or all of the functions of the facility server 13 may be realized collectively by a processing circuit. The processing circuit may also be realized by, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. Some or all of the functions of the facility server 13 may be implemented, for example, by processing or storage resources on a cloud service.

[0030] The communication unit 5c is a part equipped with a function for communicating information with devices external to the facility server 13. The communication unit 5c communicates with the external devices via the communication network 8, for example, by wired communication or wireless communication. The communication unit 5c communicates information with the external devices, for example, the mobile server 10 or the mobile 2.

[0031] The input unit 14 is a part equipped with a function for accepting input of information to the facility server 13. The input unit 14 includes, for example, an input device such as a keyboard or a mouse, or an information processing terminal device connected to the facility server 13. A part or all of the input unit 14 may be configured by the calculation unit 3c and the communication unit 5c, etc. The input unit 14 may, for example, accept communication commands via an API (Application Programming Interface) from an external device such as the mobile server 10.

[0032] Output unit 15 is a part equipped with a function to output information from facility server 13. Output unit 15 includes, for example, an output device such as a display panel, or an information processing terminal device connected to facility server 13. Part or all of output unit 15 may be configured by calculation unit 3c and communication unit 5c, etc. Output unit 15 may output communication commands via an API as instructions to an external device such as mobile server 10, for example.

[0033] The calculation unit 3c includes a facility information management unit 16 and a second generation unit 17. The facility information management unit 16 manages information about the structure of facilities related to the movement of the mobile object 2. The information managed by the facility information management unit 16 includes facility structure data including information about the connection relationships between each position in the facility. The management of information by the facility information management unit 16 includes generating, retaining, changing, deleting, and other operations of the information. The second generation unit 17 performs processing related to the generation of map information used to refer to the position of the mobile object 2 within the facility. For example, the second generation unit 17 generates facility map data using the information managed by the facility information management unit 16. The facility map data generated by the second generation unit 17 is converted into map information to be distributed to the mobile object 2 by the first generation unit 12. The second generation unit 17 may directly generate the map information to be distributed to the mobile object 2.

[0034] Here, the combination of first generator 12 and second generator 17 is an example of a generator that generates map information used by mobile object 2. In this case, mobile object server 10 and facility server 13, which include first generator 12 and second generator 17, function as a single map generation device by cooperating with each other. Note that, for example, some or all of the functions of first generator 12, second generator 17, and facility information manager 16 may be mounted on mobile object server 10 or on facility server 13. Some or all of the functions of first generator 12, second generator 17, and facility information manager 16 may be implemented on a single or multiple other server devices.

[0035] The mobile body system 1 includes a measurement system 18. The measurement system 18 is a system that measures three-dimensional measurement data used to generate map information that is used to refer to the position of the mobile body 2. The three-dimensional measurement data is, for example, three-dimensional point cloud data. The three-dimensional measurement data may be measurement data that includes information on the three-dimensional space of a facility, or may be multidimensional data that includes time or other information. The measurement system 18 that measures the three-dimensional measurement data used to generate map information, together with a map generation device that generates the map information, constitutes a map generation system. The measurement system 18 includes a plurality of markers 19, a plurality of sub-markers 20, and a measurement device 21.

[0036] Each marker 19 corresponds to a position in the facility. Each marker 19 has a three-dimensional shape and is temporarily placed at a corresponding position in the facility. At least a portion of each marker 19 has a unique shape, color, pattern, or the like so that the markers can be distinguished from one another. Each marker 19 is temporarily placed, for example, when the three-dimensional measurement data is measured. For example, each marker 19 may be placed only when the three-dimensional measurement data is measured for the first time in the facility. For example, each marker 19 is removed from the corresponding position in the facility when users can use the facility, rather than while the three-dimensional measurement data is being measured. Some or all of the markers 19 correspond to positions in the facility, such as corridor branching points and room entrances and exits.

[0037] Each sub-marker 20 corresponds to a position within the facility. Each sub-marker 20 is permanently installed at a corresponding position within the facility. Information about the corresponding position is attached to each sub-marker 20. This information is attached to the sub-marker 20 by, for example, an encoded image such as a two-dimensional code. Each sub-marker 20 can be distinguished from others by, for example, the corresponding position information. The sub-marker 20 may be, for example, a label attached to a wall surface of the facility.

[0038] The measuring device 21 is a device that measures three-dimensional measurement data. The measuring device 21 is, for example, a portable device that can be carried by a human worker within the facility and moved around the facility. In this case, the measuring device 21 may be a portable general-purpose information processing device equipped with measurement functions. The measuring device 21 may be, for example, a portable device that can be mounted on an autonomous vehicle for measurement and moved around the facility. In this case, the measuring device 21 may be a measurement module mounted on the autonomous vehicle. The measuring device 21 measures three-dimensional measurement data within the facility while being carried by, for example, a worker and moving around the facility. The measuring device 21 includes a calculation unit 3d, a memory unit 4d, a communication unit 5d, a second measurement unit 22, and a display unit 23.

[0039] The calculation unit 3d is a device such as a CPU, an arithmetic device, a microprocessor, or a microcomputer. The storage unit 4d is a device such as a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, an EPROM, or an EEPROM, or a device such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD. Part or all of the calculation unit 3d and the storage unit 4d may be configured with a dedicated processing circuit. The storage unit 4d stores, for example, programs such as software or firmware. In the measurement device 21, the calculation unit 3d executes programs stored in the storage unit 4d to perform pre-set processing, and each function is realized as a result of collaboration between hardware and software. Each function of the measurement device 21 may be realized by a separate processing circuit. Alternatively, some or all of the functions of the measurement device 21 may be realized collectively by a processing circuit. The processing circuit may also be realized by, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0040] The communication unit 5d is a part equipped with a function for communicating information with devices external to the measuring device 21. The communication unit 5d communicates with external devices, for example, by wireless communication. The communication unit 5d communicates with external devices, for example, by wireless communication through the communication network 8. The communication unit 5d communicates information with external devices, for example, the facility server 13.

[0041] The second measurement unit 22 is a part including devices for measuring information about the environment surrounding the measurement device 21. The second measurement unit 22 includes, for example, a sensor 24 and a camera 25. The sensor 24 is equipped with a function for measuring three-dimensional measurement data, such as three-dimensional point cloud data. The sensor 24 may be, for example, a LiDAR, an IMU, an ultrasonic sensor, a ToF sensor (ToF: Time of Flight), a satellite positioning system such as GPS, an indoor positioning system, or other measurement system modules. The sensor 24 may also be a combination of multiple measurement modules. The sensor 24 is equipped with a function for measuring the relative position, such as the distance, between the measurement device 21 and an object. The sensor 24 is equipped with a function for measuring the attitude of the measurement device 21. The camera 25 is equipped with a function for capturing images. The camera 25 may be, for example, an RGBD camera, a stereo camera, or a VSLAM camera (VSLAM: Visual SLAM, SLAM: Simultaneous Localization And Mapping). The camera 25 may also be equipped with a function for measuring three-dimensional measurement data. The second measurement unit 22 integrates information acquired by one or both of the sensor 24 and the camera 25 using, for example, a SLAM algorithm. As a result, the second measurement unit 22 simultaneously records the coordinates of the measurement device 21 during movement and three-dimensional measurement data of the surrounding area, using the coordinates of the measurement device 21 at the start of movement as reference coordinates.

[0042] The display unit 23 is a part that includes a device for displaying information, and is, for example, an output device such as a display panel that displays information including images and text to the worker carrying the measuring device 21. The display unit 23 displays information output from the calculation unit 3d, etc.

[0043] The calculation unit 3d of the measurement device 21 includes a detection unit 26. The detection unit 26 identifies and detects each marker 19. The detection unit 26 identifies and detects the marker 19, for example, based on an image captured by the camera 25 of the second measurement unit 22. The detection unit 26 may identify and detect the marker 19, for example, based on a shape measured by the sensor 24 of the second measurement unit 22. The detection unit 26 identifies and detects each sub-marker 20. The detection unit 26 may identify and detect the sub-marker 20, for example, based on an encoded image captured by the camera 25 of the second measurement unit 22.

[0044] Each marker 19 is provided with features that enhance the accuracy of detection by the detection unit 26. These features include, for example, the shape, color, or pattern of at least a portion of the surface of the marker 19. For example, the marker 19 has a portion with a constant positive or negative curvature on at least a portion of its surface. A shape with a constant curvature may be, for example, a spherical surface. For example, at least a portion of the surface of the marker 19 is provided with a periodic pattern of discrete gradations. The periodic pattern of discrete gradations may be, for example, a two-tone stripe or checkerboard pattern with one of two different colors as the background color. Such a pattern allows the shape of the surface of the marker 19 to be clearly detected. For example, the marker 19 does not include a surface with a metallic luster or a mirrored surface. For example, when the sensor 24 or camera 25 of the second measurement unit 22 measures or photographs using light or electromagnetic waves of a specific wavelength, the reflectivity of the surface of the marker 19 at that wavelength is lower than the reflectivity of the facility structures surrounding the marker 19.

[0045] FIG. 2 is a diagram showing the relationship between data used to generate map information in the mobile body system 1 according to the first embodiment.

[0046] The facility information management unit 16 of the facility server 13 manages facility structure data. The facility structure data is data that represents the structure of a facility, such as the connection relationships between each location in the facility. The facility structure data includes information on the locations within the facility to which each marker 19 corresponds. In this example, the facility structure data is graph data that represents a graph structure in which the locations within the facility to which the marker 19 corresponds are nodes and includes information on the nodes and the edges connecting the nodes. When nodes in the facility structure data are connected by edges, the graph structure is formed so that a mobile object 2 can pass between the locations corresponding to the nodes. In the facility structure data, each edge corresponds to a section, such as a passageway, within the facility. The route on the graph in the facility structure data corresponds to the movement line of the mobile object 2 within the facility. The movement line of the mobile object 2 is, for example, a path along the longitudinal direction of the passageway.

[0047] The measuring device 21 measures three-dimensional measurement data for each section of the facility. When measuring the three-dimensional measurement data, for example, a marker 19 is placed at a corresponding position within the facility. The measured three-dimensional measurement data is output to the facility server 13 via the communication unit 5d or the like.

[0048] Second generation unit 17 of facility server 13 generates facility map data based on the three-dimensional measurement data measured by measuring device 21 and the facility structure data managed by facility information management unit 16. The generated facility map data is output to mobile server 10 via communication unit 5c or the like.

[0049] The first generation unit 12 of the mobile server 10 generates map information referenced by the mobile object 2 based on the facility map data generated by the second generation unit 17. In this example, each mobile object 2 uses 3D point cloud data as map information of the facility. At this time, the second generation unit 17 generates 3D point cloud data covering multiple sections of the facility based on the facility map data. If the facility map data includes individual 3D point cloud data for each section, the second generation unit 17, for example, integrates the 3D point cloud data for the individual sections to generate map information referenced by the mobile object 2. If the facility map data includes CAD data of the facility, the second generation unit 17 may, for example, be equipped with a simulator function that virtually simulates sensor behavior. At this time, the second generation unit 17 acquires measurement information of a virtual sensor on the simulator for the CAD data of the facility and generates map information in the form of 3D point cloud data referenced by the mobile object 2. The generated map information is output to each mobile object 2 via the communication unit 5b or the like.

[0050] Each mobile object 2 operates to move within the facility and provide services while referring to its location within the facility using map information distributed from the mobile object server 10 or the like.

[0051] FIG. 3 is a diagram showing an example of the structure of facility structure data in the mobile body system 1 according to the first embodiment.

[0052] The facility structure data includes data for each floor of the facility. The facility structure data includes data on a logical structure, such as the connection relationships between each position on each floor. The logical structure for each floor is represented by a graph structure.

[0053] In the facility structure data, the logical structure for each floor includes information on multiple edges. Each edge corresponds to, for example, one of the sections of the facility. Each edge includes information on the corresponding section. The information on the corresponding section includes, for example, the identifier, name, attributes, or other information of the section. The information on the corresponding section may also be information indicating a data reference destination.

[0054] In the facility structure data, two nodes correspond to each edge. One node corresponding to an edge is, for example, the node at the start point of the edge. The other node corresponding to the edge is, for example, the node at the end point of the edge. Each node corresponds, for example, to a location in the facility. Each node includes information about the corresponding location. The information about the corresponding location includes, for example, an identifier, name, attributes, or other information about the location. The information about the corresponding location may be information indicating a data reference destination. The attributes of the location may be, for example, information such as the purpose of the room including the location.

[0055] FIG. 4 is a diagram showing another example of the structure of facility map data in the mobile body system 1 according to the first embodiment.

[0056] The facility map data includes data for each floor of the facility. The facility map data includes logical structure data such as the connection relationships between each position on each floor. The logical structure for each floor is represented by a graph structure. The graph structure includes, for example, edge and node information similar to the facility structure data. Reference coordinates are set in the graph structure data. The reference coordinates include information such as the origin of the coordinate system and the directions of the coordinate axes.

[0057] The data for each node includes information on the node identifier and node coordinate information. The node identifier is set uniquely within a facility. The node coordinate information includes information on the coordinates of the position to which the node corresponds. The node coordinate information is expressed, for example, by coordinate values ​​in a coordinate system of reference coordinates set in the graph. The data for each edge can be uniquely specified from a pair of two nodes. Data including the edge identifier and the like is associated with each edge. The identifier for each edge is set uniquely within a facility.

[0058] The facility map data includes three-dimensional shape information. The three-dimensional shape information is, for example, data for each floor of the facility. The three-dimensional shape information includes CAD model information and measurement information.

[0059] The CAD model information includes information on the shapes of structures such as walls, floors, ceilings, columns, and beams of the facility. The CAD model information may also include information on the shapes of features such as desks and fire extinguishers placed in the facility. Reference coordinates are set in the CAD model information. The reference coordinates include information such as the origin of a coordinate system and the direction of the coordinate axes. In this example, the CAD model information includes information on the relationship between the reference coordinates of the CAD model information and the reference coordinates of the graph structure data. The information on the relationship between the two reference coordinates may be information on transformations such as translation, rotation, and inversion between the coordinate systems of these reference coordinates.

[0060] The measurement information includes information obtained by measuring the shapes of structures such as walls, floors, ceilings, columns, and beams of the facility using a measurement system 18 or the like. The measurement information is, for example, three-dimensional point cloud data. The measurement information may also be, for example, an occupancy grid map. Reference coordinates are set in the measurement information. The reference coordinates include information such as the origin of a coordinate system and the directions of the coordinate axes. In this example, the measurement information includes information on the relationship between the reference coordinates of the measurement information and the reference coordinates of the graph-structured data. The information on the relationship between two reference coordinates may be information on transformations such as translation, rotation, and inversion between the coordinate systems of these reference coordinates.

[0061] The CAD model information may be, for example, preset design information. Alternatively, the CAD model information may be, for example, information updated based on measurement information. For example, information such as the position and orientation of a wall surface or feature in the CAD model information may be updated by fitting point cloud data or other processing. This processing is performed, for example, by the second generation unit 17. The measurement information may be information measured by the measurement system 18, or may be information measured by, for example, another device or system external to the mobile body system 1.

[0062] The facility map data includes multiple pieces of image information. The image information is information about images captured at the facility. For example, the image information is information about images captured by the camera 25 of the measurement device 21. Each piece of image information includes information about the image itself, such as a still image or video, and information about the circumstances under which the image was captured. The information about the capturing circumstances includes information about the capturing location, field of view, and capturing time. The information about the capturing location is represented, for example, by coordinate values ​​in a reference coordinate system of the graph-structured data for the camera 25 that captured the image or the position of the subject being captured. The information about the field of view is represented, for example, by the attitude of the measurement device 21 and the angle of view of the camera 25. Each piece of image information may be information acquired by the measurement system 18, or may be information acquired, for example, by another device or system external to the mobile system 1.

[0063] Next, examples of measuring three-dimensional measurement data at a facility and generating map information will be described using Figures 5 to 10. Figure 5 is a diagram showing an example of a marker 19 placed at a facility in the mobile body system 1 according to the first embodiment. Figure 6 is a diagram showing an example of a section in which three-dimensional measurement data is measured in the mobile body system 1 according to the first embodiment. Figures 7 and 8 are diagrams explaining an example of alignment when measuring three-dimensional measurement data in the mobile body system 1 according to the first embodiment. Figures 9 and 10 are diagrams explaining an example of the data structure of measurement information that stores three-dimensional measurement data measured in the mobile body system 1 according to the first embodiment.

[0064] As shown in FIG. 5 , a worker performing measurement work for 3D measurement data temporarily places a marker 19 at a corresponding position before measurement. The worker carries a measurement device 21 to perform the measurement work. The worker uses a camera 25 of the measurement device 21 to capture an image of the marker 19 placed at the corresponding position. At this time, a detection unit 26 identifies and detects the marker 19 based on the captured image of the marker 19. When the detection unit 26 detects the marker 19, a sensor 24 measures the orientation of the measurement device 21. The sensor 24 also measures the distance to the marker 19 as a relative position. The captured image of the marker 19 is stored as image information in facility map data along with information on the shooting conditions, such as the shooting position and field of view, generated using the measured orientation and relative position information. The 3D measurement data includes multiple markers 19, such as markers 19a and 19b in FIG. 5 .

[0065] 6, the worker moves through the facility with the measuring device 21 in a section that includes the position corresponding to the marker 19 whose image was captured. The worker moves with the measuring device 21 to a position in the section that corresponds to another marker 19 different from the marker 19 whose image was captured initially. While the worker is moving through the section, the sensor 24 of the measuring device 21 that moves through the section with the worker measures three-dimensional measurement data for the section.

[0066] The worker then uses the camera 25 to capture an image of another marker 19 different from the marker 19 whose image was captured initially. The detection unit 26 then identifies and detects the other marker 19 based on the captured image of the other marker 19. When the detection unit 26 detects the other marker 19 following the detection of the marker 19 whose image was captured initially in the section, the three-dimensional measurement data measured by the sensor 24 for the section is stored in the facility map data as part of the measurement information. The data is stored by, for example, the second generation unit 17. The measurement device 21 transmits, for example, information identifying each of the two markers 19 detected by the detection unit 26 in the section of the facility and the three-dimensional measurement data measured for the section to the facility server 13 via the communication units 5d and 5c. The communication unit 5c of the facility server 13 is an example of an acquisition unit that acquires the three-dimensional measurement data from the measurement device 21. The second generation unit 17 identifies the edge corresponding to the section based on the information identifying the two markers 19 transmitted from the measurement device 21. The second generation unit 17 stores the three-dimensional measurement data transmitted from the measurement device 21 as measurement information corresponding to the edge.

[0067] Here, the second generation unit 17 removes data of the portion corresponding to the marker 19 detected by the detection unit 26 from the 3D measurement data measured by the second measurement unit 22 for the section, and stores the data in the measurement information. At this time, the second generation unit 17 stores relative position information of the two markers 19 from the 3D measurement data as information accompanying the 3D measurement data, and then deletes the data of the portion corresponding to the marker 19. In this example, the second generation unit 17 removes data of points corresponding to the marker 19 from the 3D measurement data, which is point cloud data, and stores the data in the measurement information. The second generation unit 17 may identify the point corresponding to the marker 19, for example, by pattern matching to the three-dimensional shape of the marker 19, or may identify the point based on the relative position of the marker 19 and the measurement device 21 measured by the sensor 24 or the like.

[0068] Thereafter, the worker may continue to measure 3D measurement data for other sections. At this time, the measurement device 21 may treat the marker 19 detected last in the section where measurement was first performed as the marker 19 detected first in the section where measurement is to be performed next. This allows the measurement device 21 to continuously measure 3D measurement data for each of the sections corresponding to multiple edges that are connected to each other by sharing nodes.

[0069] Note that a worker may measure the 3D measurement data for the same section multiple times. The worker may perform the second and subsequent measurements some time after the first measurement. For example, the worker may perform the second and subsequent measurements on a different day from the first measurement. The second and subsequent measurements may be performed by a different worker than the worker who performed the first measurement.

[0070] 7 shows an example of the display on the display unit 23 of the measuring device 21 during the second or subsequent measurement. In this example, a worker performs measurement work for three-dimensional data in the facility without placing markers 19. When measuring the three-dimensional measurement data, the worker aligns the measuring device 21 using the display on the display unit 23 of the measuring device 21.

[0071] The display unit 23 displays the current image being captured by the camera 25. In FIG. 7 , the current image is indicated by a solid line. The display unit 23 displays an image of the surroundings of the position corresponding to the marker 19, which was captured by the camera 25 in the past when the marker 19 was placed, superimposed on the current image, for example, by using a transparent display. In FIG. 7 , the past image is indicated by a dashed line. Here, the displayed past image refers to image information stored in the facility map data as the image corresponding to the marker 19. For example, based on the identification information of the marker 19 input by the worker, the measurement device 21 requests the facility information management unit 16 of the facility server 13 to acquire an image corresponding to the marker 19 from the image information of the facility map data. For example, based on the position and orientation of the measurement device 21 in the facility, the measurement device 21 may request the facility information management unit 16 of the facility server 13 to acquire an image corresponding to the marker 19 from the image information of the facility map data. The position of the measurement device 21 in the facility may be measured by the second measurement unit 22 or the like using, for example, a satellite positioning system, an indoor positioning system, or other measurement system.

[0072] The worker aligns the measuring device 21 by adjusting the position and orientation of the measuring device 21 so that the current image and the past image displayed on the display unit 23 overlap. After aligning the measuring device 21, the worker captures an image of the surroundings of the position corresponding to the marker 19. At this time, the detection unit 26 identifies and virtually detects the marker 19 corresponding to the position based on an image of the surroundings of the position corresponding to the marker 19 captured by the camera 25 when the marker 19 is not placed. This detection of the position corresponding to the marker 19 based on an image of the surroundings of the marker 19 at the corresponding position is referred to as virtual detection of the marker 19. Note that in cases where there is no difference in processing before and after detection between the detection of the actually placed marker 19 and the virtual detection of the marker 19, the virtual detection of the marker 19 may be simply referred to as detection of the marker 19.

[0073] When the detection unit 26 virtually detects the marker 19, the sensor 24 measures the attitude of the measuring device 21. The sensor 24 also virtually measures the distance to the marker 19 as a relative position. For example, the sensor 24 measures the relative position of the marker 19 when the marker 19 is virtually placed at the corresponding position, based on the relative positions of the marker 19 and surrounding facility structures around the corresponding position. An image captured around the position corresponding to the marker 19 is stored as image information in the facility map data, along with information on the shooting situation, such as the shooting position and field of view, which is generated using the measured attitude and relative position information.

[0074] Thereafter, the worker moves through the facility with the measuring device 21 in a section that includes the position corresponding to the marker 19 that captured the image of the surrounding area of ​​the corresponding position. The worker moves with the measuring device 21 to a position in the section that corresponds to another marker 19 that is different from the marker 19 that originally captured the image of the surrounding area of ​​the corresponding position. While the worker is moving through the section, the sensor 24 of the measuring device 21 that moves through the section with the worker measures three-dimensional measurement data for the section.

[0075] Thereafter, the worker uses the camera 25 to capture an image of the surroundings of a position corresponding to another marker 19 different from the marker 19 initially captured in the image of the surroundings of the corresponding position. At this time, the detection unit 26 identifies and virtually detects the other marker 19 based on the captured image of the surroundings of the position to which the other marker 19 corresponds. When the detection unit 26 virtually detects the other marker 19 following the virtual detection of the marker 19 initially captured in the image of the surroundings of the corresponding position in the section, the three-dimensional measurement data measured by the sensor 24 for the section is stored in the facility map data as part of the measurement information. The data is stored by, for example, the second generation unit 17. The measurement device 21 transmits, for example, information identifying each of the two markers 19 virtually detected by the detection unit 26 in the section of the facility and the three-dimensional measurement data measured for the section to the facility server 13. The second generation unit 17 identifies the edge corresponding to the partition based on the information identifying the two markers 19 transmitted from the measurement device 21. The second generation unit 17 stores the three-dimensional measurement data transmitted from the measurement device 21 as measurement information corresponding to the edge. The second generation unit 17 saves relative position information of the two markers 19 from the three-dimensional measurement data as information accompanying the three-dimensional measurement data.

[0076] Here, since the marker 19 is not placed in the facility, the second generation unit 17 does not need to perform processing to remove the data corresponding to the marker 19 when storing the three-dimensional measurement data measured by the second measurement unit 22 in the measurement information.

[0077] FIG. 8 shows another example of the display on the display unit 23 of the measuring device 21 when performing measurements from the second time onward.

[0078] The display unit 23 displays the current image being captured by the camera 25. In FIG. 8 , the current image is indicated by a solid line. The display unit 23 displays an image of the surrounding area corresponding to the position of the marker 19, which was captured in the past by the camera 25 when the marker 19 was placed, superimposed on the current image, for example, by a transparent display. In FIG. 8 , the past image is indicated by a dashed line. At this time, the display unit 23 may remove the marker 19 from the past image and display it superimposed on the current image. Alternatively, the display unit 23 may display a past image captured without the marker 19 placed superimposed on the current image. The worker aligns the measurement device 21 by adjusting the position and orientation of the measurement device 21 so that images of facility structures and the like surrounding the position corresponding to the marker 19 are superimposed on the current image and the past image displayed on the display unit 23.

[0079] The three-dimensional measurement data for each section measured in this manner is managed in the measurement information of the facility map data in a tree structure such as that shown in FIG.

[0080] In FIG. 9 , the root of the tree structure, "base," represents the reference coordinate of the data in the graph structure representing the logical structure of the facility. A node in the tree structure, "markerX," represents a position corresponding to one of the markers 19, with X representing an identification code using a number or the like. A link in the tree structure, "linkX," represents an edge connecting nodes in the graph structure representing the logical structure of the facility. The link "linkX" corresponds to a section of the facility. In this example, each section corresponds to an aisle of the facility. For example, the link "link1" corresponds to the section "aisle #1."

[0081] In this example, the node "marker3" and the node "marker4" represent the same position corresponding to the same marker 19. In this way, by associating different nodes in the tree structure with the same marker 19, it is possible to represent a circular route for a facility on the tree structure.

[0082] 9, "jointX" represents a coordinate transformation. The coordinate transformation is represented by parameters such as the direction and distance of translation and the axis and angle of rotation. Furthermore, "dataX" represents three-dimensional measurement data, such as point cloud data measured for one of the sections.

[0083] In this example, the reference coordinates of the measurement information are set based on one of the markers 19. For example, the reference coordinates of the measurement information are set based on the marker 19 corresponding to the node "marker0". The coordinate conversion from the reference coordinates of the graph structure data representing the logical structure of the facility to the reference coordinates of the measurement information is represented by "joint0".

[0084] The data "data1_0" represents the initial three-dimensional measurement data obtained by placing a marker 19 and measuring the section "aisle #1" with the measuring device 21. The coordinate transformation "joint1_0" represents the coordinate transformation between the coordinates of the measuring device 21 when the detection unit 26 detects the marker 19 corresponding to the node "marker0" when measuring the data "data1_0" and the reference coordinates of the measurement information. The parameters of this coordinate transformation are determined by the second generation unit 17 or the like based on, for example, the relative position of the marker 19 measured by the measuring device 21 and the measuring device 21 when the detection unit 26 detects the marker 19 when measuring the data "data1_0" and the orientation of the measuring device 21. The coordinate transformation when placing the marker 19 and performing measurement is determined based on the reference coordinates of the measurement information.

[0085] The data "data1_1" represents the second or subsequent three-dimensional measurement data of the section "aisle #1" measured by the measuring device 21 without placing a marker 19. The coordinate transformation "joint1_1" represents a coordinate transformation between the coordinates of the measuring device 21 when the detection unit 26 virtually detects the marker 19 corresponding to the node "marker0" when measuring the data "data1_1" and the coordinates of the measuring device 21 when the detection unit 26 detects the marker 19 when measuring the data "data1_0". The parameters of this coordinate transformation are determined by the second generation unit 17 or the like based on, for example, the relative position of the marker 19 measured by the measuring device 21 and the measuring device 21 and the attitude of the measuring device 21 when the detection unit 26 virtually detects the marker 19 when measuring the data "data1_1". The coordinate transformation when performing measurement without placing the marker 19 is determined based on the coordinates after the coordinate transformation when performing measurement with the marker 19 placed.

[0086] Reflecting the above structure, the measurement information of the facility map data has a tree structure as shown in FIG. 10, for example.

[0087] 10, "map definition" represents a tree structure corresponding to the logical structure of the facility, similar to the tree structure shown in FIG. 9. In this example, the nodes "marker3" and "marker4" are associated with each other by, for example, the node identifiers so that they represent the same position corresponding to the same marker 19.

[0088] The 3D measurement data acquired through the circular route can be corrected for distortions using "loop closure," which corrects for errors in continuous coordinate transformations using optimization calculations based on the results of measuring a single object from different directions.

[0089] Each link corresponds to a section of the facility. For example, as in FIG. 9 , link "link1" corresponds to section "aisle #1." In this case, the three-dimensional measurement data measured for section "aisle #1" each time is stored in a tree structure having coordinate transformation "joint1_0" determined based on the reference coordinates of the measurement information as a parent node. The three-dimensional measurement data "data1_0" measured based on the coordinates after coordinate transformation "joint1_0" is stored as data corresponding to the coordinate transformation "joint1_0." Furthermore, coordinate transformation "joint1_1" determined based on the coordinates after coordinate transformation "joint1_0" is set as a descendant node of coordinate transformation "joint1_0." The three-dimensional measurement data "data1_1" measured based on the coordinates after coordinate transformation "joint1_1" is stored as data corresponding to the coordinate transformation "joint1_1." Similarly, three-dimensional measurement data "data1_2" measured separately from data "data1_1" by the measurement device 21 without placing a marker 19 is stored as data corresponding to the coordinate transformation "joint1_2" performed when measuring the data. In this case, the coordinate transformation "joint1_2" is set as a descendant node of the coordinate transformation "joint1_0", similar to the coordinate transformation "joint1_1". Using an identification code such as a number as X and its sub-number as Y, a link "linkX" stores a set of the corresponding three-dimensional measurement data "dataX_Y" and the coordinate transformation "jointX_Y". The three-dimensional measurement data "dataX_0" contains relative position information of the two markers 19 contained in the three-dimensional measurement data "dataX_0". The coordinate transformation "jointX_0" contains the coordinate transformation of the first marker in the three-dimensional measurement data "dataX_0" to the reference coordinates. From this, the link "linkX" can acquire the reference coordinates of the first marker of the three-dimensional measurement data "dataX_0".

[0090] Similarly, for the section "aisle #2" corresponding to the link "link2", the section "aisle #3" corresponding to the link "link3", and the section "aisle #4" corresponding to the link "link4", the three-dimensional measurement data measured for each section is stored in a tree structure.

[0091] In this way, in the measurement information of facility map data, the 3D measurement data measured for each section is managed in a tree structure linked via coordinate transformation. By sequentially applying coordinate transformation while scanning the tree structure in the depth direction from the reference coordinate, it becomes possible to more accurately match the coordinate systems of the 3D measurement data measured independently for each section. This allows the data of the range including the position of the same marker 19 to overlap in the 3D measurement data measured individually for each section. By searching the relative coordinates of the marker 19 included in the link "linkX" in the "map definition" in the depth direction, it is possible to identify the absolute coordinates of the marker 19 relative to the reference "base." In the section "aisle #X" corresponding to the link "linkX," the three-dimensional measurement data "dataX_Y" is transformed and integrated using the coordinate transformation "jointX_Y." This makes it possible to calculate set operations such as sum, difference, product, and interpolation for each piece of three-dimensional measurement data in the link "linkX" using the coordinate transformation "jointX_0" as the base. In this way, by simultaneously storing data and coordinate transformation in the link "linkX" of the "map definition," any three-dimensional measurement data "dataX_Y" can be calculated as information linked to the base "base." This makes it possible to more accurately match information with different measurement conditions and measurement methods.

[0092] The first generation unit 12 generates map information to be distributed to the mobile object 2 by integrating the 3D measurement data measured for each section based on facility map data that reflects the logical and physical structures of the facility. The first generation unit 12 integrates the 3D measurement data by, for example, sequentially applying coordinate transformation to each 3D measurement data for each section based on the facility map data while scanning the tree structure from the reference coordinates in the depth direction. The first generation unit 12 may use, for example, averaging, pattern matching, or other statistical processing to generate the map information used by the mobile object 2. The first generation unit 12 generates map information to be distributed to the mobile object 2 across multiple sections. The first generation unit 12 generates map information for the entire facility based on facility map data that includes the 3D measurement data measured for each section, for example. Each of the sections corresponding to multiple edges branching off from a node in the graph structure is an example of a first section and a second section included in the map information generated by the first generation unit 12. For example, for marker 19 corresponding to node "marker0", any of partitions "aisle #1", "aisle #2", and "aisle #3" is an example of the first partition, and any of the others is an example of the second partition.

[0093] Next, an example of a case where a sub-marker 20 is detected during measurement of three-dimensional measurement data will be described with reference to Fig. 11. Fig. 11 is a diagram showing an example of a section in which a sub-marker 20 is arranged in the mobile body system 1 according to the first embodiment.

[0094] In this example, two sub-markers 20 are placed in the section "aisle #1." In this example, three-dimensional measurement data "data1_1" is measured based on the coordinates after the coordinate transformation "joint1_1." At this time, the marker 19 is not placed. The worker measures the three-dimensional measurement data for the section "aisle #1" while capturing images of the environment around the measurement device 21 with the camera 25.

[0095] While the measurement device 21 moves through the section "aisle #1", the camera 25 captures an image of one of the sub-markers 20 placed in that section. The detection unit 26 identifies and detects the sub-marker 20 based on the captured image of the sub-marker 20. When the detection unit 26 detects the sub-marker 20, the sensor 24 measures the attitude of the measurement device 21. The sensor 24 also measures the distance to the sub-marker 20 as a relative position. Thereafter, the camera 25 captures an image of the other sub-marker 20 placed in that section. The detection unit 26 identifies and detects the sub-marker 20 based on the captured image of the sub-marker 20. When the detection unit 26 detects the sub-marker 20, the sensor 24 measures the attitude of the measurement device 21. The sensor 24 also measures the distance to the sub-marker 20 as a relative position.

[0096] When the detection unit 26 detects a sub-marker 20 while measuring three-dimensional measurement data for a certain section, the second generation unit 17 stores, for example, the three-dimensional measurement data based on a position corresponding to the sub-marker 20 in the facility map data. The second generation unit 17 stores, for example, the three-dimensional measurement data based on a position corresponding to the sub-marker 20 as a sublink of the link corresponding to the section.

[0097] In this example, data "data1_1_1" represents three-dimensional measurement data for section "aisle #1" based on a position corresponding to the sub-marker 20 detected during measurement of data "data1_1." Data "data1_1_1" represents, for example, three-dimensional measurement data from when one sub-marker 20 is detected to when the other sub-marker 20 is detected. Coordinate transformation "joint1_1_1" represents a coordinate transformation between the coordinates of the measuring device 21 when the sub-marker 20 is detected and the coordinates after the coordinate transformation "joint1_1" when measuring data "data1_1." The parameters of this coordinate transformation are determined by the second generating unit 17 or the like based on, for example, the relative position between the sub-marker 20 and the measuring device 21 measured by the measuring device 21 and the attitude of the measuring device 21 when the detecting unit 26 detects the sub-marker 20.

[0098] In the measurement information, the three-dimensional measurement data "data1_1_1" is stored as data corresponding to the coordinate transformation "joint1_1_1" performed when measuring the data. At this time, the coordinate transformation "joint1_1_1" is set as a descendant node of the coordinate transformation "joint1_1".

[0099] Here, only one sub-marker 20 may be placed in each section of the facility. In this case, the three-dimensional measurement data when the sub-marker 20 is detected in a certain section may represent three-dimensional measurement data from the detection of the sub-marker 20 until the actual or virtual detection of the marker 19 in that section, or may represent three-dimensional measurement data from the actual or virtual detection of the marker 19 in that section until the detection of the sub-marker 20. Furthermore, sub-markers 20 may not be placed in some or all sections of the facility.

[0100] The second generating unit 17 may correct the coordinate transformation parameters when measuring the three-dimensional measurement data for a certain section, based on the relative position between the sub-marker 20 and the measuring device 21 measured by the measuring device 21 when the detecting unit 26 detects the sub-marker 20 in the section. The second generating unit 17 corrects the coordinate transformation parameters, for example, based on the position and orientation of the detected sub-marker 20. Because the sub-marker 20 is permanently installed at a corresponding position, such as on a wall surface, in the facility, this position and orientation can be used as the reference.

[0101] The facility structure data may have more nodes than the markers 19. For example, a sub-marker 20 may correspond to a node. When a node corresponds to both a marker 19 and a sub-marker 20, the facility map data may manage a link for an edge connecting the node of the marker 19 and the node of the sub-marker 20. For example, the coordinates of the sub-marker 20 are relative information with respect to the three-dimensional measurement data "dataX_Y" measured from the marker 19, and are saved together with a coordinate transformation "jointX_Y_Z." Here, Z represents a branch number. The relative positions of the marker 19 and the sub-marker 20 are converted into the coordinates of the three-dimensional measurement data "dataX_Y" using the coordinate transformation "jointX_Y_Z," and then further converted into the coordinates of the three-dimensional measurement data "dataX" using the coordinate transformation "jointX_Y." This causes the relative information between the marker 19 and the sub-marker 20 to be linked to the reference "base," making it possible to add a new sub-marker 20 to the "map definition." While the marker 19 is information temporarily placed to configure the facility map data, the sub-marker 20 is permanently placed at the facility. Therefore, by using the sub-marker 20 as the starting point, it is possible to add new links to the "map definition" of the facility map data. As a result, the structure of the desired facility can be set using fewer markers 19 than the number of nodes handled in the facility structure data.

[0102] Furthermore, the detection unit 26 may detect two or more markers 19 in an image captured with the camera 25 by a worker performing measurement work for three-dimensional measurement data with the markers 19 temporarily placed. The detection unit 26 saves the results of identifying two or more markers 19, and the second generation unit 17 stores the three-dimensional measurement data measured by the sensor 24 for the sections of the first marker 19 and the last marker 19, and transmits the data to the facility server 13 via communication. Furthermore, when two or more markers 19 are detected, the second generation unit 17 divides the three-dimensional measurement data so that each section simultaneously includes two markers 19, and identifies edges corresponding to the divided sections. The second generation unit 17 stores the divided information of the three-dimensional measurement data transmitted from the measurement device 21 as measurement information corresponding to the edge. For example, in the case shown in FIG. 10 , when data "data1_0" includes information about markers 19 corresponding to nodes "marker0" and "marker1," data "data1_0" is stored in link "link1" configured from nodes "marker0" and "marker1." Here, when the data "data1_0" further includes information about marker 19 corresponding to node "marker1'," a portion of the data "data1_0" may be stored in link "link1'" configured from nodes "marker0" and "marker1'." Alternatively, in this case, another portion of the data "data1_0" may be stored in another link configured from nodes "marker1'" and "marker1."

[0103] As described above, the map generation system according to the first embodiment is a system that generates map information to be used by a mobile object 2. The map information is used to reference the location within a facility of a mobile object 2 moving within the facility. The facility includes multiple sections. The map generation system includes multiple markers 19, a measurement device 21, a first generation unit 12, and a second generation unit 17. Each marker 19 has a three-dimensional shape. The markers 19 are temporarily placed at corresponding positions within the facility. The measurement device 21 is a portable device that includes a second measurement unit 22 and a detection unit 26. In a section including a position corresponding to a marker 19, the second measurement unit 22 measures three-dimensional measurement data of the surroundings of the section. The detection unit 26 identifies and detects each marker 19. The first generation unit 12 and the second generation unit 17 cooperate with each other to function as a generation unit. The first section and the second section are different sections, each including a position corresponding to a certain marker 19. The generation unit generates map information of the facility across multiple sections, including the first section and the second section, by combining the different three-dimensional measurement data measured by the second measurement unit 22 for each of the first section and the second section, using the position of the marker 19 detected by the detection unit 26 as a reference.

[0104] With this configuration, 3D measurement data such as point cloud data can be divided and measured for each section of a facility. Furthermore, the divided and measured 3D measurement data is integrated across multiple sections including the marker 19, using the marker 19 as a reference. This allows the measurement of 3D measurement data for a facility to be divided and performed for each section, enabling measurement work to be performed without restricting access to the entire facility by facility users. This reduces management costs, such as generating map information for the facility used by the mobile object 2. Furthermore, because 3D measurement data can be measured for each section, when a change occurs in a section, only that section can be measured and updated, keeping the map information of the facility up to date. This reduces management costs, such as maintaining map information. Furthermore, when the entire facility is continuously measured, errors can accumulate during movement, causing distortion. In contrast, because 3D measurement data can be measured for each section, the accumulation of errors and the generation of distortion are suppressed. Furthermore, 3D measurement data is measured from different directions at branching points and junctions in the facility, and map information is generated by integrating this data. This reduces blind spots when measuring three-dimensional measurement data, and improves the accuracy of map information.

[0105] In addition, the generation unit removes the data corresponding to the marker 19 detected by the detection unit 26 from the three-dimensional measurement data measured by the second measurement unit 22 in each of the first and second sections, and then generates map information of the facility across multiple sections including the first and second sections.

[0106] With this configuration, when the mobile body 2 is in operation, data of markers 19 that are not placed in the facility are removed from the map information for the mobile body 2, so that the mobile body 2 can move naturally through the facility where no markers 19 are placed.

[0107] The generation unit also performs coordinate transformation on one or both of the three-dimensional measurement data measured by the second measurement unit 22 in each of the first and second sections so that the data overlaps with the data in the range including the position of the marker 19 detected by the detection unit 26. The generation unit then generates map information of the facility across multiple sections including the first and second sections.

[0108] With this configuration, the coordinates of the individual measurement data are transformed so that they overlap at the common portions, so that the three-dimensional measurement data measured for each section at each time is aligned with higher accuracy.

[0109] Furthermore, the second measurement unit 22 measures the relative position between the marker 19 detected by the detection unit 26 and the measurement device 21, and the orientation of the measurement device 21. The generation unit determines the parameters of the coordinate transformation based on the relative position between the marker 19 and the measurement device 21 and the orientation of the measurement device 21 when the detection unit 26 detects the marker 19 when measuring the three-dimensional measurement data for each section.

[0110] With this configuration, coordinate transformation between each section is set with the marker 19 as the reference, so that the three-dimensional measurement data measured for each section can be matched with higher accuracy.

[0111] The map generation system also includes a plurality of sub-markers 20. The sub-markers 20 are permanently installed at corresponding positions in the facility. The detection unit 26 identifies and detects each of the sub-markers 20. The second measurement unit 22 measures the relative position of the measurement device 21 and the sub-marker 20 detected by the detection unit 26. The generation unit corrects coordinate transformation parameters based on the relative position of the sub-marker 20 and the measurement device 21 and the attitude of the measurement device 21 when the detection unit 26 detects the sub-marker 20 when measuring three-dimensional measurement data for the first section.

[0112] With this configuration, the coordinate transformation parameters are corrected based on the sub-markers 20 permanently installed in the facility, thereby improving the accuracy of coordinate transformation between each section. This allows the three-dimensional measurement data measured for each section to be aligned with higher accuracy.

[0113] The map generation system also includes a facility information management unit 16. The facility information management unit 16 manages graph data of the facility, including a plurality of nodes corresponding to positions where markers 19 are placed in the facility, and edges connecting the plurality of nodes. A mobile object 2 can pass between nodes connected by edges in the graph data of the facility. The first section and the second section correspond to different edges that start or end at the same node in the graph data. The generation unit generates map information of the facility across a plurality of sections, including the first section and the second section, based on the graph data.

[0114] With this configuration, the 3D measurement data obtained for each section is reconstructed into graph data that reflects the logical structure of the facility, enabling more accurate generation of map information across multiple sections.

[0115] The second measurement unit 22 also includes a camera 25 that captures images. The detection unit 26 detects the marker 19 based on the image of the marker 19 captured by the camera 25. The detection unit 26 virtually detects the marker 19 corresponding to the position based on an image of the surrounding area of ​​the position corresponding to the marker 19 captured by the camera 25 when the marker 19 is not placed.

[0116] With this configuration, when three-dimensional measurement data is to be measured for a certain section multiple times, the measurement work can be performed without placing the marker 19 each time. This reduces the workload of the measurement work. It also reduces the occurrence of errors due to misalignment of the marker 19.

[0117] The measuring device 21 also includes a display unit 23. The display unit 23 displays an image of the surroundings of the position corresponding to the marker 19, which was previously captured by the camera 25 when the marker 19 was placed, and the current image being captured by the camera 25, in an overlapping manner.

[0118] With this configuration, the worker performing the measurement work can visually align the measuring device 21 using the display on the display unit 23. This allows the worker to perform the measurement work more easily.

[0119] Furthermore, each marker 19 is provided with features that enhance the accuracy of detection by the detection unit 26. For example, the marker 19 has a portion with a constant positive or negative curvature on at least a portion of its surface. A shape with a constant curvature, such as a spherical surface, allows the detection unit 26 to detect the marker 19 with higher accuracy than other complex shapes. For example, the marker 19 has a periodic pattern with discrete gradations on at least a portion of its surface. Such a pattern allows the surface shape of the marker 19 to be clearly detected. Furthermore, the marker 19 does not include, for example, a surface with metallic luster or a mirror surface. This reduces the occurrence of detection errors due to reflection.

[0120] When measuring three-dimensional measurement data, the display unit 23 may highlight sections that are more likely to be measured and display them to the worker. Sections that are more likely to be measured may be highlighted, for example, by a color map that is overlaid on a map image of the facility. Sections that are more likely to be measured include sections that have been measured less frequently, sections where the most recent measurement was performed earlier, and sections that have been changed since the most recent measurement. This allows the worker to visually grasp sections that require three-dimensional measurement data from the display on the display unit 23.

[0121] The facility information management unit 16 of the map generation system has a function of setting "linkY" in the branch structure of the facility map data for two nodes "markerY1" and "markerY2" that are on different floors, and registering an edge corresponding to "linkY" in the facility structure data. Here, two corresponding nodes in the facility structure data each store data representing a floor. In other words, two nodes that are at the same position on the floor surface of a facility but different in height are identified as two different nodes, "markerY1" and "markerY2," even if the same marker is placed thereon. In addition, a coordinate transformation "jointY" is generated for "markerY1" and "markerY2" by referencing floor information.

[0122] Two nodes at different heights are, for example, markers placed inside an elevator car. Because elevators move up and down in the vertical direction, a node in the facility map data is set for each floor. In this case, the "jointY" corresponding to "linkY" is described as a coordinate transformation that translates in the vertical direction by the distance traveled by the elevator. Alternatively, the difference in the height direction of the floors is obtained from the drawing and a coordinate transformation that translates in the vertical direction is used. In this case, the worker performing the measurement takes the measurement with the elevator car door open.

[0123] The facility information management unit 16 of the map generation system may set the root "base" of the tree structure of the facility map data to the inside of the elevator car or the car door. In this case, the worker performing the measurement will measure with the elevator car door open.

[0124] Because facilities are divided into floors by ceilings and floors, it is technically difficult to acquire 3D point clouds inside the facility. For this reason, 3D point clouds are acquired including areas where multiple floors are open in the vertical direction, such as staircases and voids. However, since it is not possible to correct errors due to loop confinement, if an error occurs in the tilt direction relative to the floor, a large error will occur in the horizontal direction. 3D point clouds have the issue of easily accumulating errors in the tilt direction when measuring in places where there is no reference plane such as a floor or ceiling.

[0125] Elevators are constructed with precision to maintain perpendicularity to the facility, so facility map data can be created by accurately compiling the height direction of the 3D point clouds measured on each floor. For this reason, coordinate transformation based on the elevator is suitable for use as a map for robots that move within the same plane. However, elevators have slight errors in the height direction each time a passenger gets on or off. When the doors are closed, there are areas where the reference plane is not continuous before and after the measurement, which causes errors. When performing coordinate transformation based on the elevator, it is important that the worker taking the measurements measures with the elevator doors open.

[0126] Because elevator doors close after a set period of time, it is difficult for a robot to measure multiple sections simultaneously with the elevator car open. Also, because the robot is restricted from moving vertically in the facility while generating a map, the method of using the robot to convert coordinates based on the elevator places a heavy burden on the facility and is costly.

[0127] Elevators are required by law to undergo periodic maintenance inspections, allowing the doors to remain open. Furthermore, during maintenance inspections, elevators cannot be called from other floors. Therefore, the map generated by workers will not interfere with the operation of the building, even if they place markers inside the elevator car for an extended period of time.

[0128] Two nodes at different heights could be, for example, markers placed at the entrance or exit of an escalator. The escalator is installed accurately relative to the drawing. The escalator's housing has a structure independent of the building. The escalator is a piece of equipment that moves parallel to the floor, and by fixing a measuring device to the steps, it is possible to obtain accurate 3D measurement data in the height direction. In this case, the "jointY" corresponding to "linkY" is described by a coordinate transformation that translates the escalator's travel distance in the longitudinal and height directions.

[0129] Two nodes that are different in the height direction may be set as a pair of sub-markers, for example. However, a pair of sub-markers is equipment that has the characteristic of being calibrated in the height direction. For example, a pair of sub-markers is a combination that can be observed simultaneously in an atrium, etc. Alternatively, it is a combination in which the installation positions are calibrated for structures with the same height direction, such as pillars.

[0130] Two nodes that differ in the height direction may be set as a pair of marker position information output by an imaging device calibrated in the height direction, for example. However, facility equipment is defined as equipment that has the feature of being able to calibrate distortion caused by construction, such as by simultaneously measuring and calibrating the same sub-markers installed in a facility. For example, the result is an imaging device calibrated with sub-markers that can be observed from multiple floors, such as in an atrium, outputting position information for different markers placed on different floors. In this case, the "jointY" corresponding to "linkY" is calculated by using the sub-marker used for calibration as the root "base" of the tree structure, and then calculating the "jointY" of "markerY1" and "markerY2."

[0131] The two nodes with different heights are set by the equipment installed in the facility using a function that can identify the height and estimate the horizontal deviation, and are not specified in terms of equipment such as elevators or escalators, or algorithms such as sub-markers or imaging equipment.

[0132] Embodiment 2 In Embodiment 2, differences from the example disclosed in Embodiment 1 will be described in particular detail. For features not described in Embodiment 2, any of the features of the example disclosed in Embodiment 1 may be adopted.

[0133] FIG. 12 is a configuration diagram of a mobile system 1 according to the second embodiment.

[0134] The mobile body system 1 includes a support server 27. The support server 27 is a part that supports the generation of map information used by the mobile body 2. The support server 27 is, for example, a server device consisting of one or more server computers. Part or all of the support server 27 may be included in the facility server 13 or the mobile body server 10 that function as a map generation device in the mobile body system 1. The support server 27 may be an external device to the mobile body system 1. The multiple server devices that make up the support server 27 may be located in different locations. In this case, the multiple server devices communicate information with each other, for example, via a communication network 8. The support server 27 includes a calculation unit 3e, a memory unit 4e, and a communication unit 5e.

[0135] The calculation unit 3e is a device such as a CPU, an arithmetic device, a microprocessor, or a microcomputer. The storage unit 4e is a device such as a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, an EPROM, or an EEPROM, or a device such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD. The calculation unit 3e and part or all of the storage unit 4e may be configured with a dedicated processing circuit. The storage unit 4e stores, for example, programs as software or firmware. In the support server 27, the calculation unit 3e executes the programs stored in the storage unit 4e to perform pre-set processing, and each function is realized as a result of collaboration between hardware and software. Each function of the support server 27 may be realized by a separate processing circuit. Alternatively, some or all of the functions of the support server 27 may be realized collectively by a processing circuit. The processing circuit may also be realized by, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. Some or all of the functions of the support server 27 may be implemented, for example, by processing or storage resources on a cloud service.

[0136] The communication unit 5e is a part equipped with a function for communicating information with devices external to the support server 27. The communication unit 5e communicates with external devices via the communication network 8, for example, by wired communication or wireless communication. The communication unit 5e communicates information with external devices, for example, the facility server 13 or the mobile server 10.

[0137] The calculation unit 3e includes a learning unit 28. The learning unit 28 is a part that outputs data corresponding to input data based on a trained model. The learning unit 28 learns the trained model based on training data. The learning unit 28 has multiple phases including a learning phase and an inference phase. In the learning phase, the learning unit 28 accepts input of training data and learns the trained model. The learning unit 28 learns the trained model, for example, by supervised learning, unsupervised learning, reinforcement learning, or other machine learning algorithms. Data of the trained model is stored in, for example, the memory unit 4e. In the inference phase, the learning unit 28 uses the trained model to output data corresponding to the input data.

[0138] FIG. 13 is a block diagram showing an example of learning and inference by the learning unit 28 according to the second embodiment.

[0139] In this example, the trained model receives facility structure data that does not include 3D measurement data and outputs facility map data that includes 3D measurement data. The facility structure data includes attributes of the positions in the facility that correspond to the markers 19 and information on the connections between the positions.

[0140] The training data may be, for example, data including multiple pairs of input data and corresponding output data. In this example, the training data may be data including multiple pairs of facility structure data and corresponding facility map data. The facility map data corresponding to the facility structure data is generated based on the facility structure data and 3D measurement data actually measured by the measurement system 18 for each section. The facility map data is generated, for example, by the second generation unit 17. The training data that combines the facility structure data with the corresponding facility map data is generated, for example, by the second generation unit 17. At this time, the facility server 13 outputs the training data to the learning unit 28 via the communication unit 5c or the like. The training data that combines the facility structure data with the corresponding facility map data may be generated by the learning unit 28. At this time, the support server 27 acquires the facility structure data and the corresponding facility map data from the facility server 13 via the communication unit 5e or the like. In the learning phase, the learning unit 28 learns a trained model using the training data, for example, by a method such as supervised learning.

[0141] In the inference phase, the learning unit 28 accepts input of facility structure data. The learning unit 28 accepts input of facility structure data from, for example, the second generation unit 17. The learning unit 28 uses the trained model to output facility map data corresponding to the input facility structure data. The support server 27 transmits the facility map data output by the learning unit 28 to the second generation unit 17 via the communication unit 5e or the like. In this way, the second generation unit 17 generates facility map data including three-dimensional measurement data such as point cloud data by inputting the facility structure data to the learning unit 28 via the communication unit 5c or the like.

[0142] This configuration allows the system to learn the correspondence between the connection relationships and attributes of each location and 3D measurement data such as point cloud data, enabling the learning unit 28 to generate 3D measurement data for features and other objects that are expected to be located at specific locations within a facility based on their attributes. For example, in facility structure data, the attributes of a location may include information about the purpose of a room that includes the location. Consider a case where the attribute of a room's location in the facility structure data has been changed from "archive" to "conference room." At the location with the purpose "archive," features such as bookshelves are expected to be located at specific locations based on the attributes. Similarly, at the location with the purpose "conference room," features such as desks are expected to be located at specific locations based on the attributes. In this case, the learning unit 28 receives input of facility structure data in which the attributes of a location have been changed to "conference room," and outputs facility map data including 3D measurement data that reflects the placement and shape of features such as desks that are not included in drawings of the facility itself.

[0143] The trained model may take facility map data including 3D measurement data as input and output facility map data including corrected 3D measurement data. In this case, the input facility map data may include, for example, attributes of each position in the facility to which the markers 19 correspond and information on the connection relationships between the positions.

[0144] For example, when the learning unit 28 receives input of facility structure data in which the attribute of a certain location has been changed from use "archive" to use "conference room," it outputs corrected facility map data that includes 3D measurement data that reflects the arrangement and shape of features such as desks that are not included in the drawings of the facility itself, after excluding 3D measurement data of features such as bookshelves that are not included in the drawings of the facility itself.

[0145] In addition, the trained model may take facility map data including three-dimensional measurement data including markers 19 placed at the facility as input, and output facility map data including three-dimensional measurement data that has been modified so that the portion corresponding to the markers 19 is removed.

[0146] The trained model may also be one that receives as input a set of 3D measurement data of the entire facility including markers 19 placed at the facility or map information for the mobile object 2 and data representing the physical structure of the facility, such as a CAD model, and outputs data representing a logical structure, such as the connection relationships between facility positions. Because the markers 19 are not included in the physical structure of the facility, the location and shape of each marker 19 can be easily extracted from the difference between the 3D measurement data of the entire facility including markers 19 placed at the facility and the data representing the physical structure, such as a CAD model of the facility. As a result, data representing the logical structure of the facility is generated by the learning unit 28 using the 3D measurement data of the entire facility.

[0147] The trained model may also be configured to receive a set of facility structure data and data on measurement conditions for measurements by the second measurement unit 22, and output facility map data including three-dimensional measurement data. The measurement conditions include, for example, information on the time when the measurement is performed and the operating status of facility equipment in the facility when the measurement is performed. The facility equipment includes, for example, elevators, escalators, and other elevators, and access control devices such as automatic doors and security gates.

[0148] Embodiment 3. In embodiment 3, differences from the examples disclosed in embodiment 1 and embodiment 2 will be described in particular detail. Features not described in embodiment 3 may be any of the features of the examples disclosed in embodiment 1 and embodiment 2.

[0149] A large-scale language model (LLM) is installed in the learning unit 28 of the support server 27 of embodiment 3. The large-scale language model is an example of a language model. Data of the large-scale language model is stored, for example, in the memory unit 4e. The large-scale language model is trained in advance so that it can interpret natural language. The large-scale language model is also trained in advance so that it can interpret graph languages, which are formal languages ​​that describe graph structures. In this example, the facility structure data is described in a graph language that can be interpreted by the large-scale language model. The large-scale language models may be trained in advance for multiple types of natural languages ​​and graph languages. The large-scale language models may be trained in advance so that they can interpret other formal languages. The large-scale language models may be trained in advance so that they can interpret drawings or other data.

[0150] The learning unit 28 receives input of advance explanations using natural language and formal language, thereby learning a correspondence model in advance. The correspondence model is a model that associates a description of facility structure data in natural language with a description of facility structure data in a formal language including a graph language. Information about the correspondence model is stored in, for example, the storage unit 4e. The correspondence model is, for example, an intermediate representation used internally in the learning unit 28.

[0151] The preliminary explanation is input, for example, from the facility information management unit 16 of the facility server 13. The preliminary explanation is input, for example, when starting operation of the measurement system 18. The preliminary explanation includes common information such as definitions of terms used in the measurement system 18, regardless of the facility to which the measurement system 18 is applied. The preliminary explanation is written, for example, in natural language.

[0152] In this example, the advance explanation to the learning unit 28 includes definitions of terms for the facility structure data. The advance explanation may include a description of a method for expressing elements such as nodes or edges on the facility structure data in natural language. The advance explanation may include a description of naming rules for elements on the facility structure data. The advance explanation to the learning unit 28 also includes definitions of terms for each function of the measurement system 18. The advance explanation to the learning unit 28 also includes descriptions of the structures of each piece of data, such as the facility structure data, facility map data, and 3D measurement data, and the correspondence between them. The advance explanation may include information on the generation rules for each piece of data. Part of the advance explanation may be written in a graph language, a modeling language, or other formal language or data that can be interpreted by a large-scale language model.

[0153] The learning unit 28, which has previously learned the correspondence model, receives a management document written in natural language and generates facility structure data written in formal language. The management document is a document that describes the structure of the facility and the placement of markers 19 in natural language. The management document is, for example, a work report written by a worker who placed the markers 19 and performed measurement work. The management document is input to the second generation unit 17 together with three-dimensional measurement data, for example, from the measurement device 21.

[0154] The second generation unit 17 inputs the input management document to the learning unit 28 via the communication unit 5c or the like. The learning unit 28 interprets the management document using a large-scale language model. The learning unit 28 generates facility structure data described in a formal language based on the structure of the facility and the arrangement of the markers 19 described in the management document. The learning unit 28 outputs the generated facility structure data to the second generation unit 17 via the communication unit 5c of the facility server 13 or the like.

[0155] The second generation unit 17 generates facility map data including measurement information based on the facility structure data generated by the learning unit 28 and the three-dimensional measurement data measured by the measuring device 21. The second generation unit 17 distributes the generated facility map data to the mobile server 10. In the mobile server 10, the first generation unit 12 converts the facility map data into map information used by the mobile objects 2. The mobile server 10 distributes the converted map information to each mobile object 2.

[0156] Next, an example of facility structure data generated by the learning unit 28 will be described with reference to Fig. 14 and Fig. 15. Fig. 14 is an example of a plan view of a facility to which the mobile body system 1 according to the third embodiment is applied. Fig. 15 is an example of a graph structure of facility structure data generated for a facility to which the mobile body system 1 according to the third embodiment is applied.

[0157] The prior explanation to the learning unit 28 may suggest, for example, that "when a facility like that shown in FIG. 14 is represented by a graph like that shown in FIG. 15, markers 19 are used as nodes, and edges are associated when markers 19 can travel between markers 19, and the structure is described as a tree with a location within the facility, such as a hall, as the root."

[0158] As shown in FIG. 14 , workers place markers 19 at various locations in the facility and perform measurement work to collect three-dimensional measurement data. The workers prepare a report describing the placement of the markers 19 as a management document. The management document includes, for example, the following sentence: "The first floor of the building has an elevator, stairs, and a hall connecting two Western-style rooms. The living room is connected to the east of the hall. The living room has two windows on the south side. There is a hallway to the east of the living room. The hallway has four Western-style rooms to the north and three Western-style rooms and a toilet to the south. One marker will be placed in the hallway and one in front of the living room window. In addition, one marker will be placed in front of each Western-style room door on the hallway. You can walk between the markers in the Western-style rooms and the markers in front of the doors. The Western-style rooms are lined up from west to east in the order North 1, North 2, South 1, North 3, South 2, South 3, and North 4. You will need to pass by each marker when going from west to east." The management document may include a floor plan as shown in Figure 14. In this case, the management document mentions that the facility has multiple floors and the floor plan is information on the first floor.

[0159] By inputting such management documents, the learning unit 28 generates facility structure data with a graph structure as shown in Fig. 15. The second generating unit 17 generates facility map data including measurement information for multiple sections of the facility based on the facility structure data generated by the learning unit 28 and the three-dimensional measurement data measured by the measuring device 21.

[0160] The learning unit 28 may input facility structure data and output a management document written in natural language. In this case, the management document may be, for example, a work procedure manual to be presented to a worker who places the markers 19 and performs measurement work. Here, the facility structure data input by the learning unit 28 is data that the second generation unit 17 uses when generating facility map data.

[0161] Here, the facility structure data obtained by the learning unit 28 from the management document describing the facility in Figure 14 may yield results different from those shown in Figure 15. For example, if the learning unit 28 does not interpret the phrase "When going from west to east, you must pass near each marker" in the management document, a configuration in which the doors in front of all Western-style rooms connect to the living room will be output. In this case, if the learning unit 28 provides a prior explanation that it did not interpret part of the phrase in the management document, a corrected graph structure will be output.

[0162] Similarly, the management document obtained by the learning unit 28 from the graph language representing the graph in Fig. 15 may have an explanation different from that in Fig. 14. In this case, since there is a gap in the pre-explanation for explaining the management document in the graph language, correcting the incorrect interpretation results in outputting the corrected natural language.

[0163] For facilities consisting of multiple sections, management documents and the like are recorded in multiple sections. Since the learning unit 28 associates management documents with facility structure data, management documents may be divided into two or more adjacent sections and input to the learning unit 28. Since facility structure data differs for each section, the advance explanation may be corrected and input for each section. In this case, when updating information for the same section with facility structure data, the learning unit 28 can obtain correct management documents in natural language by using the advance explanation previously used for the same section.

[0164] Large-scale language models have a limited total amount of input data (tokens) used for inference. When there is a large amount of input data, a problem with large-scale language models is that they output results that do not take into account previously input information. The support server 27 can compress information by using facility structure data to handle management documents entered for each section. This allows the learning unit 28 to output more accurate results in less calculation time using accurate prior information and less input data.

[0165] Rather than generating the same information for the entire facility, it is better to divide the information into sections such as floors or floors and set up separate information for each section to obtain more accurate results. Similarly, areas that change significantly depending on the season can be updated more frequently, reducing the cost of measurement.

[0166] Because large-scale language models are learning models with strong nonlinearities between input and output, even if a document to be output is divided and input correctly, there is no guarantee that the output will be divided as intended. Furthermore, document division generally results in a decrease in accuracy. However, when expressing information about a facility consisting of multiple sections for data (graph structure) that can be defined by a set, such as facility structure data, a one-to-one correspondence is maintained between formal language and natural language for each section. Therefore, by dividing a natural language document representing a facility by restricting it to include two adjacent sections, the learning unit 28 preserves and converts information about the overlapping areas of the facility structure data for the two sections. Therefore, the facility structure data obtained from the original document can be restored based on the results of division and output.

[0167] As described above, the map generation system according to the third embodiment includes a learning unit 28. The learning unit 28 is equipped with a language model that interprets both natural language and formal language capable of describing facility structure data for a facility. The facility structure data includes attributes of areas including positions where markers 19 are placed and information on the connection relationships between the positions where markers 19 are placed. A generation unit consisting of the first generation unit 12 and the second generation unit 17 inputs a management document into the learning unit 28, and generates map information of the facility across multiple sections based on the facility structure data output from the learning unit 28. The management document is a document that describes the structure of the facility and the placement of markers 19 in natural language.

[0168] This configuration allows the generation unit to input the placement of markers 19 and the logical structure of the facility using management documents written in natural language. This allows facility managers who are not technical experts on the mobile body 2, or maintenance workers for the facility's equipment, to perform measurement work to generate 3D measurement data for the facility. For example, maintenance workers for the equipment can perform measurement work during breaks, such as when waiting for maintenance work. This reduces the costs of performing measurement work frequently, thereby maintaining more accurate map information for the facility. This allows measurement work that adds value to the facility during waiting times for maintenance work, thereby further increasing the added value of the work per hour of maintenance workers. Furthermore, for example, a facility manager can simultaneously issue work instructions to workers performing measurement work and set the placement of markers 19 in the mobile body system 1 using natural language. This reduces the likelihood of discrepancies between work instructions and the placement of markers 19, reducing the likelihood of errors in the facility map data.

[0169] The map generation system, map generation device, map generation method, map generation program, and measurement system can be applied to management such as the generation of map information for a facility used by a mobile object operating in the facility.

[0170] DESCRIPTION OF SYMBOLS 1 Mobile body system, 2 Mobile body, 3a, 3b, 3c, 3d, 3e Calculation unit, 4a, 4b, 4c, 4d, 4e Memory unit, 5a, 5b, 5c, 5d, 5e Communication unit, 6 First measurement unit, 7 Driving unit, 8 Communication network, 9 Action control unit, 10 Mobile body server, 11 Action planning unit, 12 First generation unit, 13 Facility server, 14 Input unit, 15 Output unit, 16 Facility information management unit, 17 Second generation unit, 18 Measurement system, 19, 19a, 19b Marker, 20 Sub-marker, 21 Measurement device, 22 Second measurement unit, 23 Display unit, 24 Sensor, 25 Camera, 26 Detection unit, 27 Support server, 28 Learning unit

Claims

1. A map generation system that generates map information used to reference the position within a facility including multiple sections of a mobile object moving through the facility, comprising: a plurality of markers having three-dimensional shapes and temporarily placed at corresponding positions in the facility; a portable measurement device that includes a measurement unit that measures three-dimensional measurement data of the surroundings of a section including the positions of the markers, and a detection unit that identifies and detects each of the markers; and a generation unit that generates map information of the facility across a plurality of sections including the first section and the second section by combining the different three-dimensional measurement data measured by the measurement unit for each of a first section and a second section that are different sections and each include the positions of the markers, using the positions of the markers detected by the detection unit as a reference.

2. The map generation system of claim 1, wherein the generation unit generates map information of the facility across multiple sections including the first section and the second section after excluding data corresponding to the marker detected by the detection unit from the three-dimensional measurement data measured by the measurement unit in each of the first section and the second section.

3. The map generation system according to claim 1 or claim 2, wherein the generation unit performs coordinate transformation on one or both of the three-dimensional measurement data measured by the measurement unit in each of the first and second sections so that data in a range including the position of the marker detected by the detection unit overlaps, and then generates map information of the facility across multiple sections including the first and second sections.

4. The map generation system according to claim 3, wherein the measurement unit measures the relative position between the marker detected by the detection unit and the measurement device, and the attitude of the measurement device; and the generation unit determines parameters of the coordinate transformation performed so that data for a range including the position of the marker overlaps, based on the relative position between the marker and the measurement device and the attitude of the measurement device when the detection unit detects the marker when measuring the three-dimensional measurement data for the first section, and the relative position between the marker and the measurement device and the attitude of the measurement device when the detection unit detects the marker when measuring the three-dimensional measurement data for the second section.

5. The map generation system according to claim 4, comprising: a plurality of sub-markers permanently installed at corresponding positions in the facility; the detection unit identifies and detects each of the sub-markers; the measurement unit measures the relative position between the sub-marker detected by the detection unit and the measurement device; and the generation unit corrects the parameters of the coordinate transformation based on the relative position between the sub-marker and the measurement device and the attitude of the measurement device when the detection unit detects the sub-marker when measuring the three-dimensional measurement data for the first section.

6. A map generation system as described in any one of claims 1 to 5, comprising: a facility information management unit that manages graph data of the facility including a plurality of nodes corresponding to each of the positions where the markers are placed in the facility and edges connecting the plurality of nodes; the nodes connected by the edges in the graph data are passable by the mobile object; the first section and the second section correspond to different edges that start or end at the same node in the graph data; and the generation unit generates map information of the facility across a plurality of sections including the first section and the second section based on the graph data.

7. A map generation system according to any one of claims 1 to 6, comprising: a facility information management unit that manages graph data of the facility, the facility including floors that are vertical divisions and equipped with equipment connecting the floors, the graph data including a plurality of nodes corresponding to the height of each floor at which the marker is placed on the equipment, and edges connecting between the nodes corresponding to each floor of the equipment, wherein the measurement unit measures the three-dimensional measurement data and images when the doors of the equipment are open.

8. A map generation system according to any one of claims 1 to 7, wherein the measurement unit includes a camera that captures an image, and the detection unit detects the marker based on an image of the marker captured by the camera.

9. The map generation system according to claim 8, wherein the detection unit virtually detects the marker corresponding to the position based on an image of the surroundings of the position to which the marker corresponds, captured by the camera when the marker is not placed.

10. A map generation system as described in claim 8 or claim 9, wherein the measuring device includes a display unit that displays an image of the surroundings of the position corresponding to the marker that was previously taken by the camera when the marker was placed, and a current image being taken by the camera, superimposed on the image.

11. A map generation system according to any one of claims 1 to 10, wherein the marker has a portion of constant positive or negative curvature on at least a portion of its surface.

12. A map generation system according to any one of claims 1 to 11, wherein the marker has a periodic pattern of discrete gradations on at least a portion of its surface.

13. A map generation system according to any one of claims 1 to 12, wherein the marker does not include a surface having a metallic luster or a specular surface.

14. A map generation system according to any one of claims 1 to 13, comprising: a learning unit that learns a trained model based on training data including facility structure data including attributes of areas including positions where the markers are placed in the facility and information on the connection relationships between the positions where the markers are placed, and the three-dimensional measurement data measured by the measurement unit for the section, using the facility structure data as input and outputting data including three-dimensional measurement data.

15. The map generation system described in claim 14, wherein the training data includes data on measurement conditions for measurements by the measurement unit, and the trained model receives the facility structure data and data on the measurement conditions as inputs and outputs data including three-dimensional measurement data.

16. A map generation system as described in any one of claims 1 to 13, comprising: a learning unit equipped with a language model that interprets both natural language and a formal language capable of describing facility structure data including attributes of the positions at which the markers are placed in the facility and information on the connection relationships between the positions at which the markers are placed; the learning unit receives input of a prior explanation using the natural language and the formal language, thereby learning in advance a correspondence model that matches the description in the natural language of the facility structure data with the description in the formal language; and the generation unit generates map information of the facility across a plurality of sections including the first section and the second section based on the facility structure data output from the learning unit by inputting a document that describes the structure of the facility and the placement of the markers in the natural language into the learning unit.

17. The map generation system of claim 16, wherein the generation unit generates map information of the facility across multiple sections including the first section and the second section based on the facility structure data output from the learning unit by inputting documents into the learning unit that are obtained by dividing documents describing the structure of the facility and the placement of the markers in natural language based on the facility structure data.

18. A map generation system as described in any one of claims 1 to 13, comprising: a communication unit that is equipped with a language model that interprets both natural language and a formal language that can describe facility structure data, including information on attributes of areas including the positions where the markers are placed in the facility and information on the connection relationships between the positions where the markers are placed; and a communication unit that communicates data with a learning unit that has previously learned a correspondence model that matches the description in natural language of the facility structure data with the description in formal language by inputting a prior explanation using the natural language and the formal language; the communication unit obtains the facility structure data output from the learning unit by inputting a document that describes the structure of the facility and the placement of the markers in natural language into the learning unit; and the generation unit generates map information of the facility across a plurality of sections including the first section and the second section based on the facility structure data obtained by the communication unit from the learning unit.

19. The map generation system described in claim 18, wherein the communication unit acquires the facility structure data output from the learning unit by inputting documents into the learning unit that are obtained by dividing documents that describe the structure of the facility and the placement of the markers in natural language based on the facility structure data.

20. A map generation device that generates map information used by a mobile body moving through a facility including multiple sections to reference its location within the facility, comprising: an acquisition unit that identifies and detects multiple markers that have three-dimensional shapes and are temporarily placed at corresponding positions in the facility, and acquires the measured three-dimensional measurement data from a portable measurement device that measures three-dimensional measurement data of the surroundings of each section in the section including the positions corresponding to the markers; and a generation unit that generates map information of the facility across multiple sections including the first and second sections by combining the different three-dimensional measurement data measured by the measurement device for first and second sections that are different sections and each include the positions corresponding to the markers, using the positions of the markers detected by the measurement device as a reference.

21. A map generation method for generating map information used to reference the position within a facility of a mobile object moving through the facility, the map generation method comprising the steps of: identifying and detecting each of a plurality of markers having three-dimensional shapes and temporarily placed at corresponding positions in the facility; acquiring the measured three-dimensional measurement data from a portable measurement device that measures three-dimensional measurement data of the surroundings of a section including the positions corresponding to the markers; and generating map information of the facility across a plurality of sections including the first and second sections by combining the different three-dimensional measurement data measured by the measurement device for a first section and a second section that are different sections and each include the positions corresponding to the markers, based on the positions of the markers detected by the measurement device.

22. A map generation method according to claim 21, which generates map information used to reference the position within a facility of a mobile object moving through the facility, the facility including floors that are sections in the vertical direction and equipped with equipment connecting the floors, the map generation method comprising: a computer executing the following steps: identifying and detecting markers having three-dimensional shapes and temporarily placed on the equipment in the facility as different markers for each floor, in the vertical direction; acquiring the measured three-dimensional measurement data including the markers attached to the equipment as information including the vertical direction of the floors; acquiring a third section including each of the markers attached to the equipment for a first section in which the three-dimensional measurement data including the markers attached to the equipment was measured and a second section in which three-dimensional measurement data including markers attached to the equipment on a different floor was measured; and generating map information of the facility across a plurality of sections including the first section, the second section, and the third section.

23. A map generation program that causes a computer to generate map information used to reference the position within a facility including multiple sections of a mobile object moving through the facility, the map generation program causing the computer to: identify and detect each of multiple markers that have three-dimensional shapes and are temporarily placed at corresponding positions in the facility, and acquire the measured three-dimensional measurement data from a portable measurement device that measures three-dimensional measurement data of the surroundings of each section in the section including the positions corresponding to the markers; and generate map information of the facility across multiple sections including the first and second sections by combining the different three-dimensional measurement data measured by the measurement device for each of first and second sections that are different sections and each include the positions corresponding to the markers, using the positions of the markers detected by the measurement device as a reference.

24. A measurement system used to generate map information used to reference the position within a facility of a mobile object moving through the facility, the measurement system comprising: a plurality of markers having three-dimensional shapes and temporarily placed at corresponding positions in the facility; and a portable measurement device, wherein the measurement device comprises: a measurement unit that measures three-dimensional measurement data of the surroundings of a section including the positions corresponding to the markers; a detection unit that identifies and detects each of the markers; and a communication unit that outputs the three-dimensional measurement data measured by the measurement unit to a map generation device that generates map information of the facility across a plurality of sections including the first and second sections, each of which includes the positions corresponding to the markers, by combining the different three-dimensional measurement data measured by the measurement unit for a first section and a second section, which are different sections and each include the positions corresponding to the markers, based on the positions of the markers detected by the detection unit.

Images