Information processing system, information processing device, information processing method, and program

Abstract

To provide an information processing system that can perform scale-reducing correction of an environmental map using a single-eye image, without requiring an auxiliary sensor or a marker other than an imaging device.SOLUTION: An information processing system (1) is installed in a moving object (10), and performs position estimation using a first imaging device 20A and a second imaging device 20B, each of which captures a single-eye image. The information processing system 1 includes: a storage unit 69 for storing environmental maps (67, 68) that show a peripheral environment respectively in which the moving object (10) moves; first and second self-position estimating units (60) which estimate self-positions (P1, P2) of the imaging devices 20A, 20B on the environmental maps (67, 68) based on the images captured by the imaging devices 20A and 20B and the environmental maps (67, 68); and a correction unit (61) that generates scale reduction correction information α for the environmental maps (67, 68) based on a distance between self positions (P1, P2) on the environmental maps (67, 68) and a distance D between the first and second imaging devices 20A, 20B.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to information processing technology, and more particularly to an information processing system, an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, there are known technologies (SLAM, Simultaneous Localization and Mapping) that simultaneously estimate the self-location of a moving object and generate an environmental map. Among them, a technology that estimates the self-location and generates an environmental map using images captured by an imaging device as input is referred to as VLSAM (Visual SLAM).

[0003] VLSAM includes technologies that combine a monocular camera with a depth sensor, or that use a stereo camera, as well as technologies that use only monocular images captured by a monocular camera as input. When only monocular images are input, the distance units in the environmental map generated by VLSAM are unclear, and separate scale correction is required to convert them into positions and distances in real space. For this reason, auxiliary sensors other than the imaging device, such as wheel odometry, are generally required for scale correction.

[0004] Japanese Patent Publication No. 2022-015978 (Patent Document 1) is known in relation to the above-mentioned conversion to a position in real space. Patent Document 1 discloses a configuration in which an unmanned aerial vehicle takes off from a position where it can photograph a position marker placed alongside the flight route, determines a reference position of the vehicle based on position information of the photographed position marker, estimates a positional deviation from the reference position while flying along the flight route, and estimates a current flight position of the vehicle. If the accuracy of the position information included in a signal from a position measurement system is lower than a threshold, transmits the estimated position information of the current flight position to a flight controller instead of the position information of the signal. The prior art of Patent Document 1 determines a reference position of the vehicle based on position information of a position marker, and estimates a positional deviation from the reference position while flying along the flight route. However, the prior art of Patent Document 1 assumes that the position marker has position information, and is not capable of completing scale correction by itself. Therefore, an auxiliary sensor other than the imaging device is required, and scale correction processing cannot be performed in an environment where markers cannot be used, so this method is not sufficient. Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure has been made in consideration of the above points, and aims to provide an information processing system that can perform scale correction of an environmental map using a monocular image without requiring any auxiliary sensors or markers other than the imaging device. [Means for solving the problem]

[0006] The present disclosure provides an information processing system that is provided on a moving body and that estimates a position using a first imaging device and a second imaging device, each capturing a monocular image, having the following characteristics: The information processing system includes a storage unit that stores an environmental map that shows a surrounding environment in which the moving body moves, a first self-position estimation unit that estimates a first self-position of the first imaging device on the environmental map based on an image captured by the first imaging device and the environmental map, a second self-position estimation unit that estimates a second self-position of the second imaging device on the environmental map based on an image captured by the second imaging device and the environmental map, and a correction unit that generates correction information for the scale of the environmental map based on the distance between the first self-position and the second self-position on the environmental map and the distance between the first imaging device and the second imaging device. [Effects of the Invention]

[0007] With the above configuration, it is possible to correct the scale of an environmental map using a monocular image without requiring any auxiliary sensors or markers other than the imaging device. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 10 is a diagram illustrating an example of creating an environmental map in a warehouse. [Figure 2] 1 is a diagram illustrating an example of the overall configuration of a map creation system according to an embodiment of the present invention. [Figure 3] FIG. 1 is a schematic diagram showing how two omnidirectional cameras are attached to a forklift in the map creation system according to this embodiment. [Figure 4] FIG. 2 is a block diagram showing an example of the hardware configuration of an in-vehicle computer that constitutes the map creation system according to the present embodiment. [Figure 5] FIG. 1 is a block diagram illustrating an example of the functional configuration of a map creation system according to an embodiment of the present invention. [Figure 6] FIG. 2 is a block diagram showing a detailed functional configuration around a mobile object position acquisition unit in the map creation system according to the present embodiment. [Figure 7]4 is a flowchart showing an example of a map creation and position estimation process performed by an in-vehicle computer that constitutes the map creation system according to the present embodiment. [Figure 8] 10A to 10C are diagrams illustrating a scale correction process for an environmental map in the map creation system according to the present embodiment. [Figure 9] 10 illustrates an example of a map display screen displayed on a touch panel information terminal provided on a forklift truck in the map creation system according to this embodiment. [Figure 10] 10 illustrates an example of a map display screen displayed on a browser screen on the display of a server device in the map creation system according to this embodiment. [Figure 11] 10 illustrates an example of a map display screen that shows the destination of an object to be transported by a forklift, and is displayed on a browser screen on the display of a server device in the map creation system according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described, but the embodiments of the present invention are not limited to the embodiments described below. Note that in the embodiments described below, as examples of an information processing system and an information processing device, a map creation system 1 including a forklift 10 as a moving body, two omnidirectional cameras 20A and 20B as first and second imaging devices, and an on-board computer 50 as an information processing device will be described with reference to the on-board computer 50. Note that in each drawing, the same components are denoted by the same reference numerals, and duplicated explanations may be omitted.

[0010] The information processing device according to this embodiment generates an environmental map based on images captured by a mobile object equipped with an imaging device while the mobile object is moving, and estimates the position of the mobile object. Although not particularly limited, in a specific embodiment, the mobile object may be a forklift, and the imaging device may be a wide-angle camera such as a spherical camera. The information processing device according to this specific embodiment generates an environmental map of the location where the forklift 10 is moving and estimates the position of the forklift 10 based on images captured by the spherical camera 20 attached to the forklift 10 while the forklift 10 is moving along a predetermined route.

[0011] Here, Fig. 1 is a diagram for explaining an example of creating an environmental map in a warehouse. Fig. 1 shows the inside of a warehouse 100 and the surrounding area of ​​the warehouse 100 as viewed from above (the ceiling side).

[0012] Warehouse 100 is a terminal warehouse (a warehouse established at a transit point in transportation). This terminal warehouse is a type of warehouse known as a cross-docking type. In a cross-docking type warehouse, multiple pallets for each product are received from a factory or wholesaler, and temporarily stored in the warehouse. Then, at the time of shipment, multiple types of pallets are combined while still packed on the same pallet, and shipped to the respective retail stores.

[0013] In Fig. 1, a truck yard 200 is located around a warehouse 100. Fig. 1 shows that the truck yard 200 has detached containers 300 transported by trailers and truck beds connected to the warehouse. A forklift 10 removes a pallet 31 from at least one of the beds of trucks that have arrived at the truck yard 200 and the containers 300 transported by trailers.

[0014] Thereafter, the forklift 10 carries the pallet 31 to the temporary storage location 40 and stores it there temporarily. Thereafter, at the time of shipping, the forklift 10 carries the pallet 31 to a location close to the truck yard 200 in the warehouse 100, arranges the items, and then loads the pallet 31 onto the bed of a truck or into a container 300.

[0015] In order to ensure flexible space for the daily changes in the types and quantities of goods coming in and out, temporary storage locations 40 often do not have designated sections for each product. However, because multiple workers temporarily store pallets 31 in arbitrary locations, when shipping, it is necessary to search for the desired pallet from among the multiple temporarily stored pallets.

[0016] To efficiently perform this search operation, it is necessary to recognize and track the movement of the pallet 31 within the warehouse 100, while effectively utilizing space by not specifying a temporary storage location for the pallet 31, and to visualize it. Furthermore, to recognize and track the movement of the pallet 31 within the warehouse 100, a technology is required that accurately estimates the position of the forklift 10 and creates an environmental map of the warehouse with high accuracy. In particular, VSLAM technology, which simultaneously estimates the self-position and creates environmental map information based on captured images, is often used to create the environmental map. In the map creation system 1 according to this embodiment, an imaging device such as a spherical camera is attached to a moving object such as the forklift 10 in an environment such as that shown in FIG. 1, and VSLAM technology is applied to estimate the self-position and create an environmental map based on images from the imaging device.

[0017] For general information on the technical details of VSLAM, see, for example, "Explanation: Current Status and Future Prospects of SLAM," by Tomono Masahiro et al., Systems / Control / Information, 2020, Vol. 64, No. 2, pp. 45-50 (https: / / www.jstage.jst.go.jp / article / isciesci / 64 / 2 / 64_45 / _article / -char / ja / ), and Sumikura, S, et. al., "OpenVSLAM: A Versatile Visual SLAM Framework," in MM '19: Proceedings of the 27th ACM International Conference on Multimedia, October 2019, Pages 2292-2295 (https: / / dl.acm.org / doi / 10.1145 / 3343031.3350539).

[0018] The map creation system 1 according to this embodiment will be described below.

[0019] (Example of the overall configuration of Map Creation System 1) FIG. 2 is a diagram illustrating an example of the overall configuration of a map creation system 1 according to this embodiment. As illustrated in FIG. 2, the map creation system 1 includes a forklift 10 as a mobile object, two omnidirectional cameras 20A and 20B as multiple imaging devices provided on the mobile object, an on-board computer 50 as an information processing device, and a touch panel information terminal 23 as a display device provided on the mobile object. The map creation system 1 may also include a server device 70 connected to the on-board computer 50 and the touch panel information terminal 23 via a network 400. The on-board computer 50, the touch panel information terminal 23, and the server device 70 are communicatively connected via the network 400, such as a LAN (Local Area Network). Note that other devices, such as other external servers or image forming devices, may also be communicatively connected to the network 400.

[0020] The forklift 10 is an example of a mobile body that transports the pallet 31 and the cargo 32 by holding and transporting cargo 32 placed on the pallet 31. Transport by a mobile body is an example of movement by a mobile body. The pallet 31 and the cargo 32 are each an example of an object. In the following, when there is no particular need to distinguish between the pallet 31 and the cargo 32, they will be collectively referred to as the object 30. The forklift 10 is a generic term for multiple forklifts, the pallet 31 is a generic term for multiple pallets, and the cargo 32 is a generic term for multiple cargoes.

[0021] The forklift 10 may transport the object 30 in response to the driving operation of an operator, or may transport the object 30 by automatic driving without the intervention of an operator.

[0022] Omnidirectional camera 20 is an example of an imaging device provided on forklift 10. Two omnidirectional cameras 20A, 20B are cameras that can capture images in all directions of 360 degrees around each of omnidirectional cameras 20A, 20B. Orientations 21A, 21B indicate the orientations in which omnidirectional cameras 20A, 20B can capture images. Note that, in the described embodiment, the forklift 10 is described as being provided with two omnidirectional cameras 20A, 20B, but three or more omnidirectional cameras 20A, 20B may be provided.

[0023] The spherical image (omnidirectional image) captured by the spherical camera 20 is an example of a captured image. By using the spherical camera 20, for example, when generating a map of an aisle, an environmental map that can correspond to an outbound route and a return route can be created without having to travel back and forth through the aisle. However, the image capturing device is not limited to the spherical camera 20, and may be any other wide-angle camera, a camera with a normal angle of view, or any device that can capture an image of the area around the forklift 10. Furthermore, the captured image does not necessarily have to be a spherical image.

[0024] The spherical image includes an image capturing a scene in the conveying direction 11 of the object 30 as seen from the forklift 10, and a scene in the vertically upward direction 12 as seen from the forklift 10. In other words, the conveying direction 11 is in front of the forklift 10, and the vertically upward direction 12 is above the forklift 10. The spherical camera 20 can capture images in all directions, and therefore can capture an image including both the front and the above of the forklift 10 in a single image. The conveying direction 11 is an example of a moving direction.

[0025] The spherical camera 20 is preferably mounted on the roof of the forklift 10. This ensures a good field of view for capturing images in front of and above the forklift 10.

[0026] 3 is a schematic diagram showing how two omnidirectional cameras 20A and 20B are attached to a forklift 10 in the map creation system 1 according to this embodiment. As shown in FIG. 3, in the embodiment being described, the two omnidirectional cameras 20A and 20B are attached to a roof 10R of the forklift 10. A camera-mounting roof carrier 22 is provided on the roof 10R of the forklift 10, and mounting fixtures such as screws (for example, screws that fit into tripod mounting screw holes of the omnidirectional cameras 20) are provided at diagonal end positions of the camera-mounting roof carrier 22 to fix the positional relationship between the two omnidirectional cameras 20. By attaching the omnidirectional cameras 20A and 20B to the mounting fixtures such as screws, a distance D between the omnidirectional cameras 20A and 20B is determined.

[0027] In this way, omnidirectional camera 20A and omnidirectional camera 20B are placed at ends spaced apart from each other on forklift 10 (ends spaced apart from each other on the installation surface (for example, the roof)). Also, omnidirectional camera 20A and omnidirectional camera 20B are provided spaced apart from each other in a direction perpendicular to the traveling direction of forklift 10 (left-right or up-down direction relative to the front-to-back direction), or in the left-right direction in the embodiment shown in FIG. 3 .

[0028] 2 again, the omnidirectional camera 20 has a wireless or wired communication function and transmits the captured omnidirectional image to the on-board computer 50. The on-board computer 50 transmits the processing result based on the captured omnidirectional image to the touch panel information terminal 23.

[0029] The touch panel information terminal 23 is a terminal device equipped with a display device provided on the mobile body side, and can display an environmental map created by the map creation system 1 on a touch panel. The touch panel information terminal 23 has a wireless communication function and may receive an environmental map from the on-board computer 50 and output it as an image on its own display device. Alternatively, the touch panel information terminal 23 may receive an environmental map from the server device 70, as will be described later. Examples of the touch panel information terminal 23 include a tablet terminal and a small liquid crystal display equipped with a touch screen sensor.

[0030] The on-board computer 50 may transmit the results of processing based on the captured spherical image to the server device 70 via the network 400, either directly or (when the on-board computer 50 does not communicate directly with the network 400) via the touch panel information terminal 23. The server device 70 collects position information from a plurality of forklifts 10 and centrally manages this position information. The environmental map may also be displayed on a liquid crystal display 706 provided in the server device 70 or connected to the server device 70.

[0031] The cargo 32 is provided with a barcode 33, which is an example of identification information that identifies the cargo 32. Such a barcode may be provided on the pallet 31 and used as identification information that identifies the pallet 31. The barcode 33 is read by a reader such as a barcode reader, and the identification information resulting from the reading is transmitted to the onboard computer 50. Note that the identification information is not limited to a barcode, and may be a QR code (registered trademark), an ID (identifier) ​​number, or the like.

[0032] The on-board computer 50 is an example of an information processing device that creates an environmental map within the warehouse 100 and estimates the position information of the forklift 10. The on-board computer 50 may be mounted on the forklift 10 and process the position information of the object 30 transported by the forklift 10. The on-board computer 50 can process the position information of the object 30 based on the position information of the forklift and identification information indicating the object 30.

[0033] (Example of hardware configuration of the on-board computer 50) 4 is a block diagram showing an example of the hardware configuration of the on-board computer 50. The on-board computer 50 is constructed by a computer.

[0034] 4, the in-vehicle computer 50 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, a HD (Hard Disk) 504, a HDD (Hard Disk Drive) controller 505, and a display 506. The in-vehicle computer 50 also includes an external device connection I / F (Interface) 508, a network I / F 509, a data bus 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0035] Of these, the CPU 501 controls the overall operation of the on-board computer 50. The ROM 502 stores programs such as IPL used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501.

[0036] The HD 504 stores various data such as programs. The HDD controller 505 controls the reading and writing of various data from and to the HD 504 under the control of the CPU 501. The display 506 displays various information such as a cursor, menus, windows, characters, or images. The display 506 can display an environmental map as described above.

[0037] The external device connection I / F 508 is an interface for connecting various external devices. In this case, the external devices are, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 509 is an interface for data communication using the network 400. The bus line 510 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 501 shown in FIG. 4.

[0038] The keyboard 511 is a type of input means equipped with multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 514 controls reading and writing of various data from a DVD-RW 513, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R, etc. The media I / F 516 controls reading and writing (storing) of data from a recording medium 515, such as a flash memory.

[0039] Although the above description has been given assuming that the on-board computer 50 includes the display 506, keyboard 511, pointing device 512, and DVD-RW drive 514, some of the components shown in Fig. 4, including these components, may be omitted. Furthermore, the on-board computer 50 may include components other than those shown in Fig. 4. Furthermore, while the hardware configuration of the on-board computer 50 has been described with reference to Fig. 4, the hardware configuration of the touch panel information terminal 23 provided on the forklift 10 and the server device 70 connected via a network can also be implemented in a configuration similar to that shown in Fig. 4 by adding or deleting components as necessary.

[0040] (Example of functional configuration of map creation system 1) Fig. 5 is a block diagram showing an example of the functional configuration of the map creation system 1. As shown in Fig. 5, the on-board computer 50 has a communication unit 51, a moving object position acquisition unit 52, a retained information acquisition unit 53, an identification information acquisition unit 54, a time acquisition unit 55, an object position acquisition unit 56, an output unit 57, a storage unit 58, and an input unit 59.

[0041] 4 operates in response to commands from CPU 501 in accordance with a program loaded from HD 504 onto RAM 503. Storage unit 58 is configured to include storage areas provided by HD 504 to RAM 503, etc.

[0042] Two omnidirectional cameras 20A and 20B are also connected to the on-board computer 50. The on-board computer 50 receives omnidirectional images captured by the omnidirectional cameras 20A and 20B, and generates an environmental map showing the surrounding environment in which the forklift 10 moves based on the omnidirectional images, and estimates position information of the forklift 10. The on-board computer 50 can also obtain position information of the object 30 based on the position information of the forklift 10 and holding information showing whether the forklift 10 is holding or not holding the object 30. The on-board computer 50 can then output the estimated position information of the forklift 10, the generated environmental map, and the obtained position information of the object 30 to the outside via the output unit 57.

[0043] Communication unit 51 receives the omnidirectional images captured by omnidirectional cameras 20A and 20B and outputs them to mobile object position acquisition unit 52 and retained information acquisition unit 53. Communication unit 51 also receives identification information read by a reader such as a barcode reader via network 400 and outputs it to identification information acquisition unit 54. In addition, communication unit 51 may receive information (such as a command) from server device 70 via network 400. Communication unit 51 may transmit the results of processing on the forklift 10 side to server device 70 via network 400.

[0044] Mobile object position acquisition unit 52 estimates the position of forklift 10 based on the multiple omnidirectional images that are continuously input, creates an environmental map that describes the surrounding environment in which forklift 10 moves, and stores the created environmental map and position information in storage unit 58. Here, input unit 59 receives input of a reference distance D (inter-camera extrinsic parameter translation amount) between two omnidirectional cameras 20A and 20B that is measured in advance or given in advance as a parameter used in generating the map, and stores it in storage unit 58. Here, reference distance D between two omnidirectional cameras 20A and 20B is an actual measurement value of the linear distance between two omnidirectional cameras 20A and 20B measured with a tape measure or a laser rangefinder, or a design value of the linear distance based on, for example, the positions of the screw holes in roof carrier 22 for mounting the cameras described above.

[0045] The generated environmental map and estimated position information may be output via the output unit 57. The output destination of the output unit 57 is the touch panel information terminal 23 on the forklift 10 side, and may be displayed on the touch panel information terminal 23. The mobile object position acquisition unit 52 also outputs the position information of the forklift 10 acquired by calculation to the object position acquisition unit 56. Alternatively, the output destination may be an external device such as a PC (Personal Computer) connected to the on-board computer 50, a display device such as the display 506, or a storage device such as the HD 504. The generated environmental map and estimated position information may be transmitted by the communication unit 51 to the server device 70 via the network 400.

[0046] Based on the input spherical image, holding information acquisition unit 53 can acquire holding information indicating whether object 30 is being held or not held by forklift 10 through calculation, and output the information to object position acquisition unit 56. Here, it is sufficient to use an image from either omnidirectional camera 20A or 20B.

[0047] The identification information acquisition unit 54 acquires the identification information by inputting the identification information from the communication unit 51, and outputs the identification information to the object position acquisition unit 56. However, acquisition of the identification information by the identification information acquisition unit 54 is not limited to via the network 400. For example, the identification information acquisition unit 54 may acquire identification information input by the user using the keyboard 511 or the pointing device 512 in FIG. 4, may acquire identification information stored in advance in the storage unit 58, or may acquire identification information via the external device connection I / F 508.

[0048] The time acquisition unit 55 acquires information indicating the time at which the spherical image and the identification information were received, and outputs the information to the object position acquisition unit 56 .

[0049] The object position acquisition unit 56 can acquire the position information of the object 30 based on the position information of the forklift 10 and the holding information. The object position acquisition unit 56 may also output the position information of the object 30, identification information indicating the object 30, and time information via the output unit 57 in a mutually associated manner.

[0050] The storage unit 58 can store parameters used when generating the map, as well as estimated position information of the forklift 10, the generated environmental map, and identification information indicating objects 30 such as pallets 31 or cargo 32.

[0051] (Details of the map creation function of Map Creation System 1) FIG. 6 is a block diagram showing a more detailed functional configuration around the mobile object position acquisition unit 52 in the map creation system 1.

[0052] The mobile object position acquisition unit 52 estimates the position of the forklift 10 and creates environmental map information based on a plurality of omnidirectional images continuously input from the omnidirectional camera 20 and parameters read from the storage unit 58.

[0053] Here, the estimated position will be described assuming that the positions of the two omnidirectional cameras 20A and 20B are fixed with respect to the forklift 10, and that the position of one of the omnidirectional cameras 20A and 20B is estimated as the position of the forklift 10. However, this is not limiting, and for example, the average of the positions of the omnidirectional cameras 20A and 20B (center of gravity position) may be estimated as the position of the forklift 10.

[0054] More specifically, the mobile object position acquisition unit 52 includes a map generation position estimation unit 60 and a scale correction unit 61.

[0055] The map generation and position estimation unit 60 generates an environmental map based on multiple images (image sequence) captured by the spherical camera 20A or the spherical camera 20B while the forklift 10 moves along a predetermined path and continuously input during that time. Here, the spherical camera used to generate the environmental map is either the spherical camera 20A or the spherical camera 20B. The path of the forklift 10 typically travels from a predetermined start position to a predetermined end position via multiple visited points. The path of the forklift 10 may include revisited points that are visited multiple times, or may include a closed loop portion where a path portion connecting multiple visited points makes a full loop. The map generation and position estimation unit 60 also estimates the self-positions of the two omnidirectional cameras 20A and 20B based on images (image pairs) captured by the two omnidirectional cameras 20A and 20B.

[0056] More specifically, the map generation position estimation unit 60 includes a tracking unit 64 , a local optimization unit 65 , and a global optimization unit 66 .

[0057] The tracking unit 64 detects key points for all consecutively input frames, performs key point matching and pose optimization, and estimates the pose of the omnidirectional camera 20 for the input frames. The tracking unit 64 also determines a key frame to determine whether to insert a new key frame. When the tracking unit 64 registers an appropriate frame as a new key frame, it outputs the key frame to be registered to the local optimization unit 65 and the global optimization unit 66. The self-position estimation result is output to the scale correction unit 61.

[0058] The local optimizer 65 performs optimization on frames near the latest frame. The local optimizer 65 triangulates points on 3D coordinates using the inserted keyframe, thereby creating and expanding a map. The local optimizer 65 also performs local bundle adjustment. The partial map around the keyframe is referred to as the local map 67.

[0059] If loop closure is possible in the travel area for which the map is to be generated, it is preferable to perform loop closure by setting revisit points in the route of the forklift 10 to eliminate distortions in the map. The global optimization unit 66 performs overall optimization using loop closure. The global optimization unit 66 executes loop detection, pose graph optimization, and global bundle adjustment. This optimization eliminates distortions in the map as a whole as much as possible. The overall map obtained so far is referred to as a global map 68.

[0060] As described above, either the omnidirectional camera 20A or the omnidirectional camera 20B is used to generate the environmental maps (local map 67 and global map 68) that describe the surrounding environment in which the forklift 10 moves. Once the environmental maps (67, 68) are generated, the map generation position estimation unit 60 can estimate the self-position of the forklift 10 by performing only tracking based on the images captured by the two omnidirectional cameras 20A, 20B, without describing the map.

[0061] When generating the environmental maps (67, 68) based on a time series of omnidirectional images captured by either of the two omnidirectional cameras 20A, 20B along the route traveled by the forklift 10, the map generation position estimation unit 60 constitutes the environmental map generation unit in this embodiment. The generated environmental maps (67, 68) are held in a storage unit 69 provided as a storage area such as a memory. The environmental maps (67, 68) are stored in the storage unit 58 as appropriate, and are read out and expanded in the storage unit 69, for example, when the on-board computer 50 is started up. The storage unit 69 in which the environmental maps (67, 68) are stored constitutes the storage unit in this embodiment. Furthermore, when the self-position on the environmental map of omnidirectional camera 20 is estimated using the omnidirectional image captured by omnidirectional camera 20A or omnidirectional camera 20B based on the omnidirectional image captured by omnidirectional camera 20A or omnidirectional camera 20B and the generated environmental map (67, 68), map generation position estimation unit 60 constitutes a first self-position estimation unit or a second self-position estimation unit in this embodiment.

[0062] Omnidirectional camera 20 has multiple optical systems and has a field of view in all directions, but unlike a stereo camera, the images it generates are equivalent to monocular images captured by a monocular camera. Here, a monocular image refers to an image captured from one viewpoint by a monocular camera, and is different from images captured by a stereo camera. Therefore, in the environmental maps (local map 67 and global map 68) generated by map generation position estimation unit 60 using either omnidirectional camera 20A or omnidirectional camera 20B, the distance units are unknown, and scale correction is required to convert them into positions and distances in real space.

[0063] Therefore, the scale correction unit 61 calculates a correction coefficient (scale coefficient) for correcting the scale of the environmental map (67, 68) based on predetermined parameters. As described above, the calculation of this correction coefficient uses the self-location estimation results of the two omnidirectional cameras 20A and 20B on the same map, which are based on two images captured by the two omnidirectional cameras 20A and 20B. The self-location estimation results here are based on omnidirectional images (image pairs) captured by the two omnidirectional cameras 20A and 20B at approximately the same time. More specifically, the scale correction unit 61 calculates the correction coefficient (scale coefficient α) based on a virtual distance between the cameras, which is based on position information of one omnidirectional camera 20 (e.g., 20A) and position information of the other omnidirectional camera 20 (e.g., 20B) on the same map, preferably at the same time (within a predetermined time range), and on a reference distance D previously assigned to the two omnidirectional cameras 20A and 20B.

[0064] The coordinates of the two omnidirectional cameras 20A and 20B on the same map (same coordinate system) are represented as P1 (x1, y1, z1) and P2 (x2, y2, z2). Then, the Euclidean distance (virtual distance) d between the two omnidirectional cameras 20A and 20B on the same map is calculated by the following formula:

number

[0065] The distance d in the map coordinate system corresponds to the distance D in the real space, so the following relational expression is used:

number

[0066] (Map creation and position estimation processing by on-board computer) 7 is a flowchart showing an example of a map creation and position estimation process by the on-board computer 50 constituting the map creation system according to this embodiment. The process shown in FIG. 7 is executed by the on-board computer 50 in response to receiving an instruction to start map creation. The instruction to start map creation may be transmitted to the on-board computer 50 in response to, for example, a start operation performed on the touch panel information terminal 23 by the user of the driver of the forklift 10 used in map creation. Alternatively, if the forklift 10 is automatically driven (an automated guided vehicle), the instruction may be transmitted to the on-board computer 50 via the network 400 in response to a start operation performed on the server device 70 by an administrator who manages the automatic operation of the forklift 10.

[0067] First, as a preliminary preparation for the process shown in Fig. 7, two omnidirectional cameras 20A and 20B are installed on forklift 10, and a reference distance D between two omnidirectional cameras 20A and 20B is input via input unit 59 and stored in storage unit 58. Before starting, forklift 10 is assumed to be parked at a predetermined starting point in the area for which a map is to be created. In map creation, an environmental map (67, 68) is generated based on this starting point.

[0068] The process shown in FIG. 7 starts at step S100, and at step S101, the on-board computer 50 reads out a predefined reference distance D between the two cameras from the storage unit 58 and acquires it as a correction parameter.

[0069] In step S102, the on-board computer 50 generates an environmental map using one of the omnidirectional cameras 20A, 20B (in the following description, omnidirectional camera 20A) of the route along which the forklift 10 will travel, by using the map generation position estimation unit 60. For example, the driver starts the forklift 10 traveling along a predetermined route. Alternatively, in the case of automatic driving, the forklift 10 starts traveling according to a predetermined algorithm. In response to the forklift 10 completing traveling along the entire route, the on-board computer 50 causes the map generation position estimation unit 60 to complete map generation. Here, the forklift 10 travels along all possible routes within the work area, and as it travels, images are captured by the omnidirectional camera 20A and the omnidirectional images are continuously input to the on-board computer 50. Note that the map obtained at this stage has an indefinite distance unit.

[0070] Subsequently, in step S103, on-board computer 50 causes map generation position estimation unit 60 to estimate positions (P1(x1, y1, z1), P2(x2, y2, z2)) of omnidirectional camera 20A and omnidirectional camera 20B in the same coordinate system at any point in time, based on the omnidirectional images captured by omnidirectional camera 20A and omnidirectional camera 20B, respectively, and the environmental map generated in step S102.

[0071] In step S104, the on-board computer 50 uses the scale correction unit 61 to calculate the distance d between the self-positions (P1(x1, y1, z1), P2(x2, y2, z2)) of the two omnidirectional cameras 20A and 20B on the environmental map.

[0072] In step S105, the on-board computer 50 causes the scale correction unit 61 to calculate the scale coefficient α of the environmental map based on the distance d and the reference distance D given in advance to the omnidirectional cameras 20A and 20B.

[0073] In step S106, the onboard computer 50 uses the scale correction unit 61 to correct the scale of the environmental map (67, 68) generated in step S102 using the calculated scale coefficient α, thereby generating a scale-corrected environmental map. More specifically, in step S106, the scale coefficient α calculated in step S105 is applied to each piece of coordinate information contained in the environmental map. By multiplying the scale coefficient by each of the coordinate information of the point cloud and the coordinate information contained in the key frame, the coordinate values ​​on the system are converted to meters, and the self-localization results using that map are also converted to meters thereafter.

[0074] The process shown in FIG. 7 ends at step S107. (Map correction processing) Hereinafter, with reference to FIG. 8, a process for generating an environmental map in a predetermined distance unit by correcting the scale of an environmental map in which the distance unit is unknown will be described.

[0075] As described above, the distance on the VSLAM system (map) generated by the monocular camera is different from the actual distance because the distance unit is unknown. To resolve this, scale correction is required. In this embodiment, the distance between the two omnidirectional cameras 20A and 20B is known and is stored in storage unit 58.

[0076] Although the distance unit is unknown in the environmental maps (67, 68) generated by the map generation position estimation unit 60 using a single omnidirectional camera 20, some distance (virtual distance d) is defined between the two cameras estimated using the environmental map and images captured by each of the two omnidirectional cameras 20A and 20B. The scale correction unit 61 calculates a scale coefficient α for converting the virtual distance (virtual distance d) in any unit between the self-estimated position of the omnidirectional camera 20A and the self-estimated position of the omnidirectional camera 20B on the environmental map (67, 68) into the reference distance D between the omnidirectional cameras 20A and 20B.

[0077] The environmental map generated (VLSAM generated) by the map generation position estimation unit 60 includes a set of points on three-dimensional coordinates with variable distance units. The scale correction unit 61 applies a scale coefficient α to each of the included sets of points on three-dimensional coordinates (including the coordinates of key frames) to generate an environmental map that corresponds to the actual distance.

[0078] In the above description, the scale factor α can be calculated with at least one pair of self-estimated positions (P1, P2) of the two omnidirectional cameras 20A and 20B at any point in time. However, the scale factor α may be calculated by performing statistical processing using a time series of pairs of self-estimated positions of the two omnidirectional cameras 20A and 20B at multiple points in time (P1(0), P2(0)), (P1(1), P2(1)), ..., (P1(t), P2(t)). For example, it is possible to find the scale factor α that minimizes the sum of squares of the error.

[0079] In the above-described embodiment, after the environmental map (67, 68) is generated once, the two omnidirectional cameras 20A and 20B capture omnidirectional images at any time point, and based on the captured images, the cameras estimate their own positions and calculate the scale factor α. However, the images used to calculate the scale factor α are not particularly limited. In other embodiments, pairs of omnidirectional images captured at one or more time points when the environmental map is generated may be stored, and the scale factor α may be calculated using the one or more pairs.

[0080] In addition, in the above-described embodiment, the method of applying the scale coefficient α has been described as generating an environmental map corresponding to the actual distance by applying the scale coefficient α to the environmental map (67, 68) with an indefinite distance unit. In this case, by using the environmental map corresponding to the corrected actual distance, the map generation position estimation unit 60 can estimate the self-position based on the actual distance. On the other hand, it is also possible to estimate the self-position using the environmental map with an indefinite distance unit, and convert the estimated position with an indefinite distance unit into a position or distance in the actual distance space each time by applying the scale coefficient α calculated at that time using the two omnidirectional cameras 20A, 20B. Furthermore, when an environmental map corresponding to the corrected actual distance is used, the estimated self-position is based on the actual distance. Furthermore, the self-position may be estimated using the corrected environmental map, and the latest scale coefficient α at that time (which is 1 in an ideal case with no error) may be calculated based on the estimated distance between the positions of the two celestial sphere cameras 20A and 20B (which is a distance converted into a tentative actual distance), and the latest scale coefficient α may be applied to deal with changes over time.

[0081] (Scale-corrected map visualization) Below we explain how to visualize a scale-corrected environmental map on a blueprint. A scale-corrected environmental map is a collection of points on a 3D coordinate system. To visualize it on a 2D map, the point cloud is projected onto the XZ plane (horizontal plane) in the camera coordinate system to create a bird's-eye view from the point cloud map, and image data is generated that serves as a point cloud-based orthoimage. Here, an orthoimage refers to an image that has been orthogonally transformed to one in which each point is positioned in the correct position (the distance between two points is correct), as if viewed from directly above, with no tilt.

[0082] The ortho-view created above is then fitted to the design map information of the work area. At this time, the scale-corrected environmental map is enlarged or reduced as needed, and is rotated, translated, or both, so that the self-location estimation result is superimposed on the design map and visualized.

[0083] The output destination for showing the position of the forklift 10 on the design map is preferably the touch panel information terminal 23 provided on the forklift 10.

[0084] Fig. 9 shows an example of a map display screen 90 displayed on the touch panel information terminal 23 of the forklift 10. The map display screen shown in Fig. 9 shows the position 92 of the forklift 10 and the positions of objects 30 such as luggage and pallets on a design map 91.

[0085] Fig. 10 shows an example of a map display screen 93 displayed on a browser screen on the display 706 of the server device 70. The map display screen 93 shown in Fig. 10 also shows the current position 95 of the forklift and the positions of objects 30 such as luggage and pallets on a design map 94.

[0086] The method for visualizing the scale-corrected environmental map on the design drawing has been described above with reference to Figures 9 and 10. However, the method for displaying the scale-corrected environmental map is not particularly limited.

[0087] For example, a scale-corrected environmental map can be displayed as image data that becomes a point cloud-based orthoimage. In this case, a scale bar (displaying units of distance) can be displayed along with the scale. FIG. 11 illustrates a map display screen 96 displayed on a browser screen on the display 706 of the server device 70, showing the destination of the object 30 transported by the forklift 10. The display screen 96 displays a position map 97. The position map 97 includes a source position 98A and a destination position 98B. The source position 98A indicates the source position, and the destination position 98B indicates the destination position. Displaying the location of a pre-registered temporary storage location or the like on the screen can provide information that makes work easier. FIG. 11 also illustrates a scale bar 99. The scale bar 99 is an example of a GUI component that indicates the displayed scale and is attached when displaying a scale-corrected environmental map.

[0088] 11 is created by the mobile object position acquisition unit 52 based on the omnidirectional image captured by the omnidirectional camera 20. The position map 97 is created by projecting a point cloud including the acquired three-dimensional coordinate information onto a two-dimensional plane. Although it is assumed that there are multiple forklifts 10, the position map 97 may be created, for example, according to the movement of one forklift 10 and used to acquire the position information of all the forklifts 10 (self-position recognition). This makes it possible to express the positions of the multiple forklifts 10 in the same coordinate system.

[0089] (When travelling across different spaces) In the above-described embodiment, the environmental map (67, 68) generated using one of the omnidirectional cameras 20A is described under the assumption that the entire map is generated based on a fixed scale, even though the distance unit is not fixed. This assumption is preferably met when generating a map of the space within a single room. On the other hand, when moving between multiple rooms, the scale of the portion of the environmental map generated in one room (first space) may differ from the scale of the portion of the environmental map generated in the other room (second space). In this case, the entire environmental map will be a continuous map, but will be partially distorted. This is because when moving between multiple rooms, it is not possible to find key points common to both rooms.

[0090] To address this problem, in the embodiment to be described, the scale of the portion of the environmental map generated in the first space and the scale of the portion of the environmental map generated in the second space can be integrated based on the fact that the distance between the two omnidirectional cameras 20A, 20B in the portion of the environmental map generated in the first space and the distance between the two omnidirectional cameras 20A, 20B in the portion of the environmental map generated in the second space should be equal.

[0091] For example, an environmental map can be divided into multiple sections corresponding to each space, and a corresponding scale factor can be set for each section. In this case, in self-localization, distance can be converted using a different scale factor depending on the section in which the current location is estimated. Alternatively, for an environmental map with an indefinite distance unit, a corresponding scale factor can be applied to the section corresponding to each space for scale correction. Furthermore, whether two spaces with different scales are crossing the map can be determined based on a threshold for the change in the calculated scale factor α. For example, by monitoring the scale factor α, if the change in the scale factor α exceeds a certain threshold, the map can be divided into multiple sections so that the section boundary passes through the position where the change was detected.

[0092] (Simultaneous map generation and scale adaptation) In the above-described embodiment, an environmental map is generated by the map generation position estimation unit 60 using one omnidirectional camera 20A on the route along which the forklift 10 travels, and then when the forklift 10 is at an arbitrary position, the map generation position estimation unit 60 estimates the self-position and the scale correction unit 61 generates correction information without using two omnidirectional cameras 20A and 20B to describe the map. However, the present invention is not limited to the above-described embodiment.

[0093] In another embodiment, the generation of an environmental map by the map generation position estimation unit 60, the estimation of the self-location by the map generation position estimation unit 60, and the generation of correction information by the scale correction unit 61 can be performed simultaneously. For example, an environmental map generated at a certain point in time using one omnidirectional camera 20A may be scale-corrected using two omnidirectional cameras 20A to generate a scale-corrected environmental map, and then the self-location may be estimated using the scale-corrected environmental map. Subsequently, scale coefficients may be calculated as needed using the two omnidirectional cameras 20A, and the scale coefficients may be applied to the environmental map. Furthermore, the map may be described to expand the map. In this case, for example, the map description may be performed based only on the omnidirectional image captured by one of the omnidirectional cameras 20A.

[0094] (Modification of functional part arrangement) In the embodiment described below, the mobile object position acquisition unit 52 is described as being provided in the on-board computer 50. However, the mobile object position acquisition unit 52 may be provided in a device other than the on-board computer 50, for example, in a server device 70, and processing by the mobile object position acquisition unit 52 may be performed in the server device 70 that receives images captured by the omnidirectional cameras 20A and 20B via the network 400. In other embodiments, the mobile object position acquisition unit 52 may be provided in the touch panel information terminal 23 or the omnidirectional camera 20.

[0095] In still another embodiment, the mobile object position acquisition unit 52 may be provided on the cloud (its physical or virtual server), and may be provided as a cloud service that provides an environmental map generated in response to continuous input of omnidirectional images from the omnidirectional cameras 20A and 20B (as video data or as a sequence of still images, and as files or stream data).

[0096] (advantage) As described above, according to this embodiment, it is possible to provide an information processing system, an information processing device, an information processing method, and a program that can perform scale correction of an environmental map using a monocular image without requiring any auxiliary sensors or markers other than the imaging device.

[0097] In particular, in a configuration in which self-position estimation and environmental map creation are performed based on captured images, it is possible to correct the scale of an environmental map with indefinite units to an environmental map with fixed units. In this case, a known reference distance D between omnidirectional cameras 20A and 20B is specified, and accurate map scale correction processing can be automatically performed by converting the movement distance in the system into a known measured distance when map creation is completed, when self-position estimation is performed after map creation is completed, and during map creation.

[0098] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a conventional circuit module designed to perform each of the above-described functions.

[0099] The devices described in the embodiments are merely illustrative of one of several computing environments for implementing the embodiments disclosed herein. In some embodiments, the information processing apparatus includes a plurality of computing devices, such as a server cluster, configured to communicate with each other via any type of communication link, including a network, shared memory, etc., and to perform the processes disclosed herein.

[0100] The above has described the embodiments and examples of the present invention, but the embodiments and examples of the present invention are not limited to the above-described embodiments and examples, and may be modified within the scope of what a person skilled in the art could conceive, such as other embodiments, other examples, additions, changes, deletions, etc., and any aspect is included in the scope of the present invention as long as it exhibits the functions and effects of the present invention.

[0101] For example, aspects of the present invention are as follows. <1> An information processing system that estimates a position using a first imaging device and a second imaging device that are provided in a moving object and capture monocular images, a storage unit that stores an environmental map showing the surrounding environment in which the moving object moves; a first self-position estimation unit that estimates a first self-position of the first imaging device on the environmental map based on an image captured by the first imaging device and the environmental map; a second self-position estimation unit that estimates a second self-position of the second image capture device on the environmental map based on an image captured by the second image capture device and the environmental map; a correction unit that generates correction information for the scale of the environmental map based on a distance between the first self-position and the second self-position on the environmental map and a distance between the first imaging device and the second imaging device; An information processing system including: <2> further including an environmental map generating unit that generates the environmental map based on a plurality of images captured by either the first imaging device or the second imaging device along a route along which the moving object moves; <1> The information processing system is described in <3> After the environmental map is generated by the environmental map generation unit along a route that the moving object is traveling, the first self-location estimation unit and the second self-location estimation unit perform self-location estimation and the correction unit generates correction information. <2> The information processing system is described in <4> The self-location estimation unit and the second self-location estimation unit estimate the self-location, and the correction unit generates correction information, while the environmental map generation unit generates the environmental map. <2> or <3> The information processing system is described in <5> a scale of the first portion of the environmental map generated in the first space and a scale of the second portion of the environmental map generated in the second space are integrated based on the fact that a distance between a first self-position of the first imaging device and a second self-position of the second imaging device in a first portion of the environmental map generated in a first space is equal to a distance between a third self-position of the first imaging device and a fourth self-position of the second imaging device in a second portion of the environmental map generated in a second space different from the first space; <1> ~ <4> The information processing system is described in any one of the above. <6> The first imaging device and the second imaging device are disposed at ends separated from each other on the moving body. <1> ~ <5> The information processing system is described in any one of the above. <7> The first imaging device and the second imaging device are provided at an interval in a direction perpendicular to the traveling direction of the moving object. <6> The information processing system is described in <8> the first imaging device and the second imaging device are each a celestial sphere camera. <1> ~ <7> The information processing system according to any one of the above items. <9> An information processing device provided in a moving object for estimating a position using a first imaging device and a second imaging device, each of which captures a monocular image, comprising: a storage unit that stores an environmental map showing the surrounding environment in which the moving object moves; a first self-position estimation unit that estimates a first self-position of the first imaging device on the environmental map based on an image captured by the first imaging device and the environmental map; a second self-position estimation unit that estimates a second self-position of the second image capture device on the environmental map based on an image captured by the second image capture device and the environmental map; a correction unit that generates correction information for the scale of the environmental map based on a distance between the first self-position and the second self-position on the environmental map and a reference distance that is given in advance to the first imaging device and the second imaging device; The present invention relates to an information processing device. <10> An information processing method for estimating a position using a first imaging device and a second imaging device that are provided on a moving object and capture monocular images, the method comprising: reading an environmental map showing the surrounding environment in which the moving object moves; estimating a first self-position of the first imaging device on the environmental map based on an image captured by the first imaging device and the environmental map; estimating a second self-position of the second imaging device on the environmental map based on an image captured by the second imaging device and the environmental map; generating correction information for the scale of the environmental map based on a distance between the first self-position and the second self-position on the environmental map and a reference distance between the first imaging device and the second imaging device; The present invention relates to an information processing method, including: <11> A program for realizing an information processing device for estimating a position using a first imaging device and a second imaging device that are provided in a moving object and capture monocular images, the program comprising: a storage unit that stores an environmental map showing the surrounding environment in which the moving object moves; a first self-position estimation unit that estimates a first self-position of the first imaging device on the environmental map based on an image captured by the first imaging device and the environmental map; a second self-position estimation unit that estimates a second self-position of the second image capture device on the environmental map based on an image captured by the second image capture device and the environmental map; and a correction unit that generates correction information for the scale of the environmental map based on a distance between the first self-position and the second self-position on the environmental map and a distance between the first imaging device and the second imaging device. This is a program that functions as a [Explanation of symbols]

[0102] 1...map creation system, 10...forklift (an example of a moving body), 11...transport direction (an example of a moving direction), 12...vertical upward direction, 20A, 20B...omnidirectional camera (an example of a plurality of imaging devices), 21A, 21B...orientation, 22...roof carrier for mounting camera, 30...object, 31...pallet, 32...cargo, 33...barcode, 40...temporary storage location, 50...in-vehicle computer (an example of an information processing device), 51...communication unit, 52...moving body position acquisition unit, 53...retained information acquisition unit, 54...identification information acquisition unit, 55...time acquisition unit, 56...object position acquisition unit, 57...output unit, 58...storage unit, 59...input unit, 60...map generation position estimation unit, 61...map correction unit, 62...omnidirectional image Image (example of image), 63...parameter (reference distance), 64...tracking unit, 65...local optimization unit, 66...global optimization unit, 67...local map, 68...global map, 100...warehouse, 200...truck yard, 300...container, 400...network, 501...CPU, 502...ROM, 503...RAM, 504...HDD, 505...HDD controller, 506...display, 508...external device connection I / F, 509...network I / F, 511...keyboard, 512...pointing device, 514...DVD-RW drive, 513...DVD-RW media, 516...media I / F, 515...recording media [Prior art documents] [Patent documents]

[0103] [Patent Document 1] Japanese Patent Publication No. 2022-015978

Claims

1. An information processing system that estimates a position using a first imaging device and a second imaging device that are provided in a moving object and capture monocular images, a storage unit that stores an environmental map showing the surrounding environment in which the moving object moves; a first self-position estimation unit that estimates a first self-position of the first imaging device on the environmental map based on an image captured by the first imaging device and the environmental map; a second self-position estimation unit that estimates a second self-position of the second image capture device on the environmental map based on an image captured by the second image capture device and the environmental map; a correction unit that generates correction information for the scale of the environmental map based on a distance between the first self-position and the second self-position on the environmental map and a distance between the first imaging device and the second imaging device; An information processing system comprising:

2. The information processing system according to claim 1 , further comprising an environmental map generation unit that generates the environmental map based on a plurality of images captured by either the first imaging device or the second imaging device along a route along which the moving body moves.

3. 3. The information processing system according to claim 2, wherein after the environmental map is generated by the environmental map generation unit along the route traveled by the moving body, self-position estimation is performed by the first self-position estimation unit and the second self-position estimation unit, and correction information is generated by the correction unit.

4. The information processing system according to claim 2 , wherein the environmental map is generated by the environmental map generation unit, and the first self-location estimation unit and the second self-location estimation unit estimate the self-location, and the correction unit generates correction information.

5. The information processing system of claim 1, wherein the scale of the first portion of the environmental map generated in the first space and the scale of the second portion of the environmental map generated in the second space are integrated based on the fact that the distance between the first self-position of the first imaging device and the second self-position of the second imaging device in the first portion of the environmental map generated in the first space is equal to the distance between the third self-position of the first imaging device and the fourth self-position of the second imaging device in the second portion of the environmental map generated in a second space different from the first space.

6. 2. The information processing system according to claim 1, wherein the first imaging device and the second imaging device are arranged at ends spaced apart from each other on the moving body.

7. 7. The information processing system according to claim 6, wherein the first imaging device and the second imaging device are spaced apart from each other in a direction perpendicular to the traveling direction of the moving object.

8. The information processing system according to claim 1 , wherein the first imaging device and the second imaging device are each a spherical camera.

9. An information processing device provided in a moving object for estimating a position using a first imaging device and a second imaging device, each of which captures a monocular image, comprising: a storage unit that stores an environmental map showing the surrounding environment in which the moving object moves; a first self-position estimation unit that estimates a first self-position of the first imaging device on the environmental map based on an image captured by the first imaging device and the environmental map; a second self-position estimation unit that estimates a second self-position of the second image capture device on the environmental map based on an image captured by the second image capture device and the environmental map; a correction unit that generates correction information for the scale of the environmental map based on a distance between the first self-position and the second self-position on the environmental map and a reference distance that is given in advance to the first imaging device and the second imaging device; An information processing device comprising:

10. An information processing method for estimating a position using a first imaging device and a second imaging device that are provided on a moving object and capture monocular images, the method comprising: reading an environmental map showing the surrounding environment in which the moving object moves; estimating a first self-position of the first imaging device on the environmental map based on an image captured by the first imaging device and the environmental map; estimating a second self-position of the second imaging device on the environmental map based on an image captured by the second imaging device and the environmental map; generating correction information for the scale of the environmental map based on a distance between the first self-position and the second self-position on the environmental map and a reference distance between the first image capturing device and the second image capturing device; An information processing method, including:

11. A program for realizing an information processing device for estimating a position using a first imaging device and a second imaging device that are provided in a moving object and capture monocular images, the program comprising: a storage unit that stores an environmental map showing the surrounding environment in which the moving object moves; a first self-position estimation unit that estimates a first self-position of the first imaging device on the environmental map based on an image captured by the first imaging device and the environmental map; a second self-position estimation unit that estimates a second self-position of the second image capture device on the environmental map based on an image captured by the second image capture device and the environmental map; and a correction unit that generates correction information for the scale of the environmental map based on a distance between the first self-position and the second self-position on the environmental map and a distance between the first imaging device and the second imaging device. A program to function as a

Images