Information processing method, information processing system, and program

By setting map creation progress information based on sensor orientation, the inefficiencies in creating environmental maps for AMRs with front-mounted sensors are addressed, ensuring complete path coverage and accurate self-position estimation.

WO2026239916A1PCT designated stage Publication Date: 2026-11-19SONY SEMICON SOLUTIONS CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2026/016619
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-16
Filing Date
2026-04-27
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Creating an environmental map for self-position estimation in autonomous mobile robots (AMRs) is inefficient when the sensor device, such as LiDAR, is mounted only on the front, leading to blind spots and difficulty in determining which passages have been traveled in both directions, especially in large environments like warehouses with irregularities.

Method used

Setting map creation progress information corresponding to the orientation of the sensor device for each region of the environmental map, using color or attitude information to visually indicate which areas have been sensed from different directions, allowing for efficient creation of the environmental map.

Benefits of technology

Enables easy identification of paths traveled in both directions, eliminating omissions and facilitating efficient environmental map creation for self-position estimation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2026016619_19112026_PF_FP_ABST
    Figure JP2026016619_19112026_PF_FP_ABST
Patent Text Reader

Abstract

The present disclosure relates to an information processing method, an information processing system, and a program that make it possible to efficiently create an environment map for self-position estimation. An information processing method according to the present disclosure includes: creating an environment map on the basis of sensor information acquired by a sensor device that senses an environment; and setting, for each region of the environment map, map creation progress information corresponding to the orientation of the sensor device when the sensor information was acquired. The present disclosure can be applied to self-position estimation using LiDAR.
Need to check novelty before this filing date? Find Prior Art

Description

Information processing method, information processing system, and program

[0001] The present disclosure relates to an information processing method, an information processing system, and a program, and particularly relates to an information processing method, an information processing system, and a program that enable efficient creation of an environmental map for self-position estimation.

[0002] When operating an AMR (autonomous mobile transport robot), an environmental map is created based on sensor information acquired by a sensor device such as LiDAR mounted on the AMR by traveling along the operation route in advance, and self-position estimation is performed using this environmental map.

[0003] Patent Document 1 describes plotting the trajectory and the body direction of an autonomous traveling work device at the time of creating an environmental map on the map.

[0004] Japanese Unexamined Patent Application Publication No. 2019-114153

[0005] When the sensing range of the sensor device is limited, such as only in front of the AMR, it is necessary to travel through the passage in both directions, but it is difficult to know which passage has been traveled in both directions, making it impossible to create the environmental map efficiently.

[0006] The present disclosure has been made in view of such circumstances, and is intended to enable efficient creation of an environmental map for self-position estimation.

[0007] The information processing method of the present disclosure is an information processing method including creating an environmental map based on sensor information acquired by a sensor device that senses an environment, and setting map creation progress information corresponding to the posture of the sensor device at the time when the sensor information is acquired, for each region of the environmental map.

[0008] The information processing system of the present disclosure is an information processing system including a sensor device that senses an environment, and an information processing device that creates an environmental map based on sensor information acquired by the sensor device, and sets map creation progress information corresponding to the posture of the sensor device at the time when the sensor information is acquired, for each of a plurality of regions in the environmental map.

[0009] The program disclosed herein is a program for causing a process to be executed which includes creating an environmental map based on sensor information acquired by a sensor device that senses the environment, and setting map creation progress information for each of a plurality of regions in the environmental map, according to the orientation of the sensor device when the sensor information was acquired.

[0010] In this disclosure, an environmental map is created based on sensor information acquired by a sensor device that senses the environment, and map creation progress information corresponding to the orientation of the sensor device at the time the sensor information was acquired is set for each of the multiple regions in the environmental map.

[0011] This is a diagram illustrating the overview of self-localization using LiDAR. This is a diagram illustrating the challenges when traveling in the opposite direction to the map creation process. This is a diagram illustrating the process of creating a map by traveling in both directions. This is a diagram illustrating the overview of the technology related to this disclosure. This is a block diagram illustrating an example of the configuration of an information processing system. This is a block diagram illustrating an example of the configuration of a mobile robot. This is a block diagram illustrating another example of the configuration of a mobile robot. This is a flowchart illustrating the operation of an information processing system. This is a flowchart illustrating the details of setting map creation progress information. This is a diagram illustrating an example of displaying map creation progress information. This is a diagram illustrating the method of determining map creation progress information. This is a diagram illustrating an example of displaying that travel has been made in both directions. This is a diagram illustrating an example of displaying that travel has been made in both directions. This is a flowchart illustrating the operation of a mobile robot during automatic travel. This is a diagram illustrating automatic travel and the setting of map creation progress information. This is a block diagram illustrating another example of the configuration of an information processing system. This is a block diagram illustrating yet another example of the configuration of an information processing system. This is a block diagram illustrating an example of the configuration of computer hardware.

[0012] The following describes the forms for implementing this disclosure (hereinafter referred to as embodiments). The explanation will be given in the following order.

[0013] 1. Challenges of the Conventional Technology and an Overview of the Technology Disclosed 2. Configuration and Operation of the Information Processing System 3. Setting of Map Creation Progress Information in Automated Driving 4. Other Examples of Information Processing System Configurations 5. Examples of Computer Hardware Configurations

[0014] <1. Issues with the Conventional Technology and an Overview of the Technology Related to This Disclosure> (Conventional Technology and its Issues) When operating an AMR, an environmental map is created based on sensor information acquired by sensor devices such as LiDAR mounted on the AMR by driving the route to be operated in advance, and self-position estimation is performed using this environmental map.

[0015] Figure 1 is a diagram illustrating the overview of self-localization using LiDAR.

[0016] Figure 1A shows the environmental map MP, created as an occupied grid map by SLAM (Simultaneous Localization and Mapping), and the point cloud PC observed by LiDAR. By searching for areas where the observed point cloud PC geometrically overlaps with the environmental map MP, the current self-position relative to the environmental map MP can be estimated. Furthermore, as shown in Figure 1B, by accumulating point cloud PCs that match the environmental map MP, it is possible to create environmental map MPs for a wider range.

[0017] On the other hand, to reduce costs, for example, LiDAR may be mounted only on the front of the AMR. In such cases, as shown in Figure 2A, the environmental map MP is created while accumulating point clouds PC of obstacles BS such as pillars within the measurement range RA of the LiDAR mounted on the front of the mobile robot 10 as an AMR. In this case, the final environmental map MP created will be a map in which point clouds of the surface of obstacles BS that are in a blind spot relative to the direction of travel of the mobile robot 10 are not recorded, as shown in Figure 2B. In this state, as shown in Figure 2C, if the mobile robot 10 travels in the opposite direction along the same path using the environmental map MP, it will observe point clouds of the surface of obstacles BS that are not recorded in the environmental map MP. In this case, it is not possible to properly match the observed point clouds PC with the environmental map MP.

[0018] Therefore, the mobile robot 10 needs to travel in both directions along the passage when creating the environmental map MP. That is, as shown in Figure 3A, the mobile robot 10 creates the environmental map MP while traveling in the forward direction (upward direction in the figure), and then, as shown in Figure 3B, the mobile robot 10 creates the environmental map MP while traveling in the reverse direction (downward direction in the figure). As a result, as shown in Figure 3C, it is possible to create an environmental map MP that can be matched with the observed point cloud PC (i.e., capable of self-localization) regardless of the direction in which the robot travels.

[0019] The environmental maps created in the manner described above are generally displayed as occupancy grid maps or point cloud maps. However, in such environmental maps, it is difficult to determine which passages were used in both directions, making it difficult to create environmental maps efficiently. Specifically, when creating environmental maps of warehouses, which are large and have many passages where AMR (Anti-Monitoring Relay) operation is envisioned, it was not practical to visually display which passages were used in both directions. In particular, in environments where blind spots occur due to irregularities on the sides (walls) of passages, it was not easy to visually indicate whether point clouds could be observed from both directions along a passage.

[0020] (Summary of the technology related to this disclosure) In the technology related to this disclosure, when creating an environmental map based on sensor information acquired by a sensor device that senses the environment, map creation progress information corresponding to the orientation of the sensor device when the sensor information was acquired is set for each area of ​​the environmental map.

[0021] In other words, when the sensor device is in a first orientation, first map creation progress information is set for the area of ​​the environmental map, and when the sensor device is in a second orientation opposite to the first orientation, second map creation progress information is set for the area of ​​the environmental map.

[0022] Furthermore, if one of the first or second map creation progress information is already set for a region, and the other of the first or second map creation progress information is set for that region, then the third map creation progress information will be set for that region. Here, the orientation of the sensor device is defined as the orientation of the sensor device in the coordinate system of the environmental map. The map creation progress information is intended to show the user or system the orientation in which the sensor device sensed the environment of the region to create the environmental map.

[0023] For example, as shown in Figure 4A, in an occupied grid map point cloud map or an environmental map displayed as an occupied grid map, the first color information C1 is displayed as map creation progress information in the area sensed by the mobile robot 10 as it moves upward in the figure. On the other hand, as shown in Figure 4B, in the environmental map, the second color information C2 is displayed as map creation progress information in the area sensed by the mobile robot 10 as it moves downward in the figure. Furthermore, as shown in Figure 4C, in the environmental map, the third color information C3 is displayed as map creation progress information in the area sensed by the mobile robot 10 as it moves in both upward and downward directions in the figure. The color corresponding to the posture (orientation) of the mobile robot 10 should be displayed not only for obstacles BS such as pillars and walls observed in the environmental map, but also for areas where no objects exist (free areas) in the occupied grid map.

[0024] This makes it easy to visually identify the paths and other routes that the mobile robot 10 has traveled in both directions, eliminating any omissions in bidirectional travel when creating the environmental map, and enabling the efficient creation of an environmental map for self-position estimation.

[0025] Furthermore, in addition to displaying colors corresponding to the posture (orientation) of the mobile robot 10 on the environmental map, the system may also set map creation progress information that can be interpreted by the system on the environmental map. In this case, the mobile robot 10 can be made to automatically travel in such a way that map creation progress information indicating that it has traveled in both directions is set.

[0026] <2. Configuration and Operation of the Information Processing System> The configuration and operation of the information processing system to which the technology related to this disclosure is applied will be described below.

[0027] (Configuration of the Information Processing System) Figure 5 is a block diagram showing an example configuration of an information processing system to which the technology relating to this disclosure is applied.

[0028] The information processing system 100 shown in Figure 5 can be realized by combining the mobile robot 10 described above with at least one of the following: a terminal device such as a smartphone or tablet for operating and monitoring the status of the mobile robot, and an information processing device such as a PC (Personal Computer) or server that can communicate with the mobile robot.

[0029] The information processing system 100 is configured to include a LiDAR 110, a self-position estimation unit 120, a map creation progress information setting unit 130, a UI control unit 140, and a UI unit 150.

[0030] The LiDAR 110 may be mounted on a mobile robot 10 or on a user-portable terminal device. The LiDAR 110 is configured as a sensor device for sensing the environment and acquires observation data (point cloud) as sensor information. The observation data acquired by the LiDAR 110 is supplied to the self-position estimation unit 120 and the map creation progress information setting unit 130. The sensor device according to this disclosure is not limited to LiDAR, but can be any device capable of map creation using SLAM. The sensor device according to this disclosure may be, for example, a camera capable of acquiring image information as sensor information, or a ToF (Time of Flight) sensor capable of acquiring distance measurement information as sensor information.

[0031] The self-position estimation unit 120 creates an environmental map MP for self-position estimation (hereinafter referred to as the self-position estimation map MP) by estimating its own position based on observation data acquired by the LiDAR 110 in response to the movement of the LiDAR 110 (mobile robot 10 or user). The self-position estimation map MP is supplied to the UI control unit 140 as appropriate. In addition, the position and orientation of the LiDAR 110 at the time the observation data was acquired, obtained through self-position estimation, are supplied to the map creation progress information setting unit 130 as self-position data.

[0032] The map creation progress information setting unit 130 sets map creation progress information for each region of the self-position estimation map MP, based on the observation data acquired by the LiDAR 110 and the self-position data from the self-position estimation unit 120, according to the orientation of the LiDAR 110 when the observation data was acquired. The map creation progress information set for each region of the self-position estimation map MP is supplied to the UI control unit 140.

[0033] The UI control unit 140 controls the display of the self-position estimation map MP in the UI unit 150. The UI control unit 140 also displays the map creation progress information from the map creation progress information setting unit 130 on the self-position estimation map MP displayed in the UI unit 150 as display information corresponding to the self-position estimation map MP.

[0034] The UI unit 150 may be a monitor mounted on the mobile robot 10, or it may be a monitor on a terminal device that can be carried by the user. The UI unit 150 may include a touchpad, buttons, etc., that the user can use to input data.

[0035] With the above configuration, while creating a self-position estimation map MP based on the observation data acquired by the LiDAR 110, map creation progress information can be set for each region of the self-position estimation map MP according to the orientation of the LiDAR 110 when the observation data was acquired.

[0036] In this first embodiment, if the information processing system 100 is composed of a terminal device equipped with LiDAR 110, it can be carried by a user moving through an environment, allowing them to create a self-position estimation map MP while simultaneously setting map creation progress information.

[0037] Furthermore, in a second embodiment, if the information processing system 100 is composed of a mobile robot 10 equipped with LiDAR 110, the user can operate the mobile robot 10 to move through the environment, thereby creating a self-position estimation map MP and setting map creation progress information.

[0038] Furthermore, in a third embodiment, if the information processing system 100 is composed of a mobile robot 10 that autonomously moves through the environment, the mobile robot 10 itself can autonomously move through the environment, thereby creating a self-position estimation map MP and setting map creation progress information.

[0039] Furthermore, in a fourth embodiment, if the information processing system 100 is configured to include an information processing device capable of communicating with a mobile robot 10 equipped with a LiDAR 110, the information processing device can communicate with the mobile robot 10 moving through the environment to create a self-position estimation map MP and set map creation progress information.

[0040] (Configuration of Mobile Robot) FIG. 6 is a diagram showing a configuration example of a general mobile robot 10 to which the technology according to the present disclosure is not applied. Here, the mobile robot 10 is configured as an AMR, but may also be configured as a cleaning robot, a food delivery robot, a drone capable of flying in the air, or a vehicle capable of autonomous driving.

[0041] The mobile robot 10 shown in FIG. 6 is configured to include a LiDAR 210, a self-position estimation unit 220, an obstacle map creation unit 230, a wheel drive unit 240, a follow-up control unit 250, a path planning unit 260, a movement control unit 270, an overall control unit 280, a UI control unit 290, and a UI unit 300.

[0042] The LiDAR 210 is mounted, for example, only in the traveling direction (front) of the mobile robot 10, and acquires observation data (point cloud) by sensing a limited sensing range. The observation data acquired by the LiDAR 210 is supplied to the self-position estimation unit 220 and the obstacle map creation unit 230.

[0043] The self-position estimation unit 220 creates a self-position estimation map MP by performing self-position estimation based on the observation data acquired by the LiDAR 210 in response to the movement of the mobile robot 10. The self-position estimation map MP is appropriately supplied to the UI control unit 290. In addition, the position and orientation (self-position data) of the LiDAR 210 (mobile robot 10) obtained by self-position estimation are supplied to the obstacle map creation unit 230 and the follow-up control unit 250.

[0044] The obstacle map creation unit 230 creates an obstacle map BM around the mobile robot 10 by mapping the observation data acquired by the LiDAR 210 based on the self-position data from the self-position estimation unit 220. The created obstacle map BM is supplied to the path planning unit 260.

[0045] The wheel drive unit 240 is composed of a servo controller, an encoder, a motor, and the like, and drives wheels (not shown) included in the mobile robot 10 under the control of the follow-up control unit 250. Note that when the mobile robot 10 is configured as a drone, a rotor drive unit that drives rotors (propellers) is provided in place of the wheel drive unit 240.

[0046] The follow-up control unit 250 generates speed command information for following the path plan generated by the path planning unit 260 based on the self-position data from the self-position estimation unit 220, thereby controlling the wheel drive unit 240.

[0047] The path planning unit 260 generates a path plan that avoids obstacles indicated by the obstacle map BM from the obstacle map creation unit 230 based on the destination and movement route acquired by the movement control unit 270, and supplies the generated path plan to the follow-up control unit 250.

[0048] The movement control unit 270 controls the movement of the mobile robot in the environment. The movement control unit 270 acquires movement route information RT including the destination and movement route specified in the UI unit 300 from among movement route information RT including various purposes and movement routes, and supplies the acquired information to the path planning unit 260.

[0049] The overall control unit 280 controls the entire mobile robot 10. For example, the overall control unit 280 determines the traveling mode of the mobile robot 10 according to the traveling mode specified in the UI unit 300.

[0050] The UI control unit 290 controls the display of the self-position estimation map MP on the UI unit 300, the position and orientation of the mobile robot 10 on the self-position estimation map MP, and the movement route information RT. In addition, the UI control unit 290 controls generation of screens displayed on the UI unit 300, invocation of functions possessed by the mobile robot 10, and the like.

[0051] The UI unit 300 is composed of a monitor or the like mounted on the mobile robot 10. The UI unit 300 may include a touchpad, buttons, or the like that allow a user to perform operation inputs.

[0052] When the mobile robot 10 configured as described above moves through an environment such as a warehouse where the information processing system 100 in Figure 5 has created a self-position estimation map MP, the user can move through the environment while holding a terminal device and import and use the self-position estimation map MP created by the information processing system 100.

[0053] Figure 7 shows an example configuration of a mobile robot 10 to which the technology described herein is applied.

[0054] The mobile robot 10 shown in Figure 7 has the same configuration as the mobile robot 10 described with reference to Figure 6, and further includes a map creation progress information setting unit 130.

[0055] The map creation progress information setting unit 130, provided on the mobile robot 10, sets map creation progress information for each region of the self-position estimation map MP, based on the observation data acquired by the LiDAR 210 and the self-position data from the self-position estimation unit 220, according to the orientation of the LiDAR 210 (mobile robot 10) when the observation data was acquired. The map creation progress information PR set for each region of the self-position estimation map MP is supplied to the movement control unit 270 and the UI control unit 290.

[0056] The movement control unit 270 updates the movement route information RT so that the map creation progress information PR set by the map creation progress information setting unit 130 indicates that the robot has traveled in both directions along the movement route indicated by the movement route information RT, and supplies this updated movement route information RT to the route planning unit 260. This allows the mobile robot 10 to be driven automatically so that map creation progress information indicating that it has traveled in both directions is set.

[0057] The UI control unit 290 displays the map creation progress information PR from the map creation progress information setting unit 130 on the self-position estimation map MP displayed on the UI unit 300 as display information that corresponds to the self-position estimation map MP.

[0058] As described above, the mobile robot 10 shown in Figure 7 can set map creation progress information PR while creating a self-position estimation map MP, either by moving the mobile robot 10 through user operation or by the mobile robot 10 autonomously moving through the environment.

[0059] (Operation of the Information Processing System) The operation of the information processing system 100, which was described with reference to Figure 5, will now be explained with reference to the flowchart in Figure 8. The information processing system 100, which is the main component of the operation, may be configured in any of the first to fourth embodiments described above, but here it will be described as being configured in the second embodiment (a mobile robot 10 that moves through the environment by user operation).

[0060] In step S1, the self-position estimation unit 120 creates a self-position estimation map MP by estimating its own position based on observation data acquired by the LiDAR 110 in response to the movement of the LiDAR 110 (mobile robot 10).

[0061] In step S2, the map creation progress information setting unit 130 sets map creation progress information for each region of the self-position estimation map MP, according to the posture of the LiDAR 110 (mobile robot 10) when the observation data was acquired.

[0062] (Details of setting map creation progress information) Next, referring to the flowchart in Figure 9, we will explain the details of setting the map creation progress information by the map creation progress information setting unit 130, which is performed in step S2 of the flowchart in Figure 8.

[0063] In step S11, the map creation progress information setting unit 130 sets map creation progress information X (X1 / X2) corresponding to the posture of the LiDAR 110 (mobile robot 10).

[0064] Here, map creation progress information X is displayed as information corresponding to the self-position estimation map MP. For example, map creation progress information X1 is red color information, and map creation progress information X2 is blue color information. In other words, in this example, the occupied and free areas of each grid of the self-position estimation map MP, which is an occupied grid map, are filled in with color information corresponding to the posture of the mobile robot 10. Note that the self-position estimation map MP, which is an occupied grid map, is not limited to a two-dimensional grid map, but may also be a three-dimensional voxel map.

[0065] The reference point for the attitude of the mobile robot 10 may be the position and attitude of the mobile robot 10 at the start of map creation in the coordinate system of the self-position estimation map MP created by the self-position estimation unit 120, or it may be defined by an AR marker placed in the environment. For example, in the self-position estimation map MP, if the mobile robot 10 is moving upwards, red map creation progress information X1 is set, and if the mobile robot 10 is moving downwards, blue map creation progress information X2 is set.

[0066] In step S12, the map creation progress information setting unit 130 acquires input point clouds (observation data) from the LiDAR 110. Environmental structures such as walls observed in the input point cloud become candidates for projection destinations (setting targets) of the map creation progress information X. Subsequent processing is performed for all acquired input point clouds.

[0067] In step S13, the map creation progress information setting unit 130 determines the coordinates of a point (grid) on the self-position estimation map MP, which will be the projection destination for the map creation progress information X (color information). If the distance between the current position of the mobile robot 10 and the projection destination coordinates is greater than a certain distance, the input point cloud for the environmental structure may not be sufficiently observed, and the subsequent processing may be skipped.

[0068] In step S14, the map creation progress information setting unit 130 determines whether map creation progress information X has already been set for the coordinates determined as the projection destination. If it is determined that map creation progress information X (X1 or X2) has already been set for the coordinates of the projection destination, the process proceeds to step S15.

[0069] In step S15, the map creation progress information setting unit 130 determines whether the set map creation progress information X is different from the map creation progress information X corresponding to the current orientation. For example, if red map creation progress information X1 is already set at a certain point, it is determined whether the current orientation is one in which blue map creation progress information X2 is set. If it is determined that the set map creation progress information X is different from the map creation progress information X corresponding to the current orientation, the process proceeds to step S16.

[0070] In step S16, the map creation progress information setting unit 130 sets map creation progress information Y for the coordinates, which is different from both map creation progress information X1 and map creation progress information X2. For example, if red map creation progress information X1 has already been set for a certain point, then when the current orientation is such that blue map creation progress information X2 is set, green map creation progress information Y is set. In other words, if the mobile robot 10 travels both upwards and downwards in the area, green map creation progress information Y will be set.

[0071] On the other hand, if it is determined in step S15 that the set map creation progress information X is the same as the map creation progress information X corresponding to the current posture, step S15 is skipped. That is, if red map creation progress information X1 has already been set at a certain point, and the current posture is the posture in which the red map creation progress information X1 is set, the mobile robot 10 is only moving upwards in that area, so the green map creation progress information Y is not set.

[0072] Furthermore, if neither of the map creation progress information X(X1, X2) is set for the projection destination coordinates in step S14, the process proceeds to step S17.

[0073] In step S17, the map creation progress information setting unit 130 sets map creation progress information X corresponding to the current orientation. That is, if the mobile robot 10 is moving upward in the area, red map creation progress information X1 is set, and if it is moving downward in the area, blue map creation progress information X2 is set.

[0074] Furthermore, the map creation progress information, which is set as color information for each area, may include color intensity information corresponding to the distance to the environmental structure (darker for closer, lighter for farther, etc.) to express its reliability.

[0075] As a result of the above processing, pathways and other areas traveled by the mobile robot 10 in both directions become easily visible, eliminating the possibility of missing bidirectional travel in the creation of the environmental map, and enabling the efficient creation of an environmental map for self-position estimation.

[0076] (Example of displaying map creation progress information) Referring to Figure 10, an example of displaying map creation progress information as display information (color information) corresponding to the self-position estimation map will be explained.

[0077] As shown in Figure 10A, the display information DP1 may be displayed as color information to indicate the progress of map creation, such that a certain area around the mobile robot 10 (LiDAR 110) is filled in, corresponding to the movement path of the mobile robot 10 in the environment.

[0078] Furthermore, if the self-position estimation map is created as a point cloud map, as shown in Figure B, display information DP2 as color information may be displayed for obstacles BS sensed by the mobile robot 10 (LiDAR 110) in the environment as map creation progress information.

[0079] Furthermore, if the self-position estimation map is created as an occupied grid map, as shown in Figure C, display information DP3 as color information may be displayed to fill in the occupied area (obstacle BS) and the free area (movement path of the mobile robot 10) as map creation progress information.

[0080] (Method for determining map creation progress information) Referring to Figure 11, the method for determining map creation progress information according to the posture of the mobile robot 10 (LiDAR 110) will be explained.

[0081] In the coordinate system of the environmental map (self-position estimation map MP) shown in Figure 11, the 3 o'clock direction is defined as 0 degrees and the 12 o'clock direction as 90 degrees. In such an environmental map, if the attitude (orientation) of the mobile robot 10 is, for example, within the attitude range AT1 (first attitude) between 45 degrees and 135 degrees, then, for example, red map creation progress information X1 is set. Also, if the attitude (orientation) of the mobile robot 10 is, for example, within the attitude range AT2 (second attitude) between 45 degrees and -135 degrees, then, for example, blue map creation progress information X2 is set.

[0082] Therefore, if the self-localization map MP is created as an occupied grid map, unobserved unknown areas are colored gray, areas where the mobile robot 10 traveled only within attitude range AT1 are colored red, areas where the mobile robot 10 traveled only within attitude range AT2 are colored blue, and areas where the mobile robot 10 traveled (in both directions) within attitude ranges AT1 and AT2 are colored green. Furthermore, obstacles and free areas in the occupied grid map may be represented by different colors or shades of color.

[0083] In typical indoor spaces such as warehouses, passageways are provided along orthogonal vertical and horizontal axes. Therefore, as explained with reference to Figure 11, determining the map creation progress information with a 45-degree boundary prevents frequent changes in the orientation determination of the mobile robot 10 (LiDAR 110). However, the boundary for determining the map creation progress information is not limited to the example in Figure 11 and may be arbitrarily set according to the environment in which the mobile robot 10 (LiDAR 110) moves.

[0084] The display information (color information) set as map creation progress information according to the posture of the mobile robot 10 (LiDAR 110) is not limited to two (two colors), but may be set as different color information for each angle, such as a color wheel. In this case, the area that the mobile robot 10 has traveled in both directions may be represented by achromatic colors obtained by mixing complementary colors. Furthermore, the display information set as map creation progress information is not limited to color information, but may also be texture information such as diagonal lines or polka dots, or color intensity information.

[0085] Furthermore, the map creation progress information is not limited to the display information described above, but may also be set as attitude information representing the posture of the mobile robot 10 (LiDAR 110). The attitude information referred to here is, for example, angle information based on the origin of the coordinate system of the self-position estimation map (environment map). In this case, it is possible to determine whether or not the mobile robot 10 has traveled in both directions through a given area by the difference between the angle information set in each area of ​​the self-position estimation map and the angle information representing the current posture of the mobile robot 10.

[0086] (Example of display indicating bidirectional movement) An example of a display indicating that the mobile robot 10 (LiDAR110) has moved in both directions will be explained.

[0087] (1) First display example When the robots travel in different directions, the respective color information sets may overlap and be represented by a mixed color information. Specifically, as shown in Figure 12, when the mobile robot 10 travels upward in the figure over a certain area, for example, red semi-transparent color information C1 is displayed. Subsequently, when the mobile robot 10 travels downward in the figure over the same area, for example, blue semi-transparent color information C2 is displayed. However, in areas where the mobile robot 10 travels in both directions, purple semi-transparent color information C12, which is a mixture of red semi-transparent and blue semi-transparent, is displayed.

[0088] (2) Second display example Even if an area has already been observed in an occupied grid map or the like, areas traveled in a different direction may be represented by color information indicating that travel was in both directions. Specifically, as shown in Figure 13, when the mobile robot 10 travels in the upward direction in the figure over an area, for example, red color information C1 is displayed. Subsequently, when the mobile robot 10 travels in the downward direction in the figure over the same area, for example, blue color information C2 is displayed. However, in areas where the mobile robot 10 traveled in both directions, green color information C3, which is different from both red and blue, is displayed.

[0089] <3. Setting Map Creation Progress Information During Automatic Driving> The mobile robot 10, as explained with reference to Figure 7, can create a map for self-position estimation (environmental map) while automatically driving and can also set map creation progress information. In the mobile robot 10, automatic driving is achieved by the cooperative operation of each functional block: the wheel drive unit 240, the follow control unit 250, the route planning unit 260, the movement control unit 270, and the overall control unit 280. In parallel with automatic driving, the mobile robot 10 also creates a map for self-position estimation by the self-position estimation unit 220 and sets map creation progress information by the map creation progress information setting unit 130.

[0090] Here, referring to the flowchart in Figure 14, the operation of the mobile robot 10 during autonomous driving will be explained. In this example, the mobile robot 10 will start autonomous driving from a certain point in an unknown environment such as a warehouse, and will begin creating an occupied grid map at the same time.

[0091] In step S31, the mobile robot 10 moves forward until it approaches an obstacle such as a wall.

[0092] In step S32, when the mobile robot 10 approaches an obstacle, it moves along the obstacle so that its body is on the right side of the obstacle (with the obstacle on its left side).

[0093] In step S33, the mobile robot 10 determines whether it has returned to a location near a place it has already passed. If it determines that it has not returned to a location near a place it has already passed, it assumes that there is a place it has not passed through with an obstacle on its left, and returns to step S32.

[0094] On the other hand, if it is determined that the robot has returned to a location it has already passed, the process proceeds to step S34, and the mobile robot 10 moves forward until an obstacle approaches to the left side of the robot.

[0095] In step S35, when the mobile robot 10 approaches an obstacle, it moves along the obstacle so that the robot is on the left side of the obstacle (with the obstacle on the right side).

[0096] In step S36, the mobile robot 10 determines whether it has returned to a location near a place it has already passed. If it determines that it has not returned to a location near a place it has already passed, it assumes that there is a place it has not passed through with an obstacle on its right, and returns to step S35.

[0097] On the other hand, if it is determined that the robot has returned to a location it has already passed through, the process proceeds to step S37 and the mobile robot 10 stops. At this point, the mobile robot 10 has traveled along all obstacles (such as walls) in the environment, and a complete self-position estimation map has been created. In addition, by searching for traversable paths and sensing the environment, the mobile robot 10 sets map creation progress information X1 or map creation progress information X2 for the area it has traveled.

[0098] In other words, in step S38, the mobile robot 10 determines whether or not there is a location where map creation progress information X1 or map creation progress information X2 has been set. Subsequent processing can be performed by the mobile control unit 270 referring to the map creation progress information set by the map creation progress information setting unit 130.

[0099] If it is determined in step S38 that there are locations where map creation progress information X1 or map creation progress information X2 has been set, the process proceeds to step S39.

[0100] In step S39, the mobile robot 10 approaches the vicinity of the location from a direction opposite to that of the set map creation progress information X (X1 / X2).

[0101] Then, in step S40, the mobile robot 10 sets the map creation progress information Y at the location. The process returns to step S38 and the subsequent processing is repeated.

[0102] Now, in step S38, if it is determined that there are no locations where map creation progress information X1 or map creation progress information X2 is set, that is, if map creation progress information Y is set for all locations, the process ends.

[0103] For example, as shown in Figure 15A, the mobile robot 10 starts automatic driving from a certain point, and then, as shown in Figure 15B, moves upward in the figure, thereby setting map creation progress information X1 in that area. Next, it approaches an obstacle and moves to the right in the figure, thereby setting map creation progress information X2 in that area. After that, as shown in Figure 15C, by driving in the opposite direction through the area where map creation progress information X1 or map creation progress information X2 has been set, map creation progress information Y is set in that area.

[0104] Through the above process, the mobile robot 10 can set map creation progress information while automatically driving, and then autonomously search for areas where it has not traveled in both directions. As a result, it becomes possible to efficiently create an environmental map for self-position estimation.

[0105] Furthermore, in the process described above, the map creation progress information may be set not only as display information, but also as attitude information (angle information) representing the attitude of the mobile robot 10. This makes it possible to determine whether or not the robot has traveled in both directions through a given area by comparing the angle information set for each area of ​​the self-position estimation map with the angle information representing the current attitude of the mobile robot 10.

[0106] Furthermore, the method for exploring the environment during the automatic driving of the mobile robot 10 is not limited to the processes executed in steps S31 to S37 of Figure 14; other exploration methods can also be used.

[0107] <4. Other Examples of Information Processing System Configurations> As an information processing system to which the technology relating to this disclosure is applied, the mobile robot 10 shown in Figure 7 is equipped with all the necessary components, but some of these components may be provided by external devices.

[0108] Figure 16 is a block diagram showing another example of an information processing system configuration.

[0109] The information processing system shown in Figure 16 consists of a mobile robot 410 and a terminal device 420.

[0110] The terminal device 420 can be configured as a UI terminal capable of communicating with the mobile robot 410. The terminal device 420 consists only of a UI control unit 290 and a UI unit 300, with the mobile robot 410 providing all other components. In such a configuration, the terminal device 420 can be made with a minimal configuration.

[0111] Alternatively, the terminal device 420 may consist only of the UI unit 300, with the mobile robot 410 providing the other components. In this case, the terminal device 420 only needs to be able to receive and display display data such as the self-position estimation map and map creation progress information created by the mobile robot 410 (UI control unit 290).

[0112] Figure 17 is a block diagram showing yet another example of the configuration of an information processing system.

[0113] The information processing system shown in Figure 17 consists of a mobile robot 510 and an information processing device 520.

[0114] The information processing device 520 can be configured as a server or the like that enables high-speed, low-latency wireless communication with the mobile robot 510. The mobile robot 510 is equipped only with a LiDAR 210 and a wheel drive unit 240, with the information processing device 520 providing all other components. In such a configuration, the mobile robot 510 can be made with a minimal configuration.

[0115] Furthermore, although not shown in the diagram, if the display accuracy of the self-position estimation map and map creation progress information is slightly reduced due to transmission delays or data loss, but this does not significantly affect the visual confirmation of bidirectional travel, then only the map creation progress information setting unit 130 may be provided by an external device (such as a UI terminal or server). In this case, it is sufficient that the external device can be closely synchronized with the sensing and self-position estimation by the mobile robot.

[0116] <5. Examples of Computer Hardware Configurations> The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up the software are installed from a program storage medium onto a computer that is built into dedicated hardware, or a general-purpose personal computer.

[0117] Figure 18 is a block diagram showing an example of the hardware configuration of a computer that executes the series of processes described above by a program. Some components of the mobile robot 10 and the information processing system 100 can be configured, for example, by a computer 900 having a configuration similar to that shown in Figure 18.

[0118] The CPU (Central Processing Unit) 901, ROM (Read Only Memory) 902, and RAM (Random Access Memory) 903 are interconnected by a bus 904.

[0119] An input / output interface 905 is further connected to the bus 904. An input unit 906 consisting of a keyboard, mouse, etc., and an output unit 907 consisting of a display, speakers, etc. are connected to the input / output interface 905. In addition, a storage unit 908 consisting of a hard disk, non-volatile memory, etc., a communication unit 909 consisting of a network interface, etc., and a drive 910 that drives removable media 911 are connected to the input / output interface 905.

[0120] In the computer 900 configured as described above, the CPU 901 loads, for example, a program stored in the memory unit 908 into the RAM 903 via the input / output interface 905 and the bus 904, and executes it, thereby performing the series of processes described above.

[0121] The program executed by the CPU 901 is recorded on removable media 911, for example, or provided via a wired or wireless transmission medium such as a local area network, the internet, or digital broadcasting, and installed in the storage unit 908.

[0122] The program executed by the computer 900 may be a program that is processed chronologically in the order described herein, or it may be a program that is processed in parallel or at necessary times, such as when a call is made.

[0123] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are located in the same enclosure. Therefore, multiple devices housed in separate enclosures and connected via a network, and a single device containing multiple modules within a single enclosure, are both considered systems.

[0124] The effects described herein are illustrative and not limited to those described herein, and other effects may also occur.

[0125] The embodiments of this disclosure are not limited to those described above, and various modifications are possible without departing from the spirit of this disclosure.

[0126] For example, an embodiment of the present disclosure can take the form of cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0127] Furthermore, each step described in the flowchart above can be performed by a single device, or it can be divided and performed by multiple devices.

[0128] Furthermore, if a single step includes multiple processes, those processes can be executed by a single device or shared among multiple devices.

[0129] The effects described herein are merely illustrative and not limited to those described herein; other effects may also occur.

[0130] Furthermore, the technology relating to this disclosure can take the following configuration: (1) An information processing method comprising creating an environmental map based on sensor information acquired by a sensor device that senses the environment, and setting map creation progress information for each region of the environmental map according to the orientation of the sensor device when the sensor information was acquired. (2) The information processing method according to (1), wherein if the orientation of the sensor device is a first orientation, first map creation progress information is set for the region, and if the orientation of the sensor device is a second orientation opposite to the first orientation, second map creation progress information is set for the region. (3) The information processing method according to (2), wherein if one of the first map creation progress information and the second map creation progress information has already been set for a region, the other of the first map creation progress information and the second map creation progress information is set for the region. (4) The information processing method according to (3), wherein the orientation of the sensor device is the orientation of the sensor device in the coordinate system of the environmental map. (5) The information processing method according to (4), wherein the map creation progress information is display information displayed in correspondence with the environment map. (6) The information processing method according to (5), wherein the display information is displayed for the movement path of the sensor device in the environment. (7) The information processing method according to (5), wherein the display information is displayed for obstacles sensed by the sensor device in the environment. (8) The information processing method according to any one of (5) to (7), wherein the display information includes at least one of color information, texture information, and color intensity information. (9) The information processing method according to (8), wherein the first map creation progress information, the second map creation progress information, and the third map creation progress information are each different color information. (10) The information processing method according to (8), wherein the first map creation progress information and the second map creation progress information are each different color information, and the third map creation progress information is a mixture of the first map creation progress information and the second map creation progress information. (11) The map creation progress information is attitude information representing the attitude of the sensor device. The information processing method described in (4).(12) The information processing method according to (11), wherein the posture information includes angle information with respect to the origin of the coordinate system. (13) The information processing method according to any one of (1) to (12), wherein the environment map is a point cloud map or an occupied grid map. (14) The information processing method according to any one of (1) to (13), wherein the terminal device equipped with the sensor device is held by a user moving through the environment, generates the environment map, and sets the map creation progress information. (15) The information processing method according to any one of (1) to (13), wherein the mobile robot equipped with the sensor device moves through the environment by user operation, generates the environment map, and sets the map creation progress information. (16) The information processing method according to any one of (1) to (13), wherein the mobile robot equipped with the sensor device moves autonomously through the environment, generates the environment map, and sets the map creation progress information. (17) An information processing method according to any one of (1) to (13), wherein a mobile robot equipped with the sensor device is moving in the environment and an information processing device capable of communicating with the mobile robot generates the environment map and sets the map creation progress information. (18) An information processing system including a sensor device for sensing an environment and an information processing device that creates an environment map based on sensor information acquired by the sensor device and sets map creation progress information for each of a plurality of regions in the environment map according to the orientation of the sensor device when the sensor information was acquired. (19) The information processing system according to (18), further including a mobile robot that moves in the environment, wherein the sensor device is mounted on the mobile robot. (20) A program for causing a computer to perform a process that includes creating an environment map based on sensor information acquired by a sensor device for sensing an environment and setting map creation progress information for each of a plurality of regions in the environment map according to the orientation of the sensor device when the sensor information was acquired.

[0131] 10 Mobile robot, 110 LiDAR, 120 Self-position estimation unit, 130 Map creation progress information setting unit, 140 UI control unit, 150 UI unit, 210 LiDAR, 220 Self-position estimation unit, 230 Obstacle map creation unit, 240 Wheel drive unit, 250 Follow control unit, 260 Route planning unit, 270 Movement control unit, 280 Overall control unit, 290 UI control unit, 300 UI unit, 410 Mobile robot, 420 Terminal device, 510 Mobile robot, 520 Information processing device

Claims

1. An information processing method comprising: creating an environmental map based on sensor information acquired by a sensor device that senses the environment; and setting map creation progress information for each region of the environmental map, corresponding to the orientation of the sensor device at the time the sensor information was acquired.

2. The information processing method according to claim 1, wherein when the orientation of the sensor device is a first orientation, first map creation progress information is set for the region, and when the orientation of the sensor device is a second orientation opposite to the first orientation, second map creation progress information is set for the region.

3. The information processing method according to claim 2, wherein if the other of the first and second map creation progress information is set for a region in which one of the first and second map creation progress information has already been set, the third map creation progress information is set for the region.

4. The information processing method according to claim 3, wherein the orientation of the sensor device is the orientation of the sensor device in the coordinate system of the environmental map.

5. The information processing method according to claim 4, wherein the map creation progress information is display information that is displayed in correspondence with the environmental map.

6. The information processing method according to claim 5, which displays the display information along the movement path of the sensor device in the environment.

7. The information processing method according to claim 5, which displays the display information for an obstacle sensed by the sensor device in the environment described above.

8. The information processing method according to claim 5, wherein the display information includes at least one of color information, texture information, and color intensity information.

9. The information processing method according to claim 8, wherein the first map creation progress information, the second map creation progress information, and the third map creation progress information are each different color information.

10. The information processing method according to claim 8, wherein the first map creation progress information and the second map creation progress information are each different color information, and the third map creation progress information is the color information obtained by mixing the first map creation progress information and the second map creation progress information.

11. The information processing method according to claim 4, wherein the map creation progress information is attitude information representing the attitude of the sensor device.

12. The information processing method according to claim 11, wherein the attitude information includes angle information with respect to the origin of the coordinate system.

13. The information processing method according to claim 1, wherein the environmental map is a point cloud map or an occupied grid map.

14. The information processing method according to claim 1, wherein the terminal device equipped with the sensor device is carried by a user moving through the environment, generates the environment map, and sets the map creation progress information.

15. The information processing method according to claim 1, wherein a mobile robot equipped with the sensor device moves through the environment by user operation, generates the environment map, and sets the map creation progress information.

16. The information processing method according to claim 1, wherein a mobile robot equipped with the sensor device autonomously moves in the environment, generates the environment map, and sets the map creation progress information.

17. The information processing method according to claim 1, wherein a mobile robot equipped with the sensor device is moving in the environment, and an information processing device capable of communicating with the mobile robot generates the environment map and sets the map creation progress information.

18. An information processing system including a sensor device for sensing the environment, and an information processing device that creates an environmental map based on sensor information acquired by the sensor device, and sets map creation progress information for each of a plurality of regions in the environmental map, corresponding to the orientation of the sensor device when the sensor information was acquired.

19. The information processing system according to claim 18, further comprising a mobile robot that moves in the environment, wherein the sensor device is mounted on the mobile robot.

20. A program for causing a computer to perform a process that includes creating an environmental map based on sensor information acquired by a sensor device that senses the environment, and setting map creation progress information for each of several regions in the environmental map, corresponding to the orientation of the sensor device when the sensor information was acquired.