Information processing device and information processing method

Abstract

An information processing device according to one embodiment of the present invention comprises: a path determination unit that uses an environmental map which represents a target space to determine a path on which radio wave intensity measurement is to be performed; an output unit that outputs a modified path obtained by modifying said path, such that a specific location in the target space for which it is predicted that a radio wave environment will become worse than a reference is measured in more detail than other locations; a movement unit that autonomously moves the device along the modified path; and a measurement unit that measures radio wave intensity.

Description

Information processing device and information processing method

[0001] The present invention relates to a technique for controlling a device relating to radio field intensity measurement.

[0002] There are known techniques relating to devices that measure radio wave intensity. For example, Patent Document 1 discloses an invention of a robot that measures radio wave intensity while autonomously moving in an indoor area.

[0003] Japanese Patent Application Laid-Open No. 2012-173051

[0004] The robot described in Patent Document 1 simply moves along a route designated by an administrator on a map of an indoor area and measures radio wave intensity.

[0005] In response to this, the present invention provides a technique for performing detailed measurements in locations where the radio wave environment is thought to be worse.

[0006] An information processing device according to one aspect of the present disclosure includes a route determination unit that determines a route for measuring radio wave intensity using an environmental map showing a target space, an output unit that outputs a modified route that modifies the route so as to measure specific locations in the target space where the radio wave environment is predicted to be worse than a standard in more detail than other locations, a movement unit that autonomously moves the device along the modified route, and a measurement unit that measures radio wave intensity.

[0007] An information processing method according to another aspect of the present disclosure includes a step in which a computer determines a route for measuring radio wave intensity using an environmental map showing a target space, a step in which the computer outputs a modified route by modifying the route so as to measure specific locations in the target space where the radio wave environment is predicted to be worse than a standard in more detail than other locations, a step in which the computer autonomously moves the device along the modified route, and a step in which the computer measures radio wave intensity.

[0008] According to the present invention, detailed measurements can be performed in locations where the radio wave environment is expected to worsen.

[0009] 1 is a diagram illustrating an example of the system configuration of an information processing system 1. A diagram illustrating an example of the functional configuration of the information processing system 1. A diagram illustrating an example of the hardware configuration of an information processing device 10. A sequence chart illustrating an example of an environmental map creation method and a route correction method according to terrain in the information processing system 1. A diagram illustrating environmental map creation using SLAM processing. A diagram illustrating a route for radio wave intensity measurement. A diagram illustrating a corrected route. A flowchart illustrating an example of a radio wave intensity measurement method and a dynamic route correction method in the information processing system 1. A diagram illustrating a measurement database 2000. A diagram illustrating an example of real-time video. A diagram illustrating a corrected route.

[0010] 1. Configuration FIG. 1 illustrates an example of the system configuration of an information processing system 1. In this example, the information processing system 1 (or simply referred to as the "system") is a system for measuring the strength of radio waves transmitted from wireless equipment for wireless communication (hereinafter referred to as "radio wave strength measurement") in a target space (hereinafter referred to as the "target space"), such as an indoor facility. The target space includes indoor facilities such as train stations, airports, offices, and commercial facilities. Wireless equipment (also referred to as wireless stations or radio equipment) is, in a broad sense, electrical equipment related to wireless communication, and includes, for example, base stations, access points, and routers. Radio wave strength measurement is, for example, a test, experiment, or survey that collects actual measurements of the radio wave strength of radio waves transmitted by an access point pre-installed in the target space, known as the received signal strength indicator (RSSI). Note that the measurement items in radio wave strength measurement are not limited to RSSI, and may also include RSRP, RSRQ, etc.

[0011] The results of radio wave strength measurements are used to estimate the position of a terminal in an indoor facility (so-called "indoor positioning"). Therefore, in field measurements, it is necessary to collect radio wave information with higher accuracy depending on the target space. However, because the radio wave conditions in the target space deteriorate due to various factors such as the surrounding environment, it is difficult to consistently obtain high-quality radio wave strength measurements overall. In particular, in indoor spaces, areas with complex topography and crowded areas are prone to adverse effects on radio wave propagation and radio wave attenuation due to human bodies. Therefore, the inventors wondered whether it would be possible to ensure the overall quality of radio wave strength measurements by prioritizing radio wave strength measurements in such areas.

[0012] In particular, in recent years, due to the need for resources such as time and personnel, demonstration tests have been conducted to introduce robots such as measurement rovers into radio wave intensity measurements. Therefore, it is important to apply the technology related to robot control to collect highly accurate radio wave information even in areas of the target space where the radio wave environment is likely to deteriorate. The present invention provides a robot control system that performs highly accurate radio wave intensity measurements according to the target space.

[0013] The information processing system 1 has a robot 100 and an access point 800 in a target space S1. The robot 100 has an information processing device 10 and an external device 20. These devices are distinguished as functional elements relating to different controls, namely, internal control and external control of the robot 100. The access point 800 (also simply abbreviated as "AP800") is a wireless communication device installed in the target space S1. The information processing device 10 (or the robot 100) and the access point 800 are connected via a network 9. Here, the functions and roles of each device will be described.

[0014] The robot 100 is a machine (or device) in the information processing system 1 that measures radio wave intensity in a target space S1. The robot 100 may be, for example, a rover, a work robot, or a mobile object. The target space S1 is a space where radio wave intensity measurement is performed. When measuring radio wave intensity, it is assumed that multiple access points 800 are installed in the target space S1. When distinguishing between the multiple access points 800, they are referred to as access point 801, access point 802, .... In this example, the robot 100 moves through the target space S1 and performs radio wave intensity measurement (or simply measurement) at predetermined times. The measurement target is the actual measured value of radio wave intensity emitted from the access point 800. Functionally, the robot 100 implements two different types of devices: an information processing device 10 and an external device 20.

[0015] The information processing device 10 is an information processing device in the information processing system 1. In this example, the information processing device 10 (an example of the device itself) is implemented in, for example, a robot 100 and performs internal control of the robot. Specifically, the information processing device 10 controls the movement and measurement of the robot 100 during on-site measurement. The information processing device 10 also cooperates with an external device 20 to acquire various data. The data includes, for example, measurement data, image data, or sensor data. Specifically, the information processing device 10 acquires data on measured radio wave intensity from the external device 20 and reads image data captured of the surrounding environment. The detailed operation of the information processing device 10 will be described later.

[0016] The external device 20 is a general term for functional means including various devices mounted on the robot 100 in the information processing system 1. The external device 20 uses various functional means to measure or acquire peripheral information and physically move the robot 100. For this purpose, the external device 20 includes, for example, an imaging means, a measuring means, a moving means, and sensors. As will be described in detail later, for example, the imaging means is a camera for capturing images of the target space S1. The measuring means is a receiver for measuring the radio wave intensity of the access point 800. The moving means is a device equipped with a drive mechanism such as wheels for moving the robot 100. The sensors are sensor devices used to acquire data requested by the information processing device 10.

[0017] The access point 800 is an example of wireless equipment in the information processing system 1. In this example, the access point 800 includes, for example, a base station, a Wi-Fi access point, a wireless access point, a wireless router (modem), a wireless LAN (Local Area Network) device, a repeater, or various wireless terminals. The radio wave intensity measurement is an investigation in which the robot 100 measures radio wave intensity while moving in the target space S1 in which these access points 800 are installed in advance. When the target of the radio wave intensity measurement is radio waves emitted from an external base station or the like, the robot 100 may acquire data related to RSRP, RSRQ, and the like.

[0018] 2 is a diagram illustrating an example of the functional configuration of the information processing system 1. In this embodiment, the information processing device 10 has functional blocks (components) including a route determination unit 11, an output unit 12, a correction unit 13, a movement unit 14, a measurement unit 15, an acquisition unit 16, an estimation unit 17, a position estimation unit 18, a recording unit 190, a storage unit 191, and a control unit 192. In this example, the storage unit 191 stores various data and programs including a database, for example. In this example, the control unit 192 performs various controls.

[0019] The route determination unit 11 determines a route for measuring radio wave intensity using an environmental map showing the target space. In this example, the environmental map is map data showing a map of the target space S1. The environmental map is created, for example, by SLAM (Simultaneous Localization and Mapping) processing based on image data acquired from the external device 20. In this example, SLAM is a technology for estimating the robot's position and generating an environmental map. The method for generating the environmental map will be described later. The route determination unit 11 determines a route for the robot 100 to travel around the entire target space S1 according to the environmental map.

[0020] The output unit 12 outputs a modified route that modifies the route so as to measure specific locations in the target space where the radio wave environment is predicted to be worse than the standard in more detail than other locations. In this example, the specific locations are locations in the target space S1 where the radio wave environment is likely to deteriorate due to geographical shape or crowding. For example, the specific locations are locations that have a specific shape on an environmental map. Alternatively, the specific locations are locations where there are crowds. The term "deterioration" here refers to a state in which the measurement accuracy of radio wave strength is lower than the standard or the error between measurements is large. The prediction of deterioration of the radio wave environment in the target space S1 is made based on various data. The modified route is a route that has been modified based on the specific locations from the route previously determined by the route determination unit 11.

[0021] The correction unit 13 sets a corrected route depending on the specific location. In this example, the correction unit 13 performs a preliminary route correction depending on the topography of the target space S1 and a dynamic route correction depending on congestion. In measuring radio wave intensity, it is important to prioritize (i.e., carefully collect as much radio wave information as possible from) the specific location over other locations. Therefore, the correction unit 13 sets a detour route to an area that includes the specific location, or sets a route that increases the sampling density, for the predetermined route. Specific methods for route correction will be described later.

[0022] The movement unit 14 autonomously moves the robot 100 along the corrected path. In this example, the robot 100 is the information processing device 10. That is, the movement unit 14 cooperates with an external device 20 (e.g., a moving means) of the robot 100 to autonomously move along the corrected path.

[0023] The measurement unit 15 measures radio wave intensity. In this example, the measurement unit 15 cooperates with an external device 20 (e.g., a measurement means) to acquire measurement data, such as an actual RSSI value, for each point where measurement is performed (hereinafter referred to as a "measurement point").

[0024] The acquisition unit 16 acquires real-time video of the target space. In this example, the real-time video is data showing video (or images) during measurement captured by an external device 20 (e.g., a capturing means). The external device 20 can acquire continuous video, video, or images using the capturing means. The acquisition unit 16 acquires the real-time video from the external device 20.

[0025] The estimation unit 17 estimates the crowd based on real-time video. In this example, the crowd is an index related to the degree of congestion or overcrowding caused by the movement of people in an indoor facility, etc. Hereinafter, the index representing the degree of congestion will be referred to as the "congestion level." The estimation unit 17 estimates the congestion level around the robot 100 and outputs it to the correction unit 13. The correction unit 13 corrects the route according to the congestion level. Specifically, because the radio wave environment is likely to deteriorate in areas with large crowds (i.e., high congestion level), the correction unit 13 can change the route to prioritize measurement of those areas. The output unit 12 outputs the corrected route dynamically corrected based on the real-time video. Here, "dynamically" means during measurement and while the robot 100 is moving. This allows the robot 100 to prioritize measurement of congested areas discovered during measurement.

[0026] The position estimation unit 18 estimates the position of the robot 100 using data obtained from the sensors. In this example, the position estimation unit 18 acquires data necessary for estimating the position of the robot 100 from an external device 20 (e.g., an imaging device or sensors). The data includes, for example, image data or depth data. The position estimation unit 18 estimates the position of the robot 100 by performing, for example, SLAM processing on various data. SLAM includes, for example, Visual SLAM, LiDAR SLAM, or Depth SLAM. The position estimation unit 18 performs position estimation based on the acquired data. The correction unit 13 corrects the route based on specific locations using the position estimation. The movement unit 14 autonomously moves the robot 100 along the dynamically corrected route.

[0027] The recording unit 190 records information indicating the state of crowds (e.g., congestion level) in addition to the results of the radio wave intensity measurement (e.g., actual measurement value or location information) in a database. The congestion level recorded by the recording unit 190 is used when performing multiple measurements and setting a route to prioritize (i.e., in detail) areas with high congestion levels or areas with large changes in congestion level in previous measurements. Furthermore, when actually performing indoor positioning of a terminal in the target space S1, this congestion level can be included in training data and trained in an indoor positioning model, etc., to enable positioning that takes into account the congestion level around the user.

[0028] The external device 20 includes, for example, an imaging means, a measuring means, a moving means, and sensors. The imaging means captures images, pictures, or videos of the target space S1. The imaging means includes, for example, a monocular camera (e.g., a wide-angle camera, a fisheye camera, or a spherical camera), a compound eye camera (e.g., a stereo camera or a multi-camera), or an RGB-D camera (e.g., a depth camera or a Time of Flight camera). The measuring means measures radio wave intensity at each measurement point. The measuring means includes, for example, an antenna, a receiver, and an analyzer. The moving means is used to move the robot 100. The moving means includes, for example, wheels, caterpillar tracks, walking legs, a flying drone, or a hovercraft. The sensors are used to acquire various data. The sensors include, for example, LiDAR (Light Detection and Ranging), a laser device, or a microphone. The sensors acquire point cloud data, depth data, audio data, or the like using various devices. The external device 20 may also have various other functional means as necessary, such as a learning means, a notification means, a display means, or a power supply means.

[0029] FIG. 3 is a diagram illustrating an example of the hardware configuration of the information processing device 10. The information processing device 10 is physically configured as a computer including a processor 101, a memory 102, a storage 103, a communication device 104, an input device (optional), a display device (optional), and a bus connecting these. Each of these devices operates using power supplied from a battery (not shown). In the following description, the term "device" can be interpreted as a circuit, device, unit, etc. The hardware configuration of the information processing device 10 may be configured to include one or more of the devices shown in FIG. 3, or may be configured without including some of the devices. Furthermore, the information processing device 10 may be configured by communicating and connecting multiple devices each having a different housing.

[0030] Each function of the information processing device 10 is realized by loading specified software (programs) onto hardware such as the processor 101, memory 102, etc., so that the processor 101 performs calculations, controls communication via the communication device 104, and controls at least one of reading and writing data in the memory 102 and storage 103.

[0031] The processor 101 controls the entire computer by running, for example, an operating system. The processor 101 may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc. Furthermore, for example, a baseband signal processing unit, a call processing unit, etc. may be realized by the processor 101.

[0032] The processor 101 reads programs (program codes), software modules, data, etc. from at least one of the storage 103 and the communication device 104 into the memory 102 and executes various processes in accordance with these. The programs used are those that cause a computer to execute at least some of the operations described below. The functional blocks of the information processing device 10 may be implemented by a control program stored in the memory 102 and running on the processor 101. Various processes may be executed by one processor 101, or may be executed simultaneously or sequentially by two or more processors 101. The processor 101 may be implemented by one or more chips. The programs may be transmitted to the information processing device 10 via a telecommunications line.

[0033] The memory 102 is a computer-readable recording medium and may be configured by at least one of, for example, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), etc. The memory 102 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 102 can store executable programs (program codes), software modules, etc. for implementing the method according to this embodiment.

[0034] Storage 103 is a computer-readable recording medium, and may be composed of at least one of, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Storage 103 may also be called an auxiliary storage device.

[0035] The communication device 104 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, or a communication module.

[0036] Each device, such as the processor 101 and the memory 102, is connected by a bus for communicating information. The bus may be configured using a single bus, or different buses may be used between each device.

[0037] The information processing device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 101 may be implemented using at least one of these pieces of hardware.

[0038] In this example, the programs stored in the storage 103 include a program (hereinafter referred to as the "controller program") for causing a computer to function as a controller of the information processing system 1. When the processor 101 is executing the controller program, the processor 101, memory 102, storage 103, and communication device 104 are examples of functional blocks for operating the information processing device 10. The processor 101 is an example of a route determination unit 11, a correction unit 13, an estimation unit 17, a position estimation unit 18, and a control unit 192. At least one of the memory 102 and the storage 103 is an example of a recording unit 190 and a storage unit 191. The communication device 104 is an example of an output unit 12, a movement unit 14, a measurement unit 15, and an acquisition unit 16. The configuration of the information processing system 1 has been described above. Next, the operation of the information processing system 1 will be described.

[0039] 2. Operation 2-1. Method for Creating an Environmental Map and Method for Modifying a Route According to the Topography Figure 4 is a sequence chart illustrating a method for creating an environmental map and a method for modifying a route according to the topography in the information processing system 1. The processing in Figure 4 assumes, for example, a case in which the robot 100 itself creates an environmental map of the target space S1. The following processing is started when the robot 100 is placed in the target space S1 and receives an operation instruction from a manager or the like.

[0040] In step S101, the external device 20 moves around an arbitrary area in the target space S1. The robot 100 moves freely within the target space S1 under autonomous control using the movement means of the external device 20. For example, the robot 100 randomly wanders around the target space S1 while avoiding obstacles using various sensors mounted thereon. The external device 20 moves thoroughly throughout the entire area in accordance with the size of the target space S1. The external device 20 estimates its own position while moving and records the starting position from which it began moving.

[0041] In step S102, the information processing device 10 acquires various data from the external device 20. The external device 20 acquires data using an imaging means (e.g., a camera) or sensors (e.g., LiDAR) while moving in the target space S1. Here, a case where the external device 20 acquires image data using a camera will be described as an example. The external device 20 periodically (or periodically, at equal distances and equal intervals) captures images of the surroundings with the camera while moving the robot 100. Note that the camera used for capturing images is preferably one with a wide angle of view (capable of viewing 360 degrees around). Alternatively, the external device 20 may capture images of a 360-degree range from one point in multiple separate captures. The external device 20 outputs the captured image data to the information processing device 10.

[0042] In step S103, the information processing device 10 performs SLAM processing on the image data. This processing is processing for creating an environmental map of the target space S1. In this example, the information processing device 10 uses Visual SLAM or the like to simultaneously estimate the self-position of the robot 100 and create an environmental map of the target space S1 from the acquired image data. Note that the self-position estimation of the robot 100 is used, for example, for movement in the target space S1 by the external device 20. Here, a method for creating the environmental map will be described.

[0043] FIG. 5 is a diagram illustrating an example of environmental map creation using SLAM processing. The map data 1000 represents an environmental map of the target space S1 and the position of the robot 100. FIG. 5 illustrates, as an example, a viewpoint of the target space S1 viewed perpendicularly from the ceiling to the floor. The robot 100 is an icon representing the robot body in motion. For convenience, FIG. 5 displays an icon representing the starting point where the robot 100 began its movement and icons representing specific positions along the way. The route R0 is the path (or trajectory) traveled by the robot 100 from the start of its movement until a certain point in time. In this example, the map data 1000 represents an environmental map in the process of being created. Therefore, FIG. 5 illustrates a portion of the target space S1 cut off. The information processing device 10 sequentially creates and updates the map data 1000 each time image data is acquired from the external device 20. Finally, the information processing device 10 records the environmental map at the time when the robot 100 has completed its circumnavigation of the target space S1 in a database. The information processing device 10 may generate the map data 1000 based on movement information such as the movement speed, movement distance, or movement direction of the robot 100, or may correct at least a part of the environmental map using various image technologies. The map data 1000 may be either a two-dimensional map with a bird's-eye view as shown in FIG. 5 or a three-dimensional map using a 3D modeling technology. When creating a 3D environmental map based on image data, representative methods such as Gaussian Splatting (GS), photogrammetry, or Neural Radiance Fields (NeRF) may be used.

[0044] The processes of steps S101 to S103 are repeated until the information processing device 10 acquires a map of the entire target space S1. Alternatively, the information processing device 10 may stop the movement of the robot 100 when the robot 100 reaches the start point (or end point) or when an environmental map corresponding to a specific scale is created. The external device 20 can return the robot 100 to the start point by acquiring position data from the information processing device 10. The information processing device 10 records the created environmental map in a database. Although the environmental map has been described using a two-dimensional map as an example, this map may also be a three-dimensional map. Various maps may be created and used depending on the case, such as when an indoor facility has a multi-layer structure, when there are spatial differences in elevation due to slopes, or when a more detailed spatial map is required.

[0045] Returning to FIG. 4 , in step S104, the information processing device 10 determines a route to be used for measuring radio wave intensity. In this example, the information processing device 10 determines a route that circles the entire target space S1 as the route for measuring radio wave intensity. For example, the route may be determined based on the size of the target space S1 or the travel distance of the robot 100. Here, the route for measuring radio wave intensity will be described.

[0046] FIG. 6 is a diagram illustrating routes for measuring radio wave intensity. In this example, map M1 is an example of a map showing an environmental map. Map M1 is a completed map of map data 1000 obtained by SLAM processing. Basically, map data similar to map M1 is used for subsequent maps. In this example, route R1 is an example of a route predetermined by the information processing device 10. The route R1 determined by the information processing device 10 traces a trajectory that circles the entire target space S1, covering the area in which the robot 100 can move. In this case, the information processing device 10 may refer to information on route R0 used to create the environmental map when determining the route. Specific locations L1 and L2 are locations on map M1 where the information processing device 10 determines that the radio wave environment will deteriorate. In the figure, specific location L is depicted with hatching. In this example, the information processing device 10 detects specific locations based on image data and the environmental map obtained by SLAM processing.

[0047] Here, areas on the environmental map where the topography is likely to result in poor radio wave conditions include, for example, areas shielded by partitions or other obstacles, secluded areas, or areas with metal structures. In these areas, radio waves transmitted from the access point 800 do not reach the area, and the radio wave environment is likely to deteriorate. In particular, in the case of radio waves with low diffraction, such as 5G communication, secluded areas affect not only the accuracy of position estimation but also communication quality. Therefore, the information processing device 10 employs techniques such as image analysis or heuristics to identify areas with poor visibility (e.g., complex and intricate areas) or estimate areas with a high concentration of metal components.

[0048] In another specific process, the information processing device 10 extracts partitions through image analysis and detects areas enclosed by the partitions on the environmental map as specific locations. Furthermore, the information processing device 10 identifies openings in these areas where the partitions are open (or absent) and calculates the ratio of the size of the openings to the size of the enclosed area (for example, the total length of the area's boundaries, if determined two-dimensionally). The information processing device 10 determines areas where this ratio is smaller than a reference value as "areas blocked by partitions" (i.e., detects them as specific locations). Similarly, the information processing device 10 detects areas in rooms, gaps, or partitioned areas where the proportion of areas containing structures (e.g., desks, chairs, shelves, etc.) is higher than a reference value, passages (e.g., hallways) accessing the partitioned areas, or passages that are narrower and longer than a reference value as specific locations. The detection of specific locations is not limited to image data, and may be performed using various data acquired from sensors, etc. Alternatively, specific locations may be detected from known information about the target space S1. This allows the information processing device 10 to detect specific locations in advance.

[0049] Returning to Fig. 4, in step S105, the information processing device 10 corrects the route in accordance with the specific location. The information processing device 10 corrects the route so that the specific location detected in the map data 1000 is measured in more detail than other locations. The corrected route will now be described.

[0050] FIG. 7 is a diagram illustrating a modified route. In this example, map M2 is an example of an environmental map showing a modified route made by the information processing device 10 to the route initially determined. Route R2 is a new modified route for measuring radio wave intensity. In this example, route R2 has been modified to prioritize and measure the areas surrounding specific locations L1 and L2 in detail. Measuring in detail means increasing the distance traveled at the specific locations compared to the initially determined route or increasing the measurement frequency to increase the sampling density. As an example, to measure the specific location in detail, the information processing device 10 changes the path of the robot 100 to a circular path (the path at specific location L1 in the figure) or a meandering path (the path at specific location L2 in the figure). This allows the information processing device 10 to increase the distance traveled within the specific location L and increase the sampling density. Alternatively, when increasing the measurement frequency to increase the sampling density, the information processing device 10 may record the number of measurements per distance traveled or per hour at the specific location L in a database in advance.

[0051] Returning to FIG. 4 , in step S106, the information processing device 10 records the corrected route R2 in the database. The information processing device 10 stores the map M2 in the database and can use it when actually measuring radio wave intensity. Note that the information processing device 10 may add, modify, or delete the map M, the route R, or the specific location L as appropriate.

[0052] As a result, the information processing system 1 can acquire an environmental map of the target space S1 and a route that allows detailed measurement of a specific location. The information processing system 1 can correct a route to a specific location in the target space where the radio wave environment is thought to be deteriorating. Next, a description will be given of route correction during actual radio wave intensity measurement by the information processing device 10.

[0053] 2-2. Radio wave intensity measurement method and dynamic route correction method Figure 8 is a flowchart illustrating a radio wave intensity measurement method and dynamic route correction method in the information processing system 1. In this example, measurement of radio wave intensity in the target space S1 is initiated, for example, at a predetermined time (or at a predetermined cycle, a predetermined frequency, etc.). The following flow represents processing executed mainly by the information processing device 10.

[0054] In step S1, the information processing device 10 measures radio wave intensity while moving the robot 100 along a route. In this example, the information processing device 10 reads out a map M2 from a database and moves the robot 100 along a route R2. At this time, the information processing device 10 works in cooperation with the external device 20 to estimate its own position using SLAM and identify its own position on the map M2. The radio wave intensity measurement is performed by the external device 20 through measurement and data collection. The external device 20 measures the radio wave intensity of the access point 800 at each measurement point along the route R2. Here, data related to the radio wave intensity measurement will be described.

[0055] FIG. 9 is a diagram illustrating the measurement database 2000. In this example, the measurement database 2000 includes multiple records. Each record corresponds to a record of each radio wave intensity measurement. Each record includes a measurement ID, a timestamp, an AP (access point) ID, a radio wave intensity value, image data, and remarks. The measurement ID is identification information unique to each measurement by the external device 20 or each location of the robot 100 (e.g., measurement location X) in the target space S1. The timestamp is time information indicating the time the measurement was performed. The AP (access point) ID is identification information unique to each access point 800 that is the source of the radio waves measured in each measurement. The AP ID includes, for example, an SSID (Service Set Identifier), particularly a BSSID (Basic Service Set Identifier). The external device 20 uniquely identifies radio waves emitted from the access point 800 installed in the target space S1. For example, the measurement target may be identified from information about an access point previously connected to the information processing device 10 via the network 9. The radio wave intensity value is the RSSI value (in dBm units) for each access point 800. The image data is an image captured simultaneously with the measurement. In this example, the image data is used, for example, to estimate the degree of congestion. The external device 20 basically simultaneously measures radio wave intensity and captures images at the same location, and outputs corresponding data groups to the information processing device 10. The degree of congestion will be described later. The notes are records that record various data. For example, the information processing device 10 may acquire from the external device 20 a travel distance (an example of travel information) indicating the numerical value of the distance accumulated since the robot 100 started moving. The measurement database 2000 may record information related to the specifications of the access point 800 (for example, power information). Alternatively, the information processing device 10 may record the degree of congestion for each measurement point.

[0056] Returning to FIG. 8 , in step S2, the information processing device 10 detects crowded areas. This process is performed while the robot 100 is moving and while the external device 20 is measuring. The external device 20 takes images of the area around the robot 100 (or records video) at the same time as the measurement. The information processing device 10 acquires image data (an example of real-time video) from the external device 20 during measurement. The information processing device 10 detects crowded areas based on the image data and estimates the crowd. A specific example is given below.

[0057] Methods for identifying crowds from images include, for example, semantic segmentation technology or person detection using histograms of oriented gradients (HOG) features. The information processing device 10 combines these technologies to estimate crowds. In this example, the information processing device 10 uses a machine learning model that can identify different objects in an image, such as cars, buildings, and people, based on semantic segmentation technology. This machine learning model learns to identify objects in advance from annotation data and calculates population density based on the area of ​​target objects classified as people and the area of ​​the image background. The population density here is an example of a crowd density quantification. The training data used to train the machine learning model includes image data (or image data that has been labeled, etc.) containing a large number of people collected from urban surveillance cameras or footage of event venues. The machine learning model can learn a reference crowd density or population density threshold, i.e., a criterion for determining whether an image is crowded or not, from images and population density data taken in locations where crowding is likely to occur. In this example, the machine learning model includes deep learning models such as U-Net, DeepLab (DeepLabV3+), and SegNet. The information processing device 10 inputs real-time video acquired from the external device 20 into the machine learning model and outputs the congestion level estimated at each location around the robot 100. In this way, the information processing device 10 can detect specific locations and estimate the congestion level.

[0058] In step S3, the information processing device 10 dynamically corrects the route in accordance with the real-time video. In this example, the information processing device 10 changes the route toward the detected congested area. This may involve, for example, moving the robot 100 in a direction different from the original route. Here, real-time route correction by the information processing device 10 will be described.

[0059] FIG. 10 is a diagram illustrating a real-time video. In this example, image P1 is an example of image data. Image P1 is image data corresponding to one frame of real-time video acquired by the information processing device 10. Image P1 represents an image of the target space S1 captured from the front of the robot 100 (i.e., the direction of movement). Arrows A (e.g., arrows A1 and A2) are displayed on the screen to indicate the direction of travel of the robot 100. Arrow A1 indicates the direction of the original route, and arrow A2 indicates the direction of movement to the corrected route. Object O is an object representing a person depicted in image P1, and includes, for example, a rectangular frame corresponding to the outline or outer edge of the person depicted on the screen. In this example, the information processing device 10 detects a crowd in which person objects O are densely packed in image P1, and if the degree of congestion is equal to or greater than a threshold, corrects the route from the direction of arrow A1, which is the original route, to the direction of arrow A2. Note that arrow A is illustrated for convenience of explanation and does not need to be displayed in the image P1 processed by the information processing device 10. As a result, the information processing device 10 can dynamically correct the route in accordance with the real-time video.

[0060] Returning to FIG. 8, in step S4, the information processing device 10 performs detailed measurements of the congested area. In this example, before performing the detailed measurements, the information processing device 10 generates and outputs a modified route to increase the sample density in the identified congested area. For example, circular movement or meandering movement as described in section 2-1 is effective for this. Here, the modified route will be described.

[0061] FIG. 11 is a diagram illustrating a corrected route. In this example, map M3 is an example of an environmental map similar to map M2. Route R2 is a route used (or currently being used) for measuring radio wave intensity. In the figure, route R2 is represented by a dotted line. Specific location L3 is a specific location detected based on real-time video. In the figure, specific location L3 is depicted with hatching. In this example, route R3 is a route corrected to measure the area of ​​specific location L3 in detail. In the figure, route R3 is represented by a solid line. Within specific location L3, the route is corrected so that the robot 100 moves in a meandering pattern to increase the measurement density. The information processing device 10 moves the robot 100 along route R3 and performs measurements. Here, route correction during measurement is assumed. Therefore, when detailed measurement of specific location L3 is completed, the information processing device 10 merges (or combines) the corrected route R3 with the original route R2. Note that congestion due to pedestrian congestion or traffic is expected within specific location L3. Therefore, the information processing device 10 controls the external device 20 to slow down the movement speed of the robot 100, to make the robot wait until people have left, and in some cases to notify people around the robot of its movement (for example, by using light or sound). As described above, the information processing device 10 can perform detailed measurements of crowded places and collect sample data at a high density.

[0062] Returning to Fig. 8, in step S5, the information processing device 10 determines whether a condition for ending measurement in the target space S1 has been met. The condition for ending measurement is, for example, whether measurement of the entire area of ​​the target space S1 has been completed or whether the robot 100 has returned to the measurement start point. If the condition for ending measurement is not met (step S5: NO), the information processing device 10 proceeds to step S6.

[0063] In step S6, the information processing device 10 causes the robot 100 to merge with the original route. If the measurement termination condition is not met, this means that the measurement is still ongoing. Therefore, after completing detailed measurements of the congested area, the information processing device 10 moves the robot 100 to the original route. Referring again to FIG. 11 , the robot 100, which has merged with the original route R2 from the corrected route R3, resumes normal measurement along the route R2. In this example, the information processing device 10 returns the process to step S1. The information processing device 10 repeats the processes of steps S1 to S6 until the termination condition is met.

[0064] On the other hand, if the termination condition is satisfied (step S5: YES), that is, if the measurement of the external device 20 in the target space S1 is completed, the information processing device 10 proceeds to step S7.

[0065] In step S7, the information processing device 10 records the measurement data in a database. The information processing device 10 can use the measured values ​​of radio wave intensity corresponding to the environmental map, for example, to create a radio wave intensity map. In this example, the radio wave intensity map is map data in which position information on the environmental map corresponds to radio wave information. For example, a specific example of a radio wave map is a map in which shading corresponding to the strength of radio wave intensity is displayed on the environmental map, a so-called heat map.

[0066] Here, the effect of measuring specific locations more in detail than other locations is expected to improve workability in the actual environment and the accuracy of measurement samples. For example, route correction increases the density of measurement points within a specific location. Generally, performing measurements more frequently tends to improve sample accuracy in the surrounding area. However, the actual environment (e.g., congestion level) in the target space S1 is constantly changing, making it necessary to measure locations (e.g., specific locations) different from the expected route. Therefore, if measurements can be focused on the area around a specific location depending on the on-site situation, even if a location where the radio wave environment is worse than the standard is discovered, that area can be covered. As a result, the measurement density at a specific location increases, and efficient and highly accurate collection of radio wave information can be expected.

[0067] The information processing device 10 also responds to fluctuations in specific locations in the target space S1 by periodically performing the processes of steps S1 to S7 (e.g., at a fixed time each day, at different times, or at random times). In this example, the information processing device 10 collects measurement data at different time periods (or time slices) and performs steady-state analysis. For example, by performing measurements multiple times across different dates, the information processing device 10 can determine whether the detected crowd is a periodic trend due to time factors or a random phenomenon. Furthermore, the information processing device 10 may periodically perform measurements at different time slices without making any changes to the route R2 in order to improve the detection accuracy of specific locations.

[0068] The information processing device 10 may use data on the congestion level from past measurements recorded in a database during the above-mentioned periodic measurements. In this example, the information processing device 10 may detect areas with high congestion levels or areas with large changes in congestion levels from past measurements as specific locations, and reconfigure the route so that measurements are performed preferentially (or in detail). This allows the information processing device 10 to perform thorough measurements of specific locations.

[0069] As a result, the information processing device 10 can dynamically correct the route according to the detected specific location, and can increase the density of measurement samples in areas where the radio wave environment deteriorates due to crowds.

[0070] 3. Modifications The present invention is not limited to the above-described embodiment, and various modifications are possible. Some modifications will be described below. Two or more of the following features may be combined and applied.

[0071] (1) Information Processing System 1 The hardware configuration and network configuration of the information processing system 1 are not limited to those exemplified in the embodiment. The information processing system 1 may have any hardware configuration and network configuration as long as the required functions can be realized. For example, multiple physical devices may cooperate to function as the information processing system 1. For example, the information processing system 1 may have a management terminal that manages the operation of the robot 100. The management terminal manages the operation of the robot 100 and, in some cases, may intervene in the control.

[0072] (2) Robot 100 The robot 100 is not limited to the example shown in the embodiment. The robot 100 may have any hardware as long as it can realize the required functions and operations. The robot 100 may be a machine that is operated by any power source. The robot 100 may have a power supply source, such as an external power source or an internal battery, for receiving power. The robot 100 basically comprises an information processing device 10 and an external device 20, and the following modifications are possible.

[0073] (3) Information Processing Device 10 The correspondence between the functional elements of the information processing device 10 and the hardware is not limited to that exemplified in the embodiment. For example, in the embodiment, at least some of the functions described as being implemented in the information processing device 10 may be implemented in another device or system, and conversely, at least some of the functions described as being implemented in another device or system may be implemented in the information processing device 10. For example, the processing performed by the information processing device 10 may be performed by the external device 20. The external device 20 may perform SLAM processing using image data to estimate the position of the robot 100.

[0074] (4) External Device 20 The external device 20 is not limited to the example illustrated in the embodiment. The external device 20 may have any functional elements as long as it can realize the required functions and operations. The external device 20 may have any measurement means. For example, in addition to the access point 800, the external device 20 may measure radio wave intensity according to various wireless communication standards other than Wi-Fi, such as Bluetooth, Zigbee, or Matter, in a smart device (e.g., smart home appliance or smart speaker). The external device 20 may have any imaging means. The external device 20 may be, for example, an analog camera, a stereo camera, a depth camera, a thermography camera, a motion camera, a network (or IP) camera, a cloud camera, or an infrared camera.

[0075] (5) Access Point 800 The access point 800 is not limited to the one exemplified in the embodiment. The access point 800 may have any hardware configuration as long as it can realize the required functions and operations. The access point 800 may be a wireless communication base station (or simply referred to as a base station). The robot 100 may measure the radio wave intensity of radio waves within a base station cell. The access point 800 may be a small cell, microcell, picocell (nanocell), femtocell, or the like installed indoors.

[0076] (6) Environmental Map Creation Method and Route Correction Method According to Terrain: The sequence chart shown in FIG. 4 merely illustrates an example of the operation, and the operation method of the information processing system 1 is not limited to this. Some of the illustrated operations may be changed or omitted, the order may be changed, or new operations may be added. The initial route of the robot 100 used to create the environmental map in step S101 may be specified by a user (e.g., a facility manager, a system user, etc.). In this example, the initial route may be automatically generated by various servers on the network or by the robot 100 itself based on a simple map. Note that the environmental map used for the initial route or radio wave intensity measurement may be created by a device separate from the robot 100 (e.g., a dedicated server not shown) and recorded in a database in advance. Furthermore, the external device 20 may acquire any data in step S101. In addition to image data, the external device 20 may acquire point cloud data obtained by LiDAR or distance data obtained by a depth camera. In this example, the information processing device 10 may create the environmental map based on any data.

[0077] In steps S104 and S105, the information processing device 10 may determine and correct the route in any way. The information processing device 10 may set an initial route in the target space S1 so that the measurement density within the area is uniform. The information processing device 10 may set a corrected route in the environmental map so that the measurement density in an area including a specific location is increased compared to other locations. Note that the map data 1000 used for route setting may be of any type. For example, the map data 1000 may be created by the information processing device 10 based on various data, or may be obtained from a web service, an application, the cloud, or the like.

[0078] (7) Radio Wave Intensity Measurement Method and Dynamic Route Correction Method The flowchart shown in FIG. 8 merely illustrates an example of the operation, and the operation method of the information processing system 1 is not limited to this. Some of the illustrated operations may be changed or omitted, the order may be changed, or new operations may be added. In step S2, the information processing device 10 may estimate the crowd level in any manner. The information processing device 10 may estimate the congestion level based on any data. For example, the information processing device 10 may perform processing based on audio (or noise) data acquired from a microphone. In step S3, the information processing device 10 may correct the route in any manner, or may not correct the route at all. If the information processing device 10 detects a specific location in a direction different from the traveling direction (e.g., the opposite direction), the information processing device 10 may not correct the route or may end the measurement earlier than planned depending on the travel distance or power of the robot 100. In step S4, the information processing device 10 may increase the number of measurements per travel distance. In step S5, any termination condition may be set. In the embodiment, the termination condition is a condition that targets the end of measurement along the original route, but it may also be a condition that is triggered by the end of measurement at a specific location that has been detected in advance (i.e., a location where radio waves are likely to deteriorate due to the terrain).

[0079] (8) Database The databases or data of the information processing system 1 shown in FIGS. 5 to 7 and 9 to 11 are not limited to those exemplified in the embodiments. In this example, any data may be recorded in the database. For example, the map data 1000 may include coordinate information and time information of measurement points. Alternatively, the map data 1000 may include various metadata related to measurements and routes (e.g., the number of measurements, measurement points, movement method, etc.). The measurement database 2000 may record movement information of the robot 100.

[0080] (9) The various other programs executed by the processor 101 may be provided by downloading via a network such as the Internet, or may be provided in a state recorded on a computer-readable non-transitory recording medium such as a DVD-ROM. Each processor may be, for example, a CPU, an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit).

[0081] The block diagrams used to explain the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of hardware and / or software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are directly or indirectly connected (e.g., wired, wireless, etc.) and these multiple devices. The functional block may also be realized by combining software with the single device or multiple devices.

[0082] Functions include, but are not limited to, judgment, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, consideration, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assignment. For example, a functional block (component) that performs transmission is called a transmitting unit or transmitter. As mentioned above, there are no particular limitations on how these functions are implemented.

[0083] For example, the information processing device 10 according to an embodiment of the present disclosure may function as a computer that performs the processing of the present disclosure.

[0084] Each aspect or embodiment described in the present disclosure may be applied to at least one of systems using LTE (Long Term Evolution), LTE-Advanced (LTE-A), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (New Radio), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, UWB (Ultra-Wide Band), Bluetooth (registered trademark), or other suitable systems, and next-generation systems enhanced based on these. Furthermore, a combination of multiple systems (e.g., a combination of at least one of LTE and LTE-A with 5G) may also be applied.

[0085] The order of the procedures, sequences, flowcharts, etc. of each aspect or embodiment described in this disclosure may be changed unless it is inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0086] Input and output information may be stored in a specific location (for example, memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be sent to another device.

[0087] The determination may be made based on a value represented by one bit (0 or 1), a Boolean value (true or false), or a numerical comparison (e.g., comparison with a predetermined value).

[0088] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure.

[0089] Software, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise, should be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc. Additionally, software, instructions, information, etc. may be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then such wired and / or wireless technologies are included within the definition of a transmission medium.

[0090] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof. Note that terms described in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings.

[0091] Furthermore, the information, parameters, etc. described in this disclosure may be expressed using absolute values, may be expressed using relative values ​​from a predetermined value, or may be expressed using other corresponding information.

[0092] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0093] As used in this disclosure, any reference to an element using a designation such as "first," "second," etc. does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must in some way precede the second element.

[0094] The "unit" in the configuration of each of the above devices may be replaced with "means," "circuit," "device," or the like.

[0095] When the terms "include," "including," and variations thereof are used in this disclosure, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, when the term "or" is used in this disclosure, it is not intended to be an exclusive or.

[0096] In this disclosure, where articles are added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0097] In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "coupled" may also be interpreted in the same way as "different."

[0098] 1...information processing system, 100...robot, 10...information processing device, 20...external device, 800...access point, 9...network, 11...route determination unit, 12...output unit, 13...correction unit, 14...movement unit, 15...measurement unit, 16...acquisition unit, 17...estimation unit, 18...position estimation unit, 190...recording unit, 191...storage unit, 192...control unit, 101...processor, 102...memory, 103...storage, 104...communication device, 1000...map data, 2000...measurement database

Claims

1. An information processing device having: a route determination unit that determines a route for measuring radio wave strength using an environmental map showing a target space; an output unit that outputs a modified route that modifies the route so that specific locations in the target space where the radio wave environment is predicted to be worse than a standard are measured in more detail than other locations; a movement unit that autonomously moves the device along the modified route; and a measurement unit that measures radio wave strength.

2. The information processing device according to claim 1, wherein the specific location is a crowded location.

3. The information processing device according to claim 2, further comprising: an acquisition unit that acquires real-time video of the target space; and an estimation unit that estimates the crowd based on the real-time video.

4. The information processing device according to claim 3, wherein the output unit outputs the modified path that is dynamically modified in accordance with the real-time video.

5. An information processing device as described in claim 4, further comprising a position estimation unit that estimates the position of the device using data obtained from a sensor, and the movement unit autonomously moves the device along the dynamically corrected modified route.

6. The information processing device according to claim 2, further comprising a recording unit that records information indicating the state of the crowd in addition to the results of the radio wave intensity measurement.

7. The information processing device according to claim 1, wherein the specific location is a location having a specific shape on the environmental map.

8. An information processing method comprising the steps of: determining a route for measuring radio wave strength using an environmental map showing a target space; outputting a modified route by modifying the route so as to measure specific locations in the target space where the radio wave environment is predicted to be worse than a standard in more detail than other locations; autonomously moving the device along the modified route; and measuring radio wave strength.

Images