Information processing equipment and programs

Abstract

Improve the accuracy of estimating the position of moving objects on the map. [Solution] According to the embodiment, an information processing device is provided. The information processing device comprises an acquisition unit, a map generation unit, and an environment update unit. The acquisition unit acquires distance sensor information, the amount of movement of the multiple radios, and received power information from each of the multiple radios, and acquires image information from an image sensor. The map generation unit generates a map showing the environment in which the multiple radios move, based on the distance sensor information and the amount of movement of the multiple radios acquired over a predetermined period. The environment update unit updates the map when it detects that the environment shown on the map has changed, based on at least one of the distance sensor information, received power information, and image information acquired after the predetermined period.

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus and a program.

Background Art

[0002] In recent years, for example, a moving body (e.g., a transport robot, etc.) that autonomously moves in a factory or the like is known. The moving body moves along a predetermined path while estimating the position of the moving body on the map based on a map showing the environment of the space in which the moving body moves and sensor information obtained from a distance sensor provided in the moving body. In the following description, the "environment" is assumed to be, for example, the positions (arrangements) of walls, pillars, doors, and obstacles.

[0003] However, for example, due to a layout change in a factory or the like, there may be a case where the environment shown on the map is different from the environment when the moving body is actually moving. In this case, the moving body may not be able to estimate the position of the moving body on the map (self-position), and the moving body may not operate correctly.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Therefore, the problem to be solved by the present invention is to provide an information processing apparatus and a program capable of improving the estimation accuracy of the position of a moving body on a map.

Means for Solving the Problems

[0006] According to the embodiment, an information processing device is provided. The information processing device comprises an acquisition unit, a map generation unit, and an environment update unit. The acquisition unit acquires distance sensor information, the amount of movement of the multiple wireless devices, and received power information from each of the multiple wireless devices, and acquires image information from an image sensor. The map generation unit generates a map showing the environment in which the multiple wireless devices move, based on the distance sensor information and the amount of movement of the multiple wireless devices acquired over a predetermined period. The environment update unit updates the map when it detects that the environment shown on the map has changed, based on at least one of the distance sensor information, the received power information, and the image information acquired after the predetermined period. [Brief explanation of the drawing]

[0007] [Figure 1] A diagram showing an example configuration of an autonomously operating mobile body according to a comparative example of the first embodiment. [Figure 2] A diagram showing an example of the basic configuration of a server according to the first embodiment. [Figure 3] A diagram showing the position of the moving object and the position of the moving object on the map estimated by the server in a comparative example of the first embodiment. [Figure 4] A diagram illustrating the principle of SLAM used in the server according to the first embodiment. [Figure 5] A diagram showing an example of the functional configuration of a server according to the first embodiment. [Figure 6] A diagram showing an example of the server hardware configuration according to the first embodiment. [Figure 7] A diagram showing an example of a target space managed by the server according to the first embodiment. [Figure 8] A diagram showing the time variation of the range over which environmental information can be acquired by a mobile device and image sensor connected to a server according to the first embodiment. [Figure 9] A diagram showing an example of a target space managed by the server according to the first embodiment. [Figure 10] A flowchart showing an example of the map update process by the server in the first embodiment. [Figure 11] A diagram showing the functional configuration of the server according to the second embodiment. [Figure 12] A diagram illustrating the interference signal used by the server according to the second embodiment. [Figure 13] A diagram illustrating an overview of a method for detecting changes in the surrounding environment according to the second embodiment. [Figure 14] A diagram illustrating an overview of a method for detecting changes in the surrounding environment according to the second embodiment. [Figure 15] A flowchart showing an example of the process flow for updating the server map according to the second embodiment. [Figure 16] A diagram illustrating the received power map used by the server according to the second embodiment. [Figure 17] A diagram illustrating the received power map used by the server according to the second embodiment. [Figure 18] A diagram showing a specific example of applying the server 20 according to the second embodiment. [Figure 19] Figure 18 shows a specific example of applying the server 20 according to the second embodiment. [Figure 20] Figure 19 shows a specific example of applying the server 20 according to the second embodiment. [Figure 21] Figure 20 shows a specific example of applying the server 20 according to the second embodiment. [Modes for carrying out the invention]

[0008] The embodiments will be described below with reference to the drawings. It should be noted that the disclosure is merely an example, and the invention is not limited by the contents described in the embodiments below. Modifications that a person skilled in the art can easily apply are naturally included within the scope of the disclosure. In order to make the explanation clearer, the size, shape, etc. of each part may be schematically represented in the drawings with modifications from the actual embodiments. In some cases, the same reference numerals may be used for corresponding elements in multiple drawings, and detailed explanations may be omitted.

[0009] (First Embodiment) First, the first embodiment will be described. The information processing apparatus (hereinafter referred to as a server) according to this embodiment is used to control a moving body that moves within a predetermined space (hereinafter referred to as a target space), such as inside a factory or a warehouse. Examples of the moving body include, for example, a transport robot, a cleaning robot, a security robot, and a guiding robot.

[0010] The moving body can be roughly classified into those that operate autonomously and those that operate based on an external command (control signal). The server according to the first embodiment is used to control the latter type of moving body.

[0011] Here, as a comparative example, a moving body that operates autonomously will be described. FIG. 1 shows a configuration example of a moving body 10 that operates autonomously.

[0012] The moving body 10 includes, for example, a distance sensor 11, a motor 12, and a control unit 13. The distance sensor 11 acquires the distance to an object (e.g., a wall, a pillar, and an obstacle) existing around the moving body 10. The distance sensor 11 is, for example, an LRF (Laser Range Finder) or a Lider, and measures the time (TOF: Time Of Flight) from when a laser (light) is irradiated until the laser is reflected back by an object, thereby measuring the distance to an object existing around the moving body 10. Hereinafter, the distance to an object existing around the moving body 10 acquired by the distance sensor 11 is referred to as "distance sensor information".

[0013] The motor 12 rotationally drives the wheels of the moving body 10 under the control of the control unit 13. The control unit 13 controls the motor 12 to move the moving body 10 to, for example, a predetermined destination.

[0014] Here, the control unit 13 includes a map generation unit 14, a position estimation unit 15, and a movement control unit 16. The control unit 13 calculates the amount of movement (odometry) of the moving body 10 based on the amount of wheel rotation and the direction (rotation angle) of the wheels.

[0015] The map generation unit 14 generates a map showing the environment of the target space based on distance sensor information and the amount of movement of the moving object 10. Such a map is also called an environment map and shows the positions of walls forming the target space, passages within the target space, and objects installed within the target space.

[0016] The position estimation unit 15 estimates the position (self-position) of the moving object 10 on the map based on the map generated by the map generation unit 14 and the distance sensor information.

[0017] The movement control unit 16 determines the direction and amount of movement of the moving object 10 based on its position on the map, and controls the movement of the moving object 10.

[0018] The autonomously operating mobile unit 10 requires high processing power to generate a map and estimate its position on the map based on distance sensor information. Therefore, the cost per unit is high, making it difficult to operate multiple mobile units 10.

[0019] In contrast, the moving object controlled by the server according to the first embodiment operates in response to control signals from the server. In this case, the processing corresponding to the map generation unit 14, the position estimation unit 15, and the movement control unit 16 is performed by the server.

[0020] Figure 2 shows an example of the basic configuration of the server 20 according to the first embodiment. Mobile units 10a to 10d are wirelessly connected to the server 20 via a base station 30 installed in the target space. Hereinafter, signals transmitted from mobile units 10a to 10d to the server 20 will be referred to as "uplink signals." Uplink signals are also called uplinks. Signals transmitted from the server 20 to mobile units 10a to 10d will be referred to as "downlink signals." Downlink signals are also called downlinks. Examples of downlink signals include control signals for controlling mobile units 10a to 10d, and synchronization signals with the base station 30 (server 20).

[0021] The mobile unit 10a includes a distance sensor 11, a motor 12, a control unit 13, and a wireless device 17. The control unit 13 controls the motor 12 based on control signals transmitted from the server 20. The control unit 13 also calculates the amount of movement based on the amount of wheel rotation and the direction of the wheels.

[0022] The radio 17 transmits distance sensor information and movement amount to the server 20 via the base station 30, and receives control signals from the server 20 via the base station 30. Although not shown in the figures, mobile units 10b to 10d also have distance sensors, motors, control units, and radios, respectively, similar to mobile unit 10a.

[0023] The server 20 includes, for example, a map generation unit 20a, a position estimation unit 20b, a process management unit 20c, a flight management unit 20d, an AMR (Autonomous Mobile Robot) control unit 20e, a route generation unit 20f, and a movement control unit 20g.

[0024] The map generation unit 20a generates a map based on distance sensor information and movement amount transmitted by the moving objects 10a to 10d. The position estimation unit 20b estimates the positions of the moving objects 10a to 10b on the map generated by the map generation unit 20a.

[0025] The process control unit 20c manages the processes (procedures) of work and other tasks performed by the mobile units 10a to 10d.

[0026] The operations management unit 20d manages the sequence of movement of each of the mobile units 10a to 10d based on the processes of work and other tasks performed by the mobile units 10a to 10d.

[0027] The path generation unit 20f generates a path for the mobile bodies 10a to 10d based on the order of operations performed by the mobile bodies 10a to 10d and the positions of the mobile bodies 10a to 10d.

[0028] The AMR control unit 20e generates control signals for performing operations at the positions of the moving objects 10a to 10d estimated by the position estimation unit 20b.

[0029] The movement control unit 20g generates control signals to control the movement of the moving bodies 10a to 10d based on the paths generated by the path generation unit 20f.

[0030] The generated control signals are transmitted to each mobile unit 10a to 10d via the base station 30.

[0031] In this way, by having the server 20 consolidate the processes of map generation and estimation of the positions of each mobile object 10a to 10d on the map, the cost for the mobile objects 10a to 10d can be reduced. Hereafter, the system in which the server 20 consolidates information on the mobile objects 10a to 10d will be referred to as "aggregation control".

[0032] Furthermore, when moving a mobile object in a straight line along a limited area of ​​a passageway, simple control by the autonomously moving mobile object is sufficient. However, when it is necessary to perform more complex movements such as turning curves or avoiding obstacles, more advanced control of the mobile object is required. In particular, when controlling multiple mobile objects, the position of other mobile objects must be taken into consideration, thus increasing the need for the control described above. With centralized control, the path and position information of mobile objects 10a to 10d can be grasped collectively, thus optimizing the operation of mobile objects 10a to 10d.

[0033] Furthermore, when receiving control signals for such advanced control via wired communication, several problems arise: control is limited to the range of the wire (cord), control becomes ineffective due to wire breakage, and wires can become entangled. In contrast, as shown in Figure 2, these problems are solved when control signals are received via wireless communication. This wireless communication can be implemented, for example, by Wi-Fi or Local 5G.

[0034] Next, we will briefly explain the process for estimating the positions of moving objects 10a to 10d on the map. Figure 3 shows the positions of the moving objects 10a to 10d and the positions of the moving objects 10a to 10d on the map estimated by the server 20. The upper part of Figure 3 shows the objects moving along a predetermined path.

[0035] The mobile units 10a to 10d move within the space and acquire distance sensor information (the distance between the mobile units 10a to 10d and surrounding walls and obstacles, etc.). They then transmit the acquired distance sensor information and the amount of movement to the base station 30.

[0036] The server 20 estimates the map and the positions of the mobile objects 10a to 10d on the map based on the distance sensor information of the mobile objects 10a to 10d received via the base station 30.

[0037] However, even though mobile object 10a is actually at location pa1 on the map, its location may be estimated as pa2. Similarly, the locations pb2, pc2, and pd2 of mobile objects on the map estimated by server 20 may differ from the actual locations pb1, pc1, and pd1 of mobile objects 10b, 10c, and 10d.

[0038] This is because, within the target space, changes in layout, movement of people near the moving objects 10a-10d, and changes in cargo loads result in differences between the environment when the map was generated and the environment when the moving objects 10a-10d are actually moving.

[0039] Here, referring to Figure 4, we will briefly explain the principle of SLAM (Simultaneous Localization and Mapping). SLAM is a commonly used technique for simultaneously estimating the positions of a map and moving objects 10a to 10d on the map.

[0040] In the following explanation, the coordinate system based on the position of the moving object 10a (distance sensor 11) will be referred to as the "sensor coordinate system," and the coordinate system based on a predetermined position on the map will be referred to as the "map coordinate system."

[0041] The mobile body 10a moves in the order of positions x1, x2, and x3 shown in Figure 4. At position x1, the mobile body 10a senses its surroundings using the distance sensor 11 and detects observation points (landmarks) L1 and L2. Subsequently, the mobile body 10a moves in the direction of a1 and detects observation points L2 and L3 at position x2. Then, the mobile body 10a moves in the direction of a2 and detects observation points L4 and L3 at position x3.

[0042] The mobile body 10a transmits distance sensor information, including the distance between the mobile body 10a and observation points L1, L2, L3, and L4, and the relative positions of observation points L1, L2, L3, and L4 as seen from the mobile body 10a (i.e., the position vectors of observation points L1, L2, L3, and L4 in the sensor coordinate system), along with the amount of movement, to the server 20.

[0043] The server 20 estimates the positions of observation points L1, L2, L3, and L4 on the map (i.e., the position vectors of observation points L1, L2, L3, and L4 in the map coordinate system) and the position of the moving object 10a on the map (i.e., the position vector of the moving object 10a in the map coordinate system) based on distance sensor information and movement amount.

[0044] Here, in order to reflect observation points L1, L2, L3, and L4 on the map, it is necessary to convert the position vectors of observation points L1, L2, L3, and L4 in the sensor coordinate system to the position vectors of observation points L1, L2, L3, and L4 in the map coordinate system. The position vectors of observation points L1, L2, L3, and L4 in the map coordinate system can be obtained by estimating the position vector of the moving object 10a in the map coordinate system.

[0045] Specifically, server 20 estimates the position vectors x1, x2, and x3 of the moving object 10a in the map coordinate system by solving the following simultaneous equations (1) to (8). Note that position vector x1 is assumed to be known as the initial position.

[0046] Furthermore, server 20 solves the following simultaneous equations (1) to (8) to obtain the position vector q1 of observation point L1, the position vector q2 of observation point L2, the position vector q3 of observation point L3, and the position vector q4 of observation point L4 in the map coordinate system.

[0047]

number

[0048] Here, a1 is a vector containing information about the amount of movement in the sensor coordinate system when moving from position x1 to position x2 (specifically, information about the distance and direction of movement of the moving object 10a). a2 is a vector containing information about the amount of movement in the sensor coordinate system when moving from position x2 to position x3. Note that the position vectors x1, x2, and x3 obtained by the system of equations (1) to (8) include information about the direction of movement θ.

[0049] Furthermore, R1, R2, and R3 are transformation matrices from the sensor coordinate system to the map coordinate system. Z1 is the position vector of observation point L1 in the sensor coordinate system acquired at position x1. Z2 is the position vector of observation point L2 in the sensor coordinate system acquired at position x1. Z3 is the position vector of observation point L2 in the sensor coordinate system acquired at position x2. Z4 is the position vector of observation point L3 in the sensor coordinate system acquired at position x2. Z5 is the position vector of observation point L3 in the sensor coordinate system acquired at position x3. Z6 is the position vector of observation point L4 in the sensor coordinate system acquired at position x3.

[0050] t1 is the position vector x1 in the map coordinate system, excluding the direction of travel θ. Similarly, t2 and t3 are the position vectors x2 and x3 in the map coordinate system, respectively, excluding the direction of travel θ.

[0051] Server 20 generates and updates the map based on the position of the moving object 10a on the map (the position vector of the moving object 10a in the map coordinate system) and the positions of observation points L1, L2, L3, and L4 (the position vectors of observation points L1, L2, L3, and L4 in the map coordinate system), which are obtained by the simultaneous equations (1) to (8).

[0052] In this way, SLAM determines the position of the mobile object 10a on the map by detecting multiple observation points as the mobile object 10a moves. Generally, it is known that the more observation points detected by the mobile object 10a, the more simultaneous equations are needed, and the more accurately the position of the mobile object 10a on the map can be estimated.

[0053] However, if, for example, an unknown obstacle 40 is newly placed between the mobile object 10a and observation point L2 due to a change in layout, the mobile object 10a will not be able to detect observation point L2. As a result, the number of simultaneous equations used by the server 20 to estimate the position of the mobile object 10a on the map decreases, and the accuracy of estimating the position of the mobile object 10a on the map decreases. Furthermore, there is a possibility that the position of obstacle 40 may be calculated as the position of the known observation point L2. Thus, a change in layout can sometimes prevent the correct estimation of the positions of the mobile object 10a and observation point L2 on the map.

[0054] Therefore, one of the objectives of this embodiment is to improve the accuracy of estimating the positions of moving objects 10a to 10d by detecting changes in the environment and updating the map to reflect those changes.

[0055] Figure 5 shows an example of the functional configuration of the server 20 according to the first embodiment. The server 20 according to the first embodiment is used to control mobile bodies 10a to 10d that move within the target space, as shown in the basic configuration example described above.

[0056] Here, the server 20 is connected to the image sensor 31. The image sensor 31 is, for example, a camera. The image sensor 31 is installed, for example, on the ceiling, in a predetermined space and takes pictures of the predetermined space from an overhead perspective.

[0057] The server 20 comprises an acquisition unit 21, a storage unit 22, a map generation unit 23, an environment update unit 24, a position estimation unit 25, and a mobile unit control unit 26. The server 20 may also have other configurations related to the control of mobile units 10a to 10d, such as a process management unit 20c, an operation management unit 20d, an AMR control unit 20e, and a route generation unit 20f, as shown in Figure 2. Furthermore, while this description focuses on controlling four mobile units 10a to 10d, the server 20 may control one mobile unit or two or more mobile units.

[0058] The acquisition unit 21 acquires distance sensor information and movement amount from the moving body 10a at predetermined intervals. Although only the moving body 10a is shown in Figure 5, the acquisition unit 21 also acquires (aggregates) distance sensor information and movement amount from the moving bodies 10b to 10d at predetermined intervals.

[0059] Furthermore, the acquisition unit 21 acquires image information from one or more image sensors 31 at predetermined intervals. Although Figure 5 shows one image sensor 31, the acquisition unit 21 may acquire (aggregate) image information from multiple image sensors 31.

[0060] The storage unit 22 stores the information acquired by the acquisition unit 21. Specifically, the storage unit 22 stores the distance sensor information and the amount of movement in association with the identification information of the moving object that transmitted the distance sensor information and the amount of movement, and the time at which the moving object acquired the distance sensor information and the amount of movement. The storage unit 22 also stores information about the imaging range of the image sensor 31.

[0061] The map generation unit 23 generates a map showing the environment of the target space based on distance sensor information acquired from the moving objects 10a to 10d and the amount of movement of the moving objects 10a to 10d over a predetermined period. The predetermined period is a steady state period during which no environmental changes occur in the target space (i.e., a period during which the environment in the target space is managed). Based on the distance sensor information and amount of movement acquired during such a predetermined period, the map generation unit 23 generates a reference map, for example, by SLAM.

[0062] The environment update unit 24 updates the map when it detects that the environment shown on the map has changed, based on at least one of the distance sensor information and image information acquired after a predetermined period. Specifically, the environment update unit 24 updates the map when the correlation (time correlation) of multiple image information arranged in time series is low. The environment update unit 24 may also update the map based on distance sensor information acquired from each of the multiple moving objects 10a to 10d at the same time. The map update process will be described in more detail later.

[0063] The position estimation unit 25 estimates the positions of the moving objects 10a to 10d on the map based on the map updated by the environment update unit 24 and at least one of the following: distance sensor information, movement amount, and image information. For example, the position estimation unit 25 estimates the positions of the moving objects 10a to 10d on the map using the SLAM described above, based on the distance sensor information and movement amount. Alternatively, the position estimation unit 25 may estimate the positions of the moving objects 10a to 10d on the map based on the positions of the moving objects 10a to 10d captured in the image information.

[0064] The mobile unit control unit 26 generates control signals to control the movement of the mobile units 10a to 10d and transmits them to the mobile units 10a to 10d. The generated control signals are transmitted to each mobile unit 10a to 10d via the base station 30.

[0065] Figure 6 shows an example of the hardware configuration of the server 20 according to the first embodiment. The server 20 includes, for example, a CPU (Central Processing Unit) 1, RAM (Random Access Memory) 2, non-volatile memory 3, communication device 4, etc.

[0066] CPU1 is a processor for controlling the operation of server 20. CPU1 is at least one processor. CPU1 executes various programs loaded from non-volatile memory 3 into RAM2. These programs include the operating system (OS) and various application programs. In addition to CPU1, server 20 may also have other processors such as a GPU (Graphics Processing Unit).

[0067] The non-volatile memory 3 is a storage medium used as an auxiliary storage device. The non-volatile memory 3 can be used as a storage area for various types of information stored by the storage unit 22. In Figure 6, only the non-volatile memory 3 is shown, but the server 20 may also be equipped with other storage devices such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive).

[0068] RAM2 is a storage medium used as main memory. RAM2 can also be used as a temporary storage area for data used in processing by server 20.

[0069] The communication device 4 is a device configured to perform wireless communication with the mobile bodies 10a to 10d and the image sensor 31. The communication device 4 includes, for example, a receiving unit that receives an uplink signal from the outside and a transmitting unit that transmits a downlink signal to the outside. The receiving unit is capable of receiving signals from each of the mobile bodies 10a to 10d and the image sensor 31. The transmitting unit is capable of transmitting signals (control signals or synchronization signals) to each of the mobile bodies 10a to 10d.

[0070] Furthermore, some or all of the acquisition unit 21, storage unit 22, map generation unit 23, environment update unit 24, position estimation unit 25, and mobile object control unit 26 shown in Figure 5 are implemented by having the CPU 1 execute a predetermined program, i.e., by software. This program may be stored and distributed on a computer-readable storage medium, or it may be downloaded to the server 20 via a network. Furthermore, some or all of the acquisition unit 21, storage unit 22, map generation unit 23, environment update unit 24, position estimation unit 25, and mobile object control unit 26 may be implemented by dedicated hardware, or by a combination of software and hardware.

[0071] Next, an overview of the processing performed by the server 20 according to the first embodiment will be described. Figure 7 is a simplified diagram showing an example of the target space 100. The target space 100 is assumed to be a space in which, for example, a person P and an obstacle 40 exist.

[0072] Two mobile units 10a and 10b are moving within the target space 100. Each mobile unit 10a and 10b acquires surrounding environmental information (distance sensor information) while moving within the target space 100. At this time, the distance sensor information obtained from mobile units 10a and 10b is information within a narrow range (local viewpoint) around the mobile units 10a and 10b. For example, from the perspective of mobile unit 10b, because person P is hidden in shadow by obstacle 40, mobile unit 10b cannot acquire distance sensor information related to person P. Furthermore, because mobile units 10a and 10b are moving, they cannot detect the passage of time for an object at a given point.

[0073] Therefore, in this embodiment, multiple image sensors 31 are installed above the target space 100. Each of the multiple image sensors 31 captures the target space 100 from an overhead perspective. Since the entire environment of the target space 100 can be seen by the multiple image sensors 31, the position and time progression of person P can be determined.

[0074] Figure 8 shows the time change in the range of distance sensor information that can be acquired by the moving body 10a and the time change in the range of image information that can be acquired by the multiple image sensors 31.

[0075] In the left diagram of Figure 8, F1 indicates the range of distance sensor information that can be acquired by the moving object 10a. As shown in the left diagram of Figure 8, the range of distance sensor information that the moving object 10a can acquire also moves as the moving object 10a moves. Therefore, object A detected at time T1 is not detected at subsequent times T2 and T3.

[0076] In contrast, F2 in the right-hand diagram of Figure 8 indicates the shooting range that can be acquired by the multiple image sensors 31. As shown in the right-hand diagram of Figure 8, the shooting range that can be acquired by the multiple image sensors 31 is wide, and the entire target space 100 can be captured. Therefore, object A can be captured at all times T1 to T3. In other words, it is possible to track the time progression of object A and determine whether object A is dynamic or static.

[0077] In this case, the information obtained by the multiple image sensors 31 and the information obtained by the moving bodies 10a and 10b are dispersed. Therefore, a mechanism is needed to aggregate the information obtained by the multiple image sensors 31 and the information obtained by the moving bodies 10a and 10b.

[0078] Therefore, as shown in Figure 9, the server 20 according to this embodiment detects changes in the environment of the target space 100 and updates the map by aggregating distance sensor information obtained by the moving bodies 10a and 10b and shooting range information obtained by the multiple image sensors 31. The server 20 also estimates the positions of the moving bodies 10a and 10b based on the map.

[0079] Here, communication between the server 20, the mobile bodies 10a, 10b, and the multiple image sensors 31 is performed using high-speed, low-latency communication such as Local 5G. With high-speed, low-latency communication, the server 20 can send and receive information with the mobile bodies 10a, 10b, and the multiple image sensors 31 without delay, so that the mobile bodies 10a to 10d can be made to operate as if they were estimating their own positions on the map and moving autonomously.

[0080] Figure 10 is a flowchart showing an example of the map update process performed by the server 20 in the first embodiment. Note that the process shown in Figure 10 is performed for each of the multiple image sensors 31 if there are multiple image sensors 31.

[0081] The acquisition unit 21 of the server 20 acquires multiple image information from the image sensor 31 at predetermined time intervals (in chronological order) within a predetermined period (step S1).

[0082] The environment update unit 24 performs image processing on each of the multiple image information acquired in step S1 to identify the objects depicted in the image information (step S2). For object identification, for example, semantic segmentation is used. Semantic segmentation is a technique that assigns a label to each pixel of image information based on pre-learned features.

[0083] Then, the environment update unit 24 calculates the correlation value (i.e., time correlation) of the multiple image information to which labels have been assigned (step S3). The method for calculating the correlation value of the multiple image information can be any method. For example, the environment update unit 24 may calculate the correlation by referring to each pixel among the multiple image information and comparing the labels attached to those pixels.

[0084] The environment update unit 24 determines whether the correlation value of multiple image information is below a predetermined threshold (step S4). A high correlation value of multiple image information means that the image information is similar. In other words, it is highly likely that the environment in the range indicated by the image information has not changed. On the other hand, a low correlation value of multiple image information means that the image information is dissimilar. In other words, it is highly likely that the environment in the range indicated by the image information has changed.

[0085] In step S4, if the correlation value is greater than a predetermined threshold (NO in step S4), there is no change in the environment, and the server 20 terminates the map update process.

[0086] On the other hand, in step S4, if the correlation value is below a predetermined threshold (YES in step S4), the environment update unit 24 detects that there has been a change in the environment and updates the map (step S5).

[0087] For example, the environment update unit 24 identifies the type (point cloud) of the part that has changed between multiple image information. Specifically, based on the label information attached to each pixel of the multiple image information, the environment update unit 24 determines whether the part that has changed between the multiple image information is a known point cloud (an object already shown on the map that has moved) or a new object that has not yet been shown on the map (an unknown point cloud). It also determines whether the part is a static object (static point cloud) or a dynamic object (dynamic point cloud) based on whether the position of the part that has changed between the multiple image information has changed.

[0088] The environment update unit 24 then updates the map by reflecting the determined objects (point cloud). The environment update unit 24 also performs a process to convert the position of objects on the image to the position of objects on the map based on the imaging range of the image sensor 31.

[0089] As described above, the server 20 according to the first embodiment detects changes in the environment and updates the map based on image information from the image sensor 31. By using image information from an overhead view, even objects located in positions that cannot be detected by the distance sensors 11 of the moving objects 10a to 10d can be reflected in the map. Furthermore, since the time progression of objects can be tracked, it is easy to determine whether an object that has changed is a known point or an unknown point, and whether it is a static or dynamic object.

[0090] According to the server 20 of this embodiment, environmental changes can be reflected in the map, thereby improving the accuracy of estimating the positions of moving objects 10a to 10d on the map. For example, as shown in Figure 4, even when an obstacle 40 is placed, by reflecting the position of the obstacle 40 in the map, it is possible to prevent malfunctions such as calculating the position of the obstacle 40 as observation point L2. Furthermore, if it is determined that the obstacle 40 is unknown and a static object, the position of the moving objects 10a to 10d on the map can be calculated using the obstacle 40 as a new observation point. In this way, by updating the map, the accuracy of estimating the positions of moving objects 10a to 10d on the map can be improved.

[0091] Furthermore, according to the server 20 of this embodiment, the map can be updated while the mobile units 10a to 10d are in operation, rather than regenerating the map. This makes it possible to improve the accuracy of position estimation for the mobile units 10a to 10d without reducing overall operational efficiency.

[0092] In this embodiment, the server 20 is connected to multiple mobile bodies 10a to 10d and multiple image sensors 31. Therefore, the server 20 may generate multiple maps based on information acquired from each of the multiple mobile bodies 10a to 10d and the multiple image sensors 31, and update the map by aggregating these multiple maps.

[0093] Specifically, the server 20 generates multiple maps corresponding to each of the multiple moving objects 10a to 10d based on multiple distance sensor information acquired from each of the multiple moving objects 10a to 10d. It also generates multiple maps corresponding to each of the multiple image sensors 31 based on multiple image information acquired from each of the multiple image sensors 31. Then, it compares the generated multiple maps and updates the environment (positions of objects, etc.) shown on the maps to minimize the error.

[0094] By updating the map based on this wealth of information, a more accurate map can be generated. As a result, the accuracy of estimating the positions of moving objects 10a to 10d on the map can be improved.

[0095] (Second Embodiment) Next, a second embodiment will be described. Figure 11 shows the functional configuration of the server 20 according to the second embodiment. The server 20 according to the second embodiment differs from the first embodiment described above in that it further acquires power information received from the mobile bodies 10a to 10d and detects changes in the environment based on the received power information.

[0096] The received power information includes the intensity of the signal received by the mobile units 10a to 10d, and the time for which the signal was received. Here, the signal received by the mobile units 10a to 10d includes the downlink signal (control signal or synchronization signal) transmitted from the server 20, as well as interference signals.

[0097] In this embodiment, the interference signal is the uplink signal received by one of the mobile bodies 10a to 10d when that mobile body transmits an uplink signal. The interference signal will be described later with reference to Figure 12.

[0098] Furthermore, the storage unit 22 included in the server 20 stores information on the intensity of the interference signal and the intensity of the downlink signal. The intensity of the interference signal is stored in association with the time and location at which the interference signal was received. The intensity of the downlink signal is stored in association with the time and location at which the downlink signal was received.

[0099] Furthermore, the memory unit 22 has interference information pre-stored, which indicates the intensity of the interference signals between the moving bodies 10a to 10d. The interference information is the intensity of the interference signals when there are no objects between the moving bodies 10a to 10d and each of the moving bodies 10a to 10d is separated by a predetermined distance. For example, in the above state, the interference information indicates the intensity of the interference signals received by the other moving bodies 10b to 10d when moving body 10a transmits an uplink signal, the intensity of the interference signals received by the other moving bodies 10a, 10c, and 10d when moving body 10b transmits an uplink signal, the intensity of the interference signals received by the other moving bodies 10a, 10b, and 10d when moving body 10c transmits an uplink signal, and the intensity of the interference signals received by the other moving bodies 10a to 10c when moving body 10d transmits an uplink signal.

[0100] The interference signal will be explained below with reference to Figure 12. Mobile unit 10a transmits an uplink signal Sa, which includes distance sensor information and movement amount, to base station 30 (more specifically, server 20 connected to base station 30). Ideally, the radio waves of the uplink signal Sa should be radiated in a straight line toward base station 30, but in reality, the radio waves of the uplink signal Sa are radiated radially around mobile unit 10a. As a result, mobile units 10b to 10d other than mobile unit 10a receive signal Ia, which is leakage of the uplink signal Sa.

[0101] Similarly, when mobile body 10b transmits an uplink signal Sb, signal Ib is radiated around mobile body 10b. Mobile bodies 10a, 10c, and 10d, other than mobile body 10b, receive signal Ib, which is leakage of the uplink signal Sb. The same applies when the other mobile bodies 10c and 10d transmit uplink signals.

[0102] In this embodiment, communication between mobile units 10a to 10d (direct communication between terminals) is not intended. In other words, signals Ia and Ib are interference signals that may interfere with the uplink signal transmitted by each mobile unit and the downlink signal received by each mobile unit.

[0103] Mobile units 10a to 10d cannot acquire information contained in interference signals even if they receive them. However, mobile units 10a to 10d can measure the intensity (received power) of the interference signal. The server 20 according to this embodiment uses this information on the intensity of the interference signal to detect changes in the surrounding environment.

[0104] Hereinafter, for example, the interference signal received by mobile unit 10b when mobile unit 10a transmits an uplink signal, or the interference signal received by mobile unit 10a when mobile unit 10b transmits an uplink signal, may be referred to as the "interference signal between mobile unit 10a and mobile unit 10b".

[0105] Here, we will describe an example of a method for measuring the intensity of interference signals. For example, in a configuration where uplink and downlink signals are distinguished by reception time (TDD: Time Division Duplex), we will describe measuring the intensity of interference signals between mobile body 10a and other mobile bodies 10b to 10d.

[0106] In this case, mobile units 10b to 10d can measure the intensity of the interference signal between mobile unit 10a and mobile units 10b to 10d by measuring, for example, RSSI (Received Signal Strength Indicator) or SIR (Signal to Interference Ratio) when mobile unit 10a transmits an uplink signal at a time when the server 20 (base station 30) is not transmitting a downlink signal to any of the mobile units 10a to 10d.

[0107] Alternatively, if the radio equipment 17 of the mobile units 10a to 10d is in a configuration that allows simultaneous transmission of uplink signals and reception of downlink signals (Full Duplex), and the timing of transmission of uplink signals and reception of downlink signals overlap, the mobile units 10b to 10d may measure both the SIR when mobile unit 10a transmits an uplink signal and the SNR (Signal to Noise Ratio) when mobile unit 10a is not transmitting an uplink signal, and compare the two to measure the intensity of the interference signal between mobile unit 10a and the other mobile units 10b to 10d.

[0108] Furthermore, if the radio equipment 17 of the mobile units 10a to 10d has an array antenna, interference signals may be identified by direction of arrival estimation. Direction of arrival estimation is a technique that estimates the direction from which a signal was received based on the time difference (phase difference) of the signals received by each element of the array antenna. If the received signal is received from a direction other than the installation location of the base station 30, the mobile units 10a to 10d determine that the signal is an interference signal caused by another mobile unit transmitting an uplink signal, and measure the intensity of the signal as the intensity of the interference signal. Normally, direction of arrival estimation creates a null for interference signals, but by focusing on the eigenvalues ​​and eigenvectors of the correlation matrix, for example, and using the second eigenvector instead of the first eigenvector as the antenna weight, the interference signal can be obtained. In this case, the mobile units 10a to 10d transmit the received power information including information on the direction from which the interference signal was received. Then, the server 20 determines, based on the information in that direction and the positions of each moving object 10a to 10d on the map estimated by the position estimation unit 25, which moving objects the intensity of the interference signal is between.

[0109] Furthermore, in order to measure the intensity of interference signals more accurately, the radio equipment 17 of a specific mobile unit may be put into sleep mode. Specifically, for example, when measuring the intensity of interference signals between mobile unit 10a and mobile unit 10b, the radio equipment 17 of mobile units 10c and 10d, other than mobile units 10a and 10b, may be put into sleep mode (i.e., not transmitting signals).

[0110] Furthermore, the method for measuring the intensity of the interference signal is not limited to the method described above; other arbitrary methods may be used, such as using a predetermined filter.

[0111] Next, with reference to Figures 13 and 14, an overview of the method for detecting changes in the surrounding environment according to the second embodiment will be described. Figures 13 and 14 show two mobile bodies 10a and 10b stopped facing each other.

[0112] As shown in Figure 13, there are no obstacles between mobile units 10a and 10b. In this case, mobile unit 10a receives interference signal Ib when mobile unit 10b transmits an uplink signal Sb to base station 30. Similarly, mobile unit 10b receives interference signal Ia when mobile unit 10a transmits an uplink signal Sa to base station 30. The strength information of these interference signals Ia and Ib is included in the uplink signals Sa and Sb, respectively, and transmitted to base station 30 (server 20).

[0113] Subsequently, the environment between the moving body 10a and the moving body 10b changes, and we assume that an obstacle 40 is placed between the moving body 10a and the moving body 10b, as shown in Figure 14.

[0114] In this case, when mobile unit 10b transmits an uplink signal Sb to base station 30, mobile unit 10a receives an interference signal Ib that has been attenuated by the obstacle 40. Similarly, when mobile unit 10a transmits an uplink signal Sa to base station 30, mobile unit 10b receives an interference signal Ia that has been attenuated by the obstacle 40.

[0115] In other words, the intensity of the interference signals Ia and Ib between the mobile body 10a and the mobile body 10b changes depending on the environment between the mobile body 10a and the mobile body 10b. In this embodiment, changes in the environment between the mobile body 10a and the mobile body 10b are detected by detecting this change.

[0116] Figure 15 is a flowchart showing an example of the map update process flow of the server 20 according to the second embodiment.

[0117] The environment update unit 24 generates a received power map for each mobile unit 10a to 10d (step S11). The received power map shows the position of each mobile unit 10a to 10d on the map and the strength of the downlink signal (control signal or synchronization signal) received by each mobile unit 10a to 10d at that position.

[0118] Here, we will explain the received power map with reference to Figures 16 and 17. Note that the following explanation describes the received power map of mobile unit 10a, but the same principles apply to the received power maps of the other mobile units 10b to 10d.

[0119] For example, as shown in Figure 16, consider a case where a mobile body 10a moves along a predetermined path within the target space 100. At each position along the movement path, the mobile body 10a receives a downlink signal from the base station 30. The mobile body 10a also transmits received power information to the base station 30, including the intensity of the downlink signal and the time the mobile body 10a received the downlink signal.

[0120] The environment update unit 24 maps the intensity of the downlink signal received by the mobile body 10a to the location where the mobile body 10a received the downlink signal, thereby generating a received power map, for example, as shown in Figure 17.

[0121] Figure 17 is an example of a received power map generated for a portion of area 101 within the target space 100 shown in Figure 16. The position where the mobile body 10a received the downlink signal is the position of the mobile body 10a on the map estimated by the position estimation unit 25 at the time the mobile body 10a received the downlink signal. The received power map may be generated simultaneously, for example, when a reference map is generated by the map generation unit 23.

[0122] Return to Figure 15. The environment update unit 24 acquires new received power information from each of the mobile bodies 10a to 10d as each of the mobile bodies 10a to 10d moves (step S12).

[0123] When the environment update unit 24 acquires received power information, it updates the received power map generated in step S11 (step S13). Specifically, for example, when it receives received power information from mobile body 10a, the environment update unit 24 refers to the position on the received power map corresponding to the position where mobile body 10a received the downlink signal and updates the information on the intensity of the downlink signal mapped to that position. The same applies when it receives received power information from other mobile bodies 10b to 10d. Hereinafter, the mobile body that transmitted the received power information will be referred to as the "target mobile body". The position on the received power map corresponding to the position where the target mobile body received the downlink signal will be referred to as the "target position".

[0124] Furthermore, the environment update unit 24 determines whether the information on the downlink signal strength at the target location has changed by more than a threshold (i.e., whether the difference between the information on the downlink signal strength at the target location before the update and the information on the downlink signal strength at the target location after the update is greater than or equal to a threshold) (step S14).

[0125] A change in the downlink signal strength information at the target location means that the path (propagation path) through which the target mobile object receives radio waves from base station 30 has changed. In other words, there is a high probability that an object (hereinafter referred to as an environmental change factor) that causes the change in the propagation path exists in the vicinity of the target location (target mobile object).

[0126] In step S14, if it is determined that the information on the intensity of the downlink signal at the target location has not changed by more than a threshold (NO in step S14), the possibility of boundary fluctuation factors being present is low, and the environment update unit 24 returns to the process in step S12.

[0127] On the other hand, in step S14, if it is determined that the information regarding the intensity of the downlink signal at the target location has changed by more than a threshold (YES in step S14), the environment update unit 24 determines whether the target location is included in the range where image information cannot be acquired by the image sensor 31 (i.e., whether it is a blind spot for the image sensor 31) (step S15). In other words, because there is a high possibility that the environment has changed near the target moving object, it determines whether image information near the target moving object can be acquired.

[0128] If the target location is within the range from which image information can be acquired by the image sensor 31 (NO in step S15), the environment update unit 24 estimates the location of environmental change factors based on the image information including the target location acquired by the image sensor 31 and updates the map (step S16). The method for updating the map based on the information from the image sensor 31 is the same as in the first embodiment described above. After updating the map, the server 20 returns to the process in step S12.

[0129] On the other hand, if the target position is within a range where image information cannot be acquired by the image sensor 31 (YES in step S15), the mobile body control unit 26 moves the mobile bodies other than the target mobile body to the vicinity of the target mobile body (the vicinity of the target position) (step S17). The mobile bodies moved to the vicinity of the target mobile body may be all the mobile bodies other than the target mobile body, or some of the mobile bodies. Here, we will explain assuming that the mobile body control unit 26 has moved all the mobile bodies 10a to 10d to the vicinity of the target position.

[0130] The environment update unit 24 acquires received power information again from the mobile bodies 10a to 10d (step S18). Specifically, the control signal from the mobile body control unit 26 causes the mobile bodies 10a to 10d to receive the interference power between them, and transmit the intensity of this interference power in the uplink signal. The environment update unit 24 acquires this information on the intensity of the interference power between the mobile bodies 10a to 10d via the acquisition unit 21.

[0131] The environment update unit 24 determines whether the amount of data acquired is sufficient (step S19). Sufficient data is, for example, the number of links between each of the mobile units 10a to 10d (i.e., the number of combinations of selecting 2 out of 4 units, 4C2). If there are n mobile units, n This is the number of C2s. Furthermore, if the determination of whether the environmental fluctuation factor is a dynamic or static object is to be made, it is necessary to acquire the intensity of the interference signal over a predetermined period and detect whether or not there is a change. Therefore, a sufficient amount of data is a predetermined multiple of the number of links between each of the moving bodies 10a to 10d (for example, if the intensity of the interference signal is acquired 100 times / second for 10 seconds, n It is C2 × 1000.

[0132] If it is determined in step S19 that the amount of data acquired is insufficient (NO in step S19), the environment update unit 24 returns to the process in step S18 and acquires further information on the intensity of interference power.

[0133] In step S19, if it is determined that the amount of data acquired is sufficient (YES in step S19), the environment update unit 24 refers to the interference information stored in the storage unit 22 and determines whether a change (amount of change) has been detected in the intensity of the interference signal between the multiple moving objects (step S20). For example, if the intensity of the interference signal between moving object 10a and moving object 10b differs from the interference information stored in the storage unit 22, it is possible to detect that there has been a change in the environment between moving object 10a and moving object 10b (i.e., there is an environmental change factor). It is also possible to determine whether the environmental change factor is a dynamic object or a static object.

[0134] If no change is detected in step S20 (NO in step S20), the server 20 returns to the process in step S17 and repeats the process of moving to another location around the mobile body and acquiring the strength of the interference signal. This is because, although there is no change in the environment between the current mobile bodies 10a to 10d, it may be possible to detect a change in the environment by moving the mobile bodies 10a to 10d and changing the position of the link between them.

[0135] If no change is detected after repeating the processes in steps S17 to S20 a predetermined number of times, the server 20 may determine that no change has occurred in the environment, and without updating the map, send a control signal to the mobile objects 10a to 10d to return to their normal paths, and return to the process in step S12.

[0136] In step S20, if a change is detected (YES in step S20), the environment update unit 24 limits the area where the environment has changed based on the change (step S21). Specifically, for example, if the intensity of the interference signal between the moving body 10a and the moving body 10b differs from the interference information stored in the storage unit 22, the area between the moving body 10a and the moving body 10b is limited as the area where the environment has changed (where there are environmental change factors).

[0137] Subsequently, the environment update unit 24 updates the information of the area limited by step S21 on the map (step S22). For example, the environment update unit 24 moves the mobile bodies 10a to 10d further using the control signal from the mobile body control unit 26 to acquire further distance sensor information in the area limited by step S21.

[0138] Alternatively, the environment update unit 24, using a control signal from the mobile unit control unit 26, directs another mobile unit equipped with a high-performance sensor (for example, a sensor capable of acquiring 3D information) towards the area to acquire detailed information.

[0139] Based on this sensor information, the environment update unit 24 estimates the location of environmental change factors within the area limited by step S21 using methods such as SLAM described above, and updates the map. The environment update unit 24 may also determine whether the objects of the environmental change factors are dynamic or static, and whether they are unknown or known, and reflect this information (point cloud) in the map. Alternatively, the environment update unit 24 may identify environmental change factors using a pre-trained learning model that predicts objects from the waveforms of signals (downlink signals or interference signals) received by the moving bodies 10a to 10d.

[0140] After updating the map, the mobile unit control unit 26 sends a control signal to the mobile units 10a to 10d to return to the normal route. Then, the server 20 returns to the process in step S12 and repeats the subsequent processes.

[0141] Next, with reference to Figures 18 to 21, a specific example of applying the server 20 according to the second embodiment will be described. Figures 18 to 21 illustrate the case where mobile bodies 10a to 10d move within a target space 100 where known obstacles 40 are placed. Mobile bodies 10a to 10d receive downlink signals (control signals or synchronization signals) from a base station 30 connected to a server 20, and transmit uplink signals containing distance sensor information, movement amount, and received power information to the base station 30. Area A1 indicates a range (blind spot) where image information cannot be acquired by the image sensor 31.

[0142] Here, as shown in Figure 19, a static, unknown object 50 (i.e., an environmental change factor) is assumed to be placed near the base station 30 and the mobile body 10c. In this case, the downlink signal transmitted from the base station 30 is reflected by the object 50 and received by the mobile body 10c. That is, the intensity of the downlink signal received by the mobile body 10c changes. By detecting this change in the intensity of the downlink signal, the server 20 can detect that the environment has changed along the propagation path of the downlink signal.

[0143] However, when the distance between the mobile device 10c and the base station 30 is large, it is difficult to pinpoint the specific location where the environmental change occurred. Furthermore, the change in the intensity of the downlink signal may not be due to an environmental change, but rather to another mobile device blocking the propagation path.

[0144] Therefore, as shown in Figure 20, the server 20 according to this embodiment moves (deploys) the mobile bodies 10a, 10b, and 10d around the mobile body 10c. The server 20 then causes the mobile bodies 10a to 10d to receive interference signals between the mobile bodies 10a to 10d, and also receives information about said interference signals.

[0145] In the example shown in Figure 20, the interference signal between the moving body 10a and the moving body 10c is shielded by object 50. Therefore, the intensity of the interference signal between the moving body 10a and the moving body 10c is different from (changes from) the intensity of the interference signal between the moving body 10a and the moving body 10c shown in the interference information stored in the storage unit 22 beforehand. By detecting this change, the server 20 can limit the area where environmental fluctuation factors are present.

[0146] In the example shown in Figure 21, the server 20 limits area A2 between mobile bodies 10a and 10c to an area where the environment has changed (where environmental change factors are present).

[0147] Subsequently, the server 20 directs the mobile objects 10a to 10d toward the vicinity of the identified area A2, acquires distance sensor information and other data within area A2, estimates the position of object 50, and reflects it on the map.

[0148] As described above, the server 20 according to the second embodiment acquires received power information from each of the mobile bodies 10a to 10d and detects environmental changes based on the received power information. Specifically, the server 20 detects environmental changes when it detects a change in the intensity of the interference signal between the mobile bodies 10a to 10d.

[0149] According to the server 20 of the second embodiment, for example, even if the number of image sensors 31 installed is limited and image information for the entire range of the target space 100 cannot be acquired, the map can be updated by detecting changes in the environment using received power information.

[0150] Furthermore, in the second embodiment, the area where environmental fluctuation factors exist is limited based on changes in the intensity of the interference signal. For example, if the distance between the base station 30 and the mobile bodies 10a to 10d is large, it is difficult to limit the area where environmental fluctuation factors exist based solely on changes in the intensity of the communication signal (uplink or downlink signal) between the base station 30 and the mobile bodies 10a to 10d. Also, the possibility that the environmental fluctuation factors are other mobile bodies cannot be ruled out. According to this embodiment, since the area where environmental fluctuation factors exist can be limited based on changes in the intensity of the interference signal, it is possible to efficiently estimate the location of the environmental fluctuation factors and update the map.

[0151] In the example described above, the detection of environmental changes was explained by combining the image sensor 31 and received power information. However, the server 20 may also detect environmental changes based solely on received power information without using the image sensor 31. In other words, after detecting a change in the intensity of the downlink signal received by the target moving object, the server 20 may move another moving object around the target moving object without performing the process of determining whether or not the area is within the range where image information can be acquired by the image sensor 31 (the process of step S15 shown in Figure 15). In this case, the server 20 identifies the area based on the received power information (intensity of the interference signal) of the moving object and updates the map based on the distance sensor information and the amount of movement.

[0152] Furthermore, even within the range where image information can be acquired by the image sensor 31, other moving objects may be moved around the target moving object, and a process to identify the area based on the received power information (intensity of interference signals) of the moving objects (processing shown in steps S17 to S21 in Figure 15) may be performed. In this case, the map may be updated not only based on distance sensor information and amount of movement, but also based on image information within the identified area.

[0153] Furthermore, the server 20 may also detect environmental changes by combining the first and second embodiments. For example, the server 20 may detect environmental changes based on information from the image sensor 31, as in the first embodiment, and in areas where images cannot be acquired by the image sensor 31, it may detect environmental changes based on received power information.

[0154] According to at least one embodiment described above, it is possible to provide an information processing device and program that can improve the accuracy of estimating the position of a moving object on a map.

[0155] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0156] 10, 10a~10d...Mobile unit, 11...Distance sensor, 12...Motor, 13...Control unit, 17...Radio unit, 20...Server, 21...Acquisition unit, 22...Storage unit, 23...Map generation unit, 24...Environment update unit, 25...Position estimation unit, 26...Mobile unit control unit, 30...Base station, 31...Image sensor, Sa, Sb...Uplink signal, Ia, Ib...Interference signal.

Claims

1. An acquisition unit that acquires distance sensor information, the amount of movement of the multiple radios, and received power information from each of the multiple radios, and acquires image information from an image sensor, A map generation unit generates a map showing the environment in which the multiple wireless devices move, based on the distance sensor information and the amount of movement of the multiple wireless devices acquired over a predetermined period. An environment update unit updates the map when it detects that the environment shown on the map has changed, based on at least one of the distance sensor information, the received power information, and the image information acquired after the predetermined period. An information processing device equipped with the following.

2. The received power information includes information on the intensity of interference signals that are different from the downlink signals of the information processing device received by the plurality of radios, When the environment update unit detects a change in the intensity of the interference signal received by the first radio among the plurality of radios, it updates the map based on at least one of the distance sensor information, received power information, and image information acquired around the first radio. The information processing apparatus according to claim 1.

3. The interference signal is the uplink signal received by the first radio when a second radio, which is different from the first radio among the plurality of radios, transmits an uplink signal to the information processing device. The information processing apparatus according to claim 2.

4. The received power information further includes the time at which the plurality of radios received the interference signal, the intensity of the downlink signal, and the time at which the plurality of radios received the downlink signal. The system further comprises a storage unit that stores the intensity of the interference signal in relation to the time the interference signal was received, and stores the intensity of the downlink signal in relation to the time the downlink signal was received. The information processing apparatus according to claim 2.

5. The acquisition unit further acquires one or more image information from each of one or more image sensors installed at a different location from the image sensor. The environment update unit updates the map when it detects a change in the environment shown on the map based on the plurality of image information. The information processing apparatus according to claim 1.

6. The acquisition unit acquires the image information at predetermined time intervals. The environment update unit labels each pixel of each of the multiple image information acquired at predetermined time intervals, calculates the time correlation of the multiple labeled image information, and updates the map if the time correlation is low. The information processing apparatus according to claim 1.

7. In areas where the image sensor cannot acquire the image information, the environment update unit detects changes in the environment shown on the map based on at least one of the distance sensor information and the received power information. The information processing apparatus according to claim 1.

8. The predetermined period is the period during which the environment in which the multiple radios move is in a steady state. The information processing apparatus according to claim 1.

9. The system further comprises a position estimation unit that estimates the position of the plurality of wireless devices on the map based on the map updated by the environment update unit, the distance sensor information, the amount of movement of the plurality of wireless devices, and at least one of the image information. The information processing apparatus according to claim 1.

10. The system further comprises a control unit that transmits control signals to the plurality of radios for controlling the movement of the plurality of radios. The information processing apparatus according to claim 1.

11. The control unit transmits a control signal to move one or more of the multiple wireless devices to an area where the image information cannot be acquired by the image sensor. The environment update unit updates the map based on the distance sensor information and received power information from one or more wireless devices. The information processing apparatus according to claim 10.

12. When the control unit detects a change in the intensity of the control signal received by the third radio among the plurality of radios, it transmits a control signal to one or more fourth radios other than the third radio to move them to the vicinity of the third radio. The environment update unit updates the map based on at least one of the following: the distance sensor information of the third radio and one or more fourth radios, the received power information of the third radio and one or more fourth radios, and the image information. The information processing apparatus according to claim 10.

13. The aforementioned multiple radios are mounted on each of the multiple mobile units. The information processing apparatus according to claim 1.

14. At least one of the aforementioned multiple mobile bodies is a transport robot. The information processing apparatus according to claim 13.

15. The environment update unit updates the map based on the distance sensor information, the movement amount of the multiple wireless devices, and the received power information while the multiple mobile devices are operating. The information processing apparatus according to claim 13.

16. An acquisition unit that acquires the strength of interference signals different from the communication signals of multiple radios from multiple radios, An environment update unit updates a pre-generated map showing the environment when it detects a change in the environment in which the multiple radios are moving, based on the time change in the intensity of the interference signal. An information processing device equipped with the following.

17. A program executed by a computer of an information processing device, To the aforementioned computer, The steps include acquiring distance sensor information, the amount of movement of the multiple wireless devices, and received power information from each of the multiple wireless devices, and acquiring image information from an image sensor. A step of generating a map showing the environment in which the multiple wireless devices move, based on the distance sensor information and the amount of movement of the multiple wireless devices acquired over a predetermined period, The steps include updating the map when it is detected that the environment shown on the map has changed, based on at least one of the distance sensor information, the received power information, and the image information acquired after the predetermined period, and A program to execute.

Images