Terminals, communication methods, and communication systems

Abstract

To appropriately control information notified from a mobile object.SOLUTION: A terminal includes a control unit that determines that the state of a moving object from among multiple candidates representing the state of the moving object on the basis of the position of a moving object and the state of a transport mechanism used by the moving object, and controls the transmission of notification information according to the state of the mobile object, and a transmitting unit that transmits the notification information to an external device that monitors the mobile object according to the transmission control.SELECTED DRAWING: Figure 19

Description

Technical Field

[0001] The present disclosure relates to a terminal, a communication method, and a communication system.

Background Art

[0002] In recent years, mobile bodies (e.g., robots) that travel autonomously or by user operation and perform work while moving indoors and / or outdoors have been studied.

[0003] For example, Patent Document 1 describes a system in which when a transport mechanism (e.g., an elevator) provided in a building transports a mobile body, map information of an area to which the mobile body is transported by the transport mechanism is transmitted to the mobile body.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, there is room for consideration regarding the control of information acquired by a mobile body and notified from the mobile body.

[0006] Non-limiting embodiments of the present disclosure contribute to providing a terminal, a communication method, and a communication system that can appropriately control information notified from a mobile body.

Means for Solving the Problems

[0007] A terminal according to one embodiment of the present disclosure includes a control unit that determines the state of the mobile body from among a plurality of candidates representing the state of the mobile body based on the position of the mobile body and the state of the transport mechanism used by the mobile body, and controls the transmission of notification information according to the state of the mobile body, and a transmission unit that transmits the notification information to an external device that monitors the mobile body in accordance with the transmission control.

[0008] A communication method according to one embodiment of the present disclosure determines the state of a mobile body from among a plurality of candidates representing the state of the mobile body based on the position of the mobile body and the state of the transport mechanism used by the mobile body, controls the transmission of notification information according to the state of the mobile body, and transmits the notification information to an external device that monitors the mobile body in accordance with the transmission control.

[0009] A communication system according to one embodiment of the present disclosure comprises a terminal and a base station, the terminal comprising: a first control unit that determines the state of a mobile body from among a plurality of candidates representing the state of the mobile body based on the position of the mobile body and the state of the transport mechanism used by the mobile body, and controls the transmission of notification information according to the state of the mobile body; and a first transmission unit that transmits the notification information to the base station in accordance with the transmission control; the base station comprising: a second receiving unit that receives the notification information; a second control unit that generates display information to be displayed on a display device that monitors the mobile body based on the notification information; and a second transmission unit that transmits the display information to the display device.

[0010] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or recording media, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0011] According to one embodiment of the present disclosure, it is possible to appropriately control the information notified from a mobile device.

[0012] Further advantages and effects of one embodiment of this disclosure will be made apparent from the specification and drawings. Such advantages and / or effects are provided by several embodiments and features described in the specification and drawings, but not all of them are necessarily provided in order to obtain one or more identical features. [Brief explanation of the drawing]

[0013] [Figure 1] A diagram showing an example of a monitoring system in one embodiment. [Figure 2] A diagram showing an example of notification information in the first example. [Figure 3] Figure showing an example of the monitoring server display in the first example. [Figure 4] A diagram showing an example of notification information in the second example. [Figure 5] A diagram showing an example of notification information in variation 1 of the second example. [Figure 6] A diagram showing an example of notification information in variation 2 of the second example. [Figure 7] Figure showing an example of the monitoring server display in the second example. [Figure 8] A diagram showing an example of notification information in the third example. [Figure 9] A diagram showing an example of notification information in the fourth example. [Figure 10] A diagram showing an example of notification information in variation 1 of the fourth example. [Figure 11] A diagram showing an example of notification information in variation 2 of the fourth example. [Figure 12] Figure showing the first display example of the monitoring server in the fourth example. [Figure 13] Figure showing a second display example of the monitoring server in the fourth example. [Figure 14] A diagram illustrating an example of the correspondence between the robot's status, notification information, and displayed content. [Figure 15] A flowchart showing the processing flow of a terminal in one embodiment. [Figure 16] A flowchart showing the processing flow of a base station in one embodiment. [Figure 17] Flowchart showing the processing flow of the monitoring server in one embodiment [Figure 18] Diagram showing an example of notification information in the fifth example [Figure 19] Diagram showing an example of notification information in the sixth example [Figure 20] Diagram showing an example of the position of the elevator [Figure 21] Diagram showing an example of notification information in the seventh example [Figure 22] Diagram showing an example of notification information in the eighth example [Figure 23] Diagram showing another example of the correspondence between the state of the robot, notification information, and display content [Figure 24] Flowchart showing a variation of the processing flow of the terminal in one embodiment [Figure 25] Flowchart showing the processing flow of S402 in FIG. 24 [Figure 26] Flowchart showing the processing flow of S401 in FIG. 24 [Figure 27] Block diagram showing a configuration example of a base station according to one embodiment [Figure 28] Block diagram showing a configuration example of a terminal according to one embodiment [Figure 29] Block diagram showing a configuration example of a monitoring server according to one embodiment

Embodiments for Carrying Out the Invention

[0014] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functions are denoted by the same reference numerals, and redundant descriptions are omitted.

[0015] (One embodiment) Technologies are being considered for seamless and highly accurate positioning both indoors and / or outdoors. Furthermore, technologies are being considered for rapid detection of which floor a person is located on among multiple floors indoors. In the following discussion, positioning and location detection may be interpreted interchangeably.

[0016] For example, when positioning is performed using satellites such as the Global Positioning System (GPS) (hereinafter referred to as satellite positioning), highly accurate positioning is possible outdoors, but the positioning accuracy may deteriorate indoors. Also, when positioning is performed using sensors such as accelerometers and gyroscopes, positioning is possible indoors, but sufficient positioning accuracy may not be obtained.

[0017] Furthermore, positioning using images captured by a camera (for example, a Time-of-Flight (TOF) camera) is being considered. For example, when using a TOF camera, it is possible to measure three-dimensional information by measuring the reflection time of pulsed light irradiated onto the subject for each pixel.

[0018] Research is underway to develop a technology that enables seamless and highly accurate positioning both indoors and / or outdoors by combining satellite positioning, TOF camera positioning, and indoor communication-based positioning.

[0019] The technology that combines the above-mentioned multiple positioning methods to perform seamless and highly accurate positioning indoors and / or outdoors is used in systems where a robot (an example of a mobile object) moves indoors and / or outdoors, and monitors its surroundings based on the robot's location information and camera images captured by the robot.

[0020] The following is an example of a use case for this embodiment, in which a robot of the monitoring system moves through a building with multiple floors, and transmits the robot's location information and camera images to a server used for monitoring the building (hereinafter referred to as the monitoring server). Then, a person performing the monitoring (hereinafter referred to as the user) checks the information displayed on the monitoring server.

[0021] Figure 1 shows an example of the monitoring system 1 in this embodiment. Figure 1 shows a base station 10, a robot 20x, a monitoring server 30, and an indoor positioning access point (AP) 40. A terminal 20 is mounted on the robot 20x.

[0022] In the following, robot 20x and terminal 20 may be substituted for each other. For example, the location information of robot 20x may be replaced with the location information of terminal 20.

[0023] In monitoring system 1, robot 20x moves around inside building B, acquiring location information and camera images, and terminal 20 transmits the robot 20x's location information and camera images to monitoring server 30.

[0024] Base station 10 connects wirelessly to terminal 20 and communicates wirelessly with terminal 20. The connection between base station 10 and terminal 20 may comply with the 5th generation mobile communication system (5G) standard, or it may comply with a different wireless communication standard. For example, Wi-Fi® communication and / or WiGig® communication may be used.

[0025] Furthermore, the base station 10 communicates with the monitoring server 30. The communication between the monitoring server 30 and the base station 10 may be wireless communication or wired communication. Alternatively, the communication between the monitoring server 30 and the base station 10 may be communication via a network such as the Internet.

[0026] The monitoring server 30 acquires information from the base station 10 and displays the acquired information on the display unit 313. The user performs monitoring by checking the information displayed on the display unit 313. If the robot 20x is moving according to the user's operation, the monitoring server 30 may control the robot 20x. For example, the monitoring server 30 may accept user operations and transmit operation information based on the accepted operations to the robot 20x (or terminal 20) via the base station 10.

[0027] The monitoring server 30 may also be referred to as a display device having a display unit 313. Alternatively, the display unit 313 may be connected to the monitoring server 30 by wire or wireless connection instead of being included in the monitoring server 30. In this case, the monitoring server 30 may also be referred to as a display control device that controls the display on the display unit 313.

[0028] In this embodiment, the base station 10 and the monitoring server 30 are shown as separate devices, but the base station 10 and the monitoring server 30 may be included in a single device. For example, the base station 10 may have the functions of the monitoring server 30, or the monitoring server 30 may have the functions of the base station 10.

[0029] Robot 20x moves autonomously or under user control. Robot 20x has at least a camera. Robot 20x acquires camera images while moving inside building B. The acquired camera images are transmitted to the monitoring server 30 via terminal 20 and base station 10. The camera on robot 20x may be, for example, a TOF camera. In the following, images taken by the camera mounted on robot 20x may be referred to as images taken by the robot.

[0030] Terminal 20 has one or more positioning devices. Terminal 20 acquires location information while moving inside building B. The acquired location information is transmitted to the monitoring server 30 via base station 10. The positioning devices of terminal 20 may include devices that determine position based on camera images. The positioning devices of terminal 20 may include devices that determine position using GPS. The positioning devices of terminal 20 may include devices that determine position based on signals transmitted from indoor positioning AP 40. Note that indoor positioning AP 40 may be a Wi-Fi AP or a WiGig AP.

[0031] The indoor positioning AP40 is installed on each floor and transmits indoor positioning signals to the terminal 20. The indoor positioning AP40 also transmits floor identification information (floor information) to the terminal 20.

[0032] The positioning device included in terminal 20 is not limited to the examples described above. For example, a positioning device using sensors such as an accelerometer and a gyroscope may be included in terminal 20.

[0033] Furthermore, while an example has been shown in which terminal 20 has a positioning device, this disclosure is not limited to this. For example, the positioning device may be mounted on a robot 20x instead of being included in terminal 20. In this case, terminal 20 may obtain positioning results (e.g., location information) from the positioning device.

[0034] While an example of a robot 20x having a camera has been shown, this disclosure is not limited to this. For example, the camera may be part of a configuration included in terminal 20.

[0035] In the example shown in Figure 1, robot 20x boards an elevator (an example of a transport mechanism) from the second floor of building B and disembarks on the third floor. The method of controlling the elevator is not particularly limited. For example, an arm provided on robot 20x may operate the elevator's control panel (operation buttons). Alternatively, robot 20x may communicate with the elevator and send information indicating instructions regarding boarding and alighting. Or, a monitoring user may remotely operate the elevator.

[0036] In the example shown in Figure 1, the robot 20x moves from the second floor to the third floor using the elevator, while the terminal 20 transmits (notifies) information to the base station 10. In the following, the information notified by the terminal 20 will be referred to as "notification information."

[0037] The following example shows how terminal 20 notifies base station 10 of notification information, but this disclosure is not limited thereto. For example, terminal 20 may notify base station 10 of notification information via indoor positioning AP 40. Furthermore, terminal 20 is not limited to notifying base station 10 of notification information, but may also notify monitoring server 30 of notification information. Base station 10 and monitoring server 30 are examples of external devices that monitor robot 20x.

[0038] When robot 20x moves between multiple floors using an elevator and patrols each floor, the appropriate control of notification information differs depending on the state of robot 20x. In this embodiment, the terminal 20 of robot 20x controls the notification of notification information according to the state of robot 20x. An example of the control of notification information is described below.

[0039] <Example 1> Terminal 20 controls the notification of notification information according to the state of robot 20x. Here, the state of robot 20x may be determined based on the position of robot 20x (for example, the position of terminal 20). Alternatively, the state of robot 20x may be determined based on the position of robot 20x and the position and state of the elevator used by robot 20x. The state of the elevator is defined, for example, by the open / closed state of the elevator doors and / or the congestion level of the elevator.

[0040] Here, the control of notification of notification information includes at least one of the following: deciding whether or not to notify notification information, and deciding on the content of the notification information (e.g., the format of the notification information) if notification information is to be notified.

[0041] The state of robot 20x is not particularly limited. In the first example, the following two states are defined:

[0042] The state in which robot 20x is outside the elevator space may be referred to as the "outside elevator state." The elevator space corresponds to the space inside the elevator separated by the elevator doors. For example, the state in which at least a part of robot 20x is outside the elevator space may be considered the outside elevator state.

[0043] The state in which robot 20x is located within the elevator space may be referred to as the "elevator-in-space state." For example, the state in which robot 20x is contained within the elevator space may be considered the elevator-in-space state.

[0044] Whether robot 20x is inside or outside the elevator space can be determined, for example, by whether or not the robot 20x's position is in a pre-stored elevator location.

[0045] In the first example, the elevator interior state includes both the state where the elevator doors are open and the state where the doors are closed.

[0046] Figure 2 shows an example of notification information in the first example. Figure 2 shows examples of notification information for both the elevator interior and the elevator exterior states. In Figure 2, the horizontal axis represents the time axis.

[0047] As shown in Figure 2, in the first example, when outside the elevator, robot 20x notifies notification information, while when inside the elevator, robot 20x does not notify notification information.

[0048] The format of the notification information displayed while inside the elevator includes a "Terminal ID" field containing information that identifies robot 20x, a "Floor Information" field containing information indicating the floor on which robot 20x is located, a "Location Information" field containing information indicating the location of robot 20x, and a "Camera Image Information" field containing information about images captured by a camera mounted on robot 20x.

[0049] For example, the floor information may be the ID of the indoor positioning AP40 to which the robot 20x has wirelessly connected on the floor where the robot 20x is located. In other words, the floor where the robot is located may be identified by the ID of the indoor positioning AP40 installed on each floor.

[0050] In the following, the information contained in the "Terminal ID" field, that is, the information that identifies robot 20x, may be referred to as "Terminal ID." Similarly, information contained in other fields may be referred to using the field name. Also, in the following, "the notification information has the same format" may be referred to as "the notification information is the same."

[0051] <Example of display for the first example> The following describes an example of how the display unit 313 of the monitoring server 30 displays information when notification information is provided, as in the first example described above.

[0052] Figure 3 shows an example of the display of the monitoring server 30 in the first example. Figure 3 shows an example of the display based on notification information when the robot 20x is inside the elevator, and an example of the display based on notification information when the robot 20x is outside the elevator. As shown in Figure 3, the information to be displayed is set individually for the inside-the-elevator state and the outside-the-elevator state.

[0053] When the robot is outside the elevator, the display unit 313 displays the position of the robot 20x and the camera image taken by the robot 20x. If the robot 20x is moving, the display unit 313 may also display the direction of the robot 20x's movement.

[0054] On the other hand, when the robot is inside the elevator, the display unit 313 displays the position of the robot 20x and information indicating that the robot 20x is inside the elevator. The information indicating that the robot 20x is inside the elevator may be represented, for example, by a message.

[0055] In addition, if the robot is inside an elevator, the display unit 313 may display information indicating which elevator the robot 20x is located inside, instead of showing the robot's position.

[0056] In the first example described above, when inside the elevator, the robot 20x's location information (i.e., the elevator's location) is known, so as shown in Figure 2, the robot 20x does not send notification information including its location. This operation reduces the notification of unnecessary information, thereby improving frequency utilization efficiency and allowing for appropriate control of the information sent from the robot 20x. Furthermore, when outside the elevator, notification information including the robot 20x's location is sent, which suppresses the deterioration of the quality of building monitoring using the robot 20x and allows for efficient monitoring. Deterioration of monitoring quality can be equivalent to the absence of information required for monitoring. In addition, the efficiency of monitoring improves along with the ease of monitoring.

[0057] Furthermore, in the first example described above, as shown in Figure 3, the information to be displayed is set individually for the elevator interior state and the elevator exterior state. This display makes it easy for the user to see and reduces the burden on the user's eyes to process the information.

[0058] <Example 2> The first example described above illustrates an elevator interior state that includes both the state where the elevator doors are open and the state where the doors are closed, but the disclosure is not limited thereto. In the second example, the elevator interior state can be further divided into several states depending on the state of the elevator doors and the floor to which the elevator has stopped.

[0059] As shown in Figure 1, for example, when robot 20x moves from the second floor to the third floor, the floor on which robot 20x boards is the second floor, and the floor on which robot 20x disembarks is the third floor.

[0060] For example, the state of being inside an elevator, where the elevator is on the floor where robot 20x is sitting, and the elevator doors are open, can be referred to as the "elevator riding state."

[0061] Furthermore, for example, the state in which the elevator is inside the elevator, and the elevator has stopped at the floor where robot 20x is disembarking, and the elevator doors are open, may be referred to as the "elevator disembarking state."

[0062] In the second example, a state inside the elevator that is neither "elevator boarding state" nor "elevator exit state" (for example, a state where the elevator doors are closed) may be referred to as the "elevator door closed state."

[0063] The method for determining whether a floor with an elevator is the floor where robot 20x boarded or the floor where robot 20x disembarks is not particularly limited. For example, robot 20x determines that its current floor is the floor where robot 20x boarded if the floor information received from the indoor positioning AP40 indicates the floor where robot 20x boarded. Also, robot 20x determines that its current floor is the floor where robot 20x disembarks if the floor information received from the indoor positioning AP40 indicates the floor where robot 20x disembarks.

[0064] The method for determining whether the elevator doors are open or closed when robot 20x is located at the elevator is not particularly limited. For example, if robot 20x can communicate wirelessly with indoor positioning AP40, robot 20x may determine that the elevator doors are open. If robot 20x cannot communicate wirelessly with indoor positioning AP40 (cannot send or receive signals), robot 20x may determine that the elevator doors are closed. Alternatively, if the signal strength of the signal received by robot 20x from indoor positioning AP40 is greater than a predetermined value, robot 20x may determine that the elevator doors are open, and if the signal strength is less than or equal to the predetermined value, robot 20x may determine that the elevator doors are closed. Alternatively, robot 20x may determine whether the elevator doors are open or closed by processing a camera image taken by robot 20x that includes the elevator doors.

[0065] Figure 4 shows an example of notification information in the second example. Figure 4 shows examples of notification information for the following states: outside the elevator, inside the elevator, with the elevator doors closed, and after exiting the elevator. In Figure 4, the horizontal axis represents the time axis.

[0066] As shown in Figure 4, in the second example, when outside the elevator, robot 20x sends notification information. Robot 20x also sends notification information when inside and outside the elevator. On the other hand, when the elevator doors are closed, robot 20x does not send notification information.

[0067] In the second example, the notification information provided may be the same as the notification information in the first example shown in Figure 2.

[0068] In the second example described above, when the elevator doors are closed, the robot 20x does not send notification information containing its location because its position is already known. This operation reduces the notification of unnecessary information, thereby improving frequency utilization efficiency and allowing for proper control of the information sent from the robot 20x. Furthermore, when the elevator is outside, notification information containing the robot 20x's location is sent, which helps to suppress the deterioration of the quality of building monitoring using the robot 20x and allows for more efficient monitoring.

[0069] Furthermore, in both the elevator boarding and elevator alighting states, notification information including the robot 20x's location is sent, similar to the state outside the elevator. This allows the monitoring server 30 to distinguish whether the robot 20x is boarding or alighting from the elevator, thereby suppressing a deterioration in monitoring quality. For example, if the robot 20x remains inside the elevator for a longer period than expected, it becomes easy to determine whether the time exceeding is due to the time taken to board or alight from the elevator, or due to an elevator malfunction.

[0070] In the second example described above, the notification information provided in the elevator boarding state and the elevator alighting state was shown to be the same as in the state outside the elevator, but this disclosure is not limited to this. Below, variations in the notification information provided in the elevator boarding state, elevator alighting state and outside the elevator state will be described.

[0071] <Variation 1 of the second example> Figure 5 shows an example of notification information in variation 1 of the second example. Similar to Figure 4, Figure 5 shows examples of notification information for the elevator outside state, elevator inside state, elevator door closed state, and elevator exit state. In Figure 5, the horizontal axis represents the time axis.

[0072] In Figure 5, the notification information format A, displayed when the user is outside the elevator, includes a "Terminal ID" field, a "Floor Information" field, an "Identification Information" field, a "Location Information" field, and a "Camera Image Information" field. Similarly, the notification information format B, displayed when the user is inside or outside the elevator, also includes a "Terminal ID" field, a "Floor Information" field, and an "Identification Information" field. The "Terminal ID" field, "Floor Information" field, "Location Information" field, and "Camera Image Information" field may be the same as those shown in Figure 2.

[0073] Here, the "Identification Information" field contains information indicating whether the "Location Information" field and the "Camera Image Information" field follow the "Identification Information" field in the notification information. For example, "Identification Information" is a 1-bit bit. In notification information sent when outside the elevator, the "Location Information" field and the "Camera Image Information" field follow the "Identification Information" field, so the 1 bit of "Identification Information" may be "1". Also, in notification information sent when inside or outside the elevator, the "Location Information" field and the "Camera Image Information" field do not follow the "Identification Information" field, so the 1 bit of "Identification Information" may be "0".

[0074] In variation 1 of the second example described above, the robot 20x's location information and camera image information are not notified when the robot is on or off the elevator. When the robot is on or off the elevator, the monitoring server 30 can distinguish whether the robot is on or off the elevator based on floor information, thus reducing the notification of unnecessary information, improving frequency utilization efficiency, and enabling appropriate control of the information notified by the robot 20x. In addition, by being able to distinguish whether the robot is on or off the elevator, the deterioration of monitoring quality can be suppressed, and monitoring can be performed efficiently.

[0075] <Variation 2 of the second example> Figure 6 shows an example of notification information in variation 2 of the second example. Similar to Figures 4 and 5, Figure 6 shows examples of notification information for the elevator outside state, elevator inside state, elevator door closed state, and elevator exit state. In Figure 6, the horizontal axis represents the time axis.

[0076] The difference between Variation 2 and Variation 1 is that in Variation 1, as shown in Figure 5, the same notification information is sent when the user is disembarking from the elevator as when the user is boarding the elevator, whereas in Variation 2, different notification information is sent when the user is disembarking from the elevator compared to when the user is boarding the elevator.

[0077] In Figure 6, the same notification information is sent when the user is outside the elevator as when the user is outside the elevator. For example, format A of the notification information sent when the user is outside the elevator includes a "Terminal ID" field, a "Floor Information" field, an "Identification Information" field, a "Location Information" field, and a "Camera Image Information" field.

[0078] In variation 2 of the second example described above, the robot 20x's position information and camera image information are notified when it disembarks from the elevator. This notification reduces the time required to determine the robot 20x's position on the floor where it disembarked, for example, when using the camera image for indoor positioning.

[0079] <Example of display for the second example> The following describes examples of how the monitoring server 30 displays notification information when it is notified, as in the second example described above and its variations.

[0080] Figure 7 shows an example of the display on the monitoring server in the second example. Figure 7 shows an example of the display based on notification information when the robot 20x is in the elevator boarding state, and an example of the display based on notification information when the robot is in the elevator disembarking state.

[0081] When the robot is in the elevator, the display unit 313 displays the robot's position and information indicating that the robot is in the elevator. In this case, since the notification information does not include camera image information, images taken by the robot 20x do not need to be displayed. The information indicating that the robot 20x is in the elevator may be represented, for example, by a message.

[0082] When the robot is in the elevator exit state, the display unit 313 displays the position of the robot 20x and information indicating that the robot 20x is exiting the elevator. In this case, since the notification information does not include camera image information, images taken by the robot 20x do not need to be displayed. The information indicating that the robot 20x is exiting the elevator may be represented, for example, by a message.

[0083] In the second example, if the elevator is outside the elevator, the display unit 313 may show the same display as in the first example (for example, Figure 3) when the elevator is outside the elevator. Also, in the second example, if the elevator doors are closed, the display unit 313 may show the same display as in the first example (for example, Figure 3) when the elevator is inside the elevator.

[0084] As described above, in the second example of the display, as shown in Figures 3 and 7, the information to be displayed is set individually for each state. This display makes it easy for the user to see and reduces the burden on the user's visual perception.

[0085] <Third example> In the second example, an elevator is present on both the floor where robot 20x boarded and the floor where it disembarked, but the disclosure is not limited to this. In the third example, robot 20x is in an elevator state, and the elevator is on a floor different from the floor where robot 20x boarded and the floor where robot 20x disembarked. Hereafter, a floor different from the floor where robot 20x boarded and the floor where robot 20x disembarked will be referred to as an intermediate floor.

[0086] For example, a state in which the elevator is inside the elevator and has stopped on a floor that is neither the floor where robot 20x boarded nor the floor where it will disembark, and the elevator doors are open, can be referred to as an "intermediate floor state."

[0087] In the third example, a state inside the elevator that is neither "elevator boarding state," "elevator alighting state," nor "intermediate floor state" (for example, a state where the elevator doors are closed) may be referred to as the "elevator door closed state."

[0088] There are no particular limitations on how to determine whether the floor where the elevator is located is the floor where robot 20x boarded, the floor where robot 20x will disembark, or an intermediate floor. For example, robot 20x determines that its current floor is the floor where robot 20x boarded if the floor information received from the indoor positioning AP40 indicates the floor where robot 20x boarded. Also, robot 20x determines that its current floor is the floor where robot 20x will disembark if the floor information received from the indoor positioning AP40 indicates the floor where robot 20x will disembark. Furthermore, robot 20x determines that its current floor is an intermediate floor if the floor information received from the indoor positioning AP40 indicates a floor that is neither the floor where robot 20x boarded nor the floor where robot 20x will disembark.

[0089] Figure 8 shows an example of notification information in the third example. Figure 8 shows examples of notification information for each of the following states: outside the elevator, inside the elevator, with the elevator doors closed, on an intermediate floor, and after exiting the elevator. In Figure 8, the horizontal axis represents the time axis.

[0090] For example, in the example shown in Figure 8, when robot 20x moves from the 1st floor to the 4th floor, the floor where the elevator stops when the robot is on the elevator is the 1st floor, and the floor where the elevator stops when the robot is alighting from the elevator is the 4th floor. In this case, the floors where the elevator stops during the two intermediate floor states are the 2nd floor and the 3rd floor.

[0091] As shown in Figure 8, in the third example, when outside the elevator, robot 20x notifies notification information. Robot 20x also notifies notification information when inside the elevator and when disembarking from the elevator. On the other hand, robot 20x does not notify notification information when the elevator doors are closed or when the elevator is on an intermediate floor.

[0092] In the third example, the format A of the notification information sent when outside the elevator may be the same as the format A of the notification information sent when outside the elevator in variation 1 of the second example shown in Figure 5. Also, in the third example, the format B of the notification information sent when inside the elevator and when alighting from the elevator may be the same as the format B of the notification information sent when inside the elevator and when alighting from the elevator in variation 1 of the second example shown in Figure 5.

[0093] In the third example described above, even if the elevator doors are open, the robot does not send notification information while the elevator is on an intermediate floor. This action reduces the notification of unnecessary information, thereby improving frequency utilization efficiency and allowing for proper control of the information sent from robot 20x.

[0094] Note that the display example for the third example may be the same as the display example for the second example (for example, Figures 7 and 3).

[0095] <Example 4> The third example shows a case where the robot 20x does not notify notification information during an intermediate floor state, but the disclosure is not limited thereto. The fourth example shows a case where the robot 20x does notify notification information during an intermediate floor state.

[0096] Figure 9 shows an example of notification information in the fourth example. Similar to Figure 8, Figure 9 shows examples of notification information for each of the following states: outside the elevator, inside the elevator, with the elevator doors closed, on an intermediate floor, and after exiting the elevator. In Figure 9, the horizontal axis represents the time axis.

[0097] The difference between the third example and the fourth example is that in the fourth example, as shown in Figure 9, the robot 20x notifies notification information during the intermediate floor state.

[0098] In the fourth example, the notification information provided when outside the elevator may be the same as the notification information provided when outside the elevator in variation 1 of the second example shown in Figure 5. Also, in the fourth example, the notification information provided when boarding the elevator, disembarking from the elevator, and on intermediate floors may be the same as the notification information provided when boarding the elevator and disembarking from the elevator in variation 1 of the second example shown in Figure 5.

[0099] In the fourth example described above, the robot 20x notifies the server 30 of notification information while the elevator is on an intermediate floor. This action allows the monitoring server 30 to monitor whether the elevator is stopped on an intermediate floor or not. For example, even if the elevator is stopped on an intermediate floor for a predetermined amount of time or longer, monitoring can be performed appropriately.

[0100] In the fourth example described above, the notification information provided during intermediate floor states was the same as that provided during elevator boarding and elevator alighting states, but this disclosure is not limited to this. The following describes variations in the notification information provided during intermediate floor states.

[0101] <Variation 1 of the fourth example> Figure 10 shows an example of notification information in variation 1 of the fourth example. Similar to Figure 8, Figure 10 shows examples of notification information for each of the following states: outside the elevator, inside the elevator, with the elevator doors closed, on an intermediate floor, and after exiting the elevator. In Figure 10, the horizontal axis represents the time axis.

[0102] In Figure 10, the notification information format A, displayed during the intermediate floor state, includes the following fields: "Terminal ID," "Floor Information," "Identification Information," "Location Information," and "Camera Image Information." In other words, the notification information format displayed during the intermediate floor state in Figure 10 is the same as the notification information format displayed when the elevator is outside the elevator in Figure 10.

[0103] In variation 1 of the fourth example described above, the robot sends notification information including location information and camera image information when it is on an intermediate floor. This notification allows the monitoring server to monitor whether the elevator is stopped on an intermediate floor or not.

[0104] Furthermore, if the number of people getting off and / or getting on the elevator (hereinafter referred to as "number of passengers") exceeds a predetermined number on an intermediate floor, the robot may temporarily exit the elevator. In such cases, in variation 1 of the fourth example described above, the robot 20x sends notification information including location information and camera image information, so that the robot 20x can be appropriately controlled remotely to temporarily exit the elevator.

[0105] <Variation 2 of the fourth example> Figure 11 shows an example of notification information in variation 2 of the fourth example. Similar to Figure 8, Figure 11 shows examples of notification information for each of the following states: outside the elevator, inside the elevator, with the elevator doors closed, on an intermediate floor, and after exiting the elevator. In Figure 11, the horizontal axis represents the time axis.

[0106] The intermediate floor states in Figure 11 include a first intermediate floor state and a second intermediate floor state. The floors on which the elevator is stopped in the first intermediate floor state are different from the floors on which the elevator is stopped in the second intermediate floor state.

[0107] For example, in the example shown in Figure 11, when robot 20x moves from the 1st floor to the 4th floor, the floor where the elevator stops when the robot is on the elevator is the 1st floor, and the floor where the elevator stops when the robot is alighting is the 4th floor. In this case, the floor where the elevator stops in the first intermediate floor state is the 2nd floor, and the floor where the elevator stops in the second intermediate floor state is the 3rd floor.

[0108] In the example shown in Figure 11, the number of people getting off and / or getting on the elevator (number of passengers getting on and off) when the elevator stops on the 2nd floor is less than or equal to a predetermined number, while the number of people getting off and / or getting on the elevator when the elevator stops on the 3rd floor is greater than or equal to a predetermined number. In other words, in the first intermediate floor state, the number of people getting off and / or getting on the elevator is less than or equal to a predetermined number, while in the second intermediate floor state, the number of people getting off and / or getting on the elevator is greater than or equal to a predetermined number. In this case, the notification information that the robot notifies in the first intermediate floor state is different from the notification information that the robot notifies in the second intermediate floor state.

[0109] The method for estimating the number of people getting off and / or getting on the elevator (total number of passengers) is not limited. The number of passengers may be estimated using image processing of camera images (for example, facial recognition processing), or using sensing technology with sensors such as infrared sensors and weight sensors.

[0110] For example, the robot 20x may photograph people inside the elevator and people waiting for the elevator, and then use image processing (e.g., facial recognition) of the captured camera images to estimate the number of people getting off the elevator and the number of people getting on the elevator.

[0111] Alternatively, a camera installed inside the elevator (for example, on the ceiling or near the doors) may photograph people inside the elevator, and the number of people about to get off the elevator may be estimated by image processing of the captured camera images. Alternatively, cameras installed on each floor may photograph people waiting for the elevator, and the number of people about to get on the elevator may be estimated by image processing of the captured camera images (for example, facial recognition). The estimated number of people may be notified to the robot 20x (terminal 20) or to the monitoring server 30. If the robot 20x receives the notification, it may determine the notification information to be sent based on the estimated number of people. Alternatively, if the monitoring server 30 receives the notification, it may determine the content (for example, the format) of the notification information to be sent based on the estimated number of people and notify the terminal 20 of the result.

[0112] Alternatively, if information such as the disembarking floor is being transmitted and / or received between a terminal held by a person inside the elevator and a communication device (e.g., an access point (AP)) installed inside the elevator, the robot 20x (terminal 20) may obtain information about the number of people disembarking from the elevator from the communication device installed inside the elevator. For example, if the number of people disembarking from the elevator is greater than a predetermined number, information about the number of people disembarking from the elevator may be notified from the communication device installed inside the elevator. In this case, if the number of people disembarking from the elevator is less than or equal to the predetermined number, information about the number of people disembarking from the elevator does not need to be notified from the communication device installed inside the elevator.

[0113] In the example in Figure 11, the notification information format B, which is notified in the first intermediate floor state, includes the "Terminal ID" field, the "Floor Information" field, and the "Identification Information" field, but does not include the "Location Information" field or the "Camera Image Information" field. On the other hand, in the example in Figure 11, the notification information format A, which is notified in the second intermediate floor state, includes the "Terminal ID" field, the "Floor Information" field, the "Identification Information" field, the "Location Information" field, and the "Camera Image Information" field.

[0114] In variation 2 of the fourth example described above, the notification information that the robot sends is determined according to the elevator congestion level at intermediate floors. For example, in the example described above, if the number of passengers getting on or off at intermediate floors exceeds a predetermined number, notification information including location information and camera image information is sent, so that the robot 20x can be appropriately controlled when remotely controlling it to temporarily disembark from the elevator. On the other hand, if the number of passengers getting on or off at intermediate floors is less than or equal to a predetermined number, notification information that does not include location information and camera image information is sent, so that frequency utilization efficiency can be improved and the information notified by the robot 20x can be appropriately controlled.

[0115] <Example of display for the fourth example> The following describes examples of how notification information is displayed on the monitoring server when it is notified, as in the fourth example described above and its variations.

[0116] Figure 12 shows a first display example of the monitoring server 30 in the fourth example. Figure 12 is a display example corresponding to variation 2 of the fourth example. Figure 12 shows a display example based on notification information when the robot is in an intermediate floor state and the number of people getting on and off is greater than a predetermined number, and a display example based on notification information when the robot is in an intermediate floor state and the number of people getting on and off is less than or equal to a predetermined number.

[0117] If the robot is in an intermediate floor state and the number of passengers boarding or alighting exceeds a predetermined number, the display unit 313 displays the robot's position and an image taken by the robot. In this case, the display unit 313 also displays information indicating the floor on which the elevator is stopped. This information may be represented, for example, by a message.

[0118] If the robot is in an intermediate floor state and the number of passengers getting on or off is less than a predetermined number, the display unit 313 displays the robot's position and information indicating the floor on which the elevator is stopped. The information indicating the floor on which the elevator is stopped may be represented, for example, by a message.

[0119] Furthermore, if the robot is in an intermediate floor state and the number of passengers boarding or alighting exceeds a predetermined number, the display content may be changed depending on whether or not the robot has moved outside the elevator. An example of this change is explained below.

[0120] Figure 13 shows a second display example of the monitoring server in the fourth example. Figure 13 is a display example corresponding to variation 2 of the fourth example. Figure 13 shows a display example based on notification information when the robot is in an intermediate floor state and the number of people getting on and off is greater than a predetermined number, and a display example based on notification information when the robot is in an intermediate floor state and the number of people getting on and off is less than or equal to a predetermined number.

[0121] When the robot moves outside the elevator, the display unit 313 displays the robot's position and an image taken by the robot. In this case, the display unit 313 also displays information indicating the floor on which the elevator is stopped and information indicating that the robot is moving outside the elevator. The information indicating the floor on which the elevator is stopped and the information indicating that the robot is moving outside the elevator may be represented, for example, by a message.

[0122] If the robot does not move outside the elevator, the display unit 313 displays the robot's position and an image taken by the robot. In this case, the display unit 313 also displays information indicating the floor the elevator is stopped on and information indicating that the robot is inside the elevator. The information indicating the floor the elevator is stopped on and the information indicating that the robot is inside the elevator may be represented, for example, by a message.

[0123] As shown in the examples above, a correspondence is defined between the robot's status, the content of the notification information, and the content displayed on the monitoring server. This correspondence is explained below.

[0124] Figure 14 shows an example of the correspondence between the robot's state, notification information, and display content. Figure 14 shows the content of the notification information and the display content corresponding to each state: "Outside the elevator," "Elevator doors closed," "Inside the elevator," "Out of the elevator," "Intermediate floor (few passengers getting on or off)," and "Intermediate floor (many passengers getting on or off)."

[0125] In the "Elevator Out" state, floor information, location information, and camera image information are notified. In the "Elevator Out" state, the monitoring server displays the location information and camera image, but no additional messages are displayed.

[0126] When the elevator doors are closed, floor information, location information, and camera image information are not transmitted. However, when the elevator doors are closed, the monitoring server displays location information indicating that the robot is inside the elevator, and an additional message stating "The robot is inside the elevator" is displayed.

[0127] In the "elevator occupancy" state, floor information is notified, but location information and camera image information are not. In the "elevator occupancy" state, the monitoring server displays location information indicating that the robot is inside the elevator, and an additional message stating "The robot is currently in the elevator" is displayed.

[0128] In the "elevator exit state," floor information is notified, but location information and camera image information are not. In the "elevator exit state," the monitoring server displays location information indicating that the robot is inside the elevator, and an additional message stating that "the robot is exiting the elevator" is displayed.

[0129] The "Intermediate Floor State (Few Passengers)" corresponds to a state where robot 20x is in an intermediate floor state and the number of passengers getting on or off is below a predetermined number. In the "Intermediate Floor State (Few Passengers)" state, floor information is notified, but location information and camera image information are not. Furthermore, in the "Intermediate Floor State (Few Passengers)" state, location information indicating that the robot is inside the elevator is displayed on the monitoring server, and an additional message indicating "The robot is stopped on floor X (the floor where the elevator is currently stopped)" is displayed.

[0130] "Intermediate Floor State (High Number of Boarding / Alighting)" corresponds to a state where robot 20x is in an intermediate floor state and the number of people boarding or alighting exceeds a predetermined number. In the "Intermediate Floor State (High Number of Boarding / Alighting)" state, floor information, location information, and camera image information are notified. In the "Intermediate Floor State (High Number of Boarding / Alighting)" state, the monitoring server displays location information and camera images indicating that the robot is inside the elevator, and an additional message indicating that "the robot is stopped on floor X (the floor where the elevator is stopped)" is displayed. In this state, an additional message indicating whether the robot is moving outside the elevator may also be displayed. Furthermore, if the number of people boarding or alighting exceeds a predetermined number, an additional message indicating that the number of people boarding or alighting exceeds a predetermined number may also be displayed. In this case, the number of people boarding and alighting may be displayed separately. For example, additional messages such as "High Number of Boarders," "High Number of Alighters," or "High Number of Boarding / Alighting" may be displayed.

[0131] The examples and variations described above are just examples, and this disclosure is not limited to them. For example, the robot's actions (terminal actions) may be the same whether the robot is inside the elevator during an intermediate floor state or whether the robot gets off the elevator and gets back on during an intermediate floor state.

[0132] <Processing flow in this embodiment> Next, an example of the processing flow for the terminal 20, base station 10, and monitoring server 30 in this embodiment will be described.

[0133] Figure 15 is a flowchart showing the processing flow of terminal 20 in this embodiment. For example, the flowchart shown in Figure 15 may start when terminal 20 is powered on, or when the robot 20x on which terminal 20 is mounted starts operating. Also, the flowchart shown in Figure 15 may end when the operation of terminal 20 stops, or when the operation of the robot 20x on which terminal 20 is mounted stops. Alternatively, the start and / or end of the flowchart may be controlled by user operation.

[0134] Terminal 20 determines its state (S201). For example, terminal 20 determines the state of the robot 20x on which it is mounted, based on its location information, etc.

[0135] Terminal 20 determines whether the determined state is outside the elevator (S202).

[0136] If the determined state is outside the elevator (YES in S202), terminal 20 determines whether or not the notification timing has arrived (S203). The notification timing may be set in advance or instructed by base station 10.

[0137] If the notification timing has not yet arrived (NO in S203), the flow returns to S201.

[0138] When the notification timing arrives (YES in S203), terminal 20 generates notification information and transmits the generated notification information to base station 10 (S204). The notification information generated in S204 may include floor information, location information, and camera image information. Then, the flow returns to S201.

[0139] If the determined state is not outside the elevator (NO in S202), terminal 20 determines whether the determined state is on an intermediate floor (S205).

[0140] If the determined state is an intermediate floor state (YES in S205), terminal 20 determines whether the number of people getting on and off the elevator is greater than a predetermined number (S206).

[0141] If the number of people getting on or off the elevator exceeds a predetermined number (YES in S206), the process in S203 is executed.

[0142] If the number of people getting on and off the elevator is not greater than a predetermined number (NO in S206), terminal 20 determines whether or not the notification timing has arrived (S207).

[0143] If the notification timing has not yet arrived (NO in S207), the flow returns to S201.

[0144] When the notification timing arrives (YES in S207), terminal 20 generates notification information and sends the generated notification information to base station 10 (S208). The notification information generated in S208 includes floor information, but does not need to include location information or camera image information. The flow then returns to S201.

[0145] If the determined state is not an intermediate floor state (NO in S205), terminal 20 determines whether the determined state is an elevator boarding state (S209).

[0146] If the determined state is elevator boarding state (YES in S209), terminal 20 determines whether or not the notification timing has arrived (S210).

[0147] If the notification timing has not yet arrived (NO in S210), the flow returns to S201.

[0148] When the notification timing arrives (YES in S210), terminal 20 generates notification information and sends the generated notification information to base station 10 (S211). The notification information generated in S211 includes floor information, but does not need to include location information or camera image information. The flow then returns to S201.

[0149] If the determined state is not the elevator boarding state (NO in S209), terminal 20 determines whether the determined state is the elevator disembarking state (S212).

[0150] If the determined state is the elevator exit state (YES in S212), terminal 20 determines whether or not the notification timing has arrived (S213).

[0151] If the notification timing has not yet arrived (NO in S213), the flow returns to S201.

[0152] When the notification timing arrives (YES in S213), terminal 20 generates notification information and sends the generated notification information to base station 10 (S214). The notification information generated in S214 includes floor information, but does not need to include location information or camera image information. The flow then returns to S201.

[0153] If the determined state is not the elevator exit state (NO in S212), in other words, if the determined state is the elevator door closed state, terminal 20 does not send notification information, and the flow returns to S201.

[0154] Next, an example of the processing flow of the base station 10 will be described. Figure 16 is a flowchart of the processing flow of the base station 10 in this embodiment. For example, the flowchart shown in Figure 16 may start when the base station 10 establishes a wireless connection with the terminal 20, or it may start when the robot 20x equipped with the terminal 20 that establishes a wireless connection with the base station 10 starts operating. Also, the flowchart shown in Figure 16 may end when the wireless connection between the base station 10 and the terminal 20 is lost, or it may end when the robot 20x equipped with the terminal 20 that establishes a wireless connection with the base station 10 stops operating. Alternatively, the start and / or end of the flowchart may be controlled by user operation.

[0155] The base station 10 determines whether or not it has received notification information from the terminal 20 (S101).

[0156] If notification information is received (YES in S101), the base station 10 updates the state of the robot 20x (S102). For example, the base station 10 determines and updates the state of the robot 20x based on the correspondence between the information contained in the notification information and the state of the robot 20x (for example, Figure 14).

[0157] If no notification information is received (NO in S101), the base station 10 performs the process in S103 without updating the status of the robot 20x.

[0158] The base station 10 determines whether the robot 20x is outside the elevator (S103).

[0159] If the robot 20x is in an outside elevator state (YES in S103), the base station 10 generates display information to be shown to the monitoring server 30 in the outside elevator state and sends the generated display information to the monitoring server 30 (S104). The display information generated in S104 may include state information indicating that the robot 20x is in an outside elevator state. The flow then returns to S101.

[0160] If the state of robot 20x is not outside the elevator (NO in S103), the base station 10 determines whether the state of robot 20x is on an intermediate floor (S105).

[0161] If the robot 20x is in an intermediate floor state (YES in S105), the base station 10 determines whether the number of people getting on and off the elevator is greater than a predetermined number (S106).

[0162] If the number of passengers getting on and off the elevator is not greater than a predetermined number (NO in S106), the base station 10 generates display information to be shown to the monitoring server 30 when the number of passengers getting on and off is small while the elevator is on an intermediate floor, and transmits the generated display information to the monitoring server 30 (S107). The display information generated in S107 may include status information indicating that the robot 20x is on an intermediate floor and the number of passengers getting on and off is small. The flow then returns to S101.

[0163] If the number of passengers getting on or off the elevator is greater than a predetermined number (YES in S106), the base station 10 generates display information to be shown to the monitoring server 30 when there are many passengers getting on or off at an intermediate floor, and transmits the generated display information to the monitoring server 30 (S108). The display information generated in S108 may include status information indicating that the robot 20x is at an intermediate floor and that there are many passengers getting on or off. The flow then returns to S101.

[0164] If the robot 20x is not in an intermediate floor state (NO in S105), the base station 10 determines whether the robot 20x is in an elevator riding state (S109).

[0165] If the robot 20x is in the elevator riding state (YES in S109), the base station 10 generates display information to be displayed on the monitoring server 30 in the elevator riding state and transmits the generated display information to the monitoring server 30 (S110). The display information generated in S110 may include state information indicating that the robot 20x is in the elevator riding state. The flow then returns to S101.

[0166] If the robot 20x is not in the elevator boarding state (NO in S109), the base station 10 determines whether the robot 20x is in the elevator disembarking state (S111).

[0167] If the robot 20x is in the elevator disembarkation state (YES in S111), the base station 10 generates display information to be shown to the monitoring server 30 in the elevator disembarkation state and transmits the generated display information to the monitoring server 30 (S112). The display information generated in S112 may include state information indicating that the robot 20x is in the elevator disembarkation state. The flow then returns to S101.

[0168] If the robot 20x is not in the elevator exit state (NO in S111), in other words, if the robot 20x is in the elevator door closed state, the base station 10 does not transmit display information, and the flow returns to S101.

[0169] Next, an example of the processing flow of the monitoring server 30 will be described. Figure 17 is a flowchart of the processing flow of the monitoring server 30 in this embodiment. For example, the flowchart shown in Figure 17 may start when the monitoring server 30 is started, or when display information is acquired from the base station 10, or when the robot 20x equipped with a terminal 20 that is wirelessly connected to the base station 10 starts operating. Also, the flowchart shown in Figure 17 may end when the monitoring server 30 stops, or when no display information is acquired from the base station 10 for a certain period of time, or when the robot 20x equipped with a terminal 20 that is wirelessly connected to the base station 10 stops operating. Alternatively, the start and / or end of the flowchart may be controlled by user operation.

[0170] The monitoring server 30 determines whether or not it has received status information of the robot 20x from the base station 10 (S301).

[0171] If status information is received (YES in S301), the monitoring server 30 updates the status of robot 20x to the status indicated by the status information (S302). Alternatively, the monitoring server 30 may determine the status based on information contained in the display information received from the base station 10 instead of the status information. For example, the monitoring server 30 may determine and update the status of robot 20x based on the correspondence between the information contained in the display information and the status of robot 20x (for example, Figure 14).

[0172] If status information is not received (NO in S301), the monitoring server 30 performs the process in S303 without updating the status of robot 20x.

[0173] The monitoring server 30 determines whether the robot 20x is outside the elevator (S303).

[0174] If the robot 20x is outside the elevator (YES in S303), the monitoring server 30 determines whether or not it has received display information from the base station 10 (S304).

[0175] If no display information has been received (NO in S304), the flow returns to S301.

[0176] If display information is received (YES in S304), the monitoring server 30 updates the display on the display unit 313 based on the display information (S305). The display information displayed in S305 may include floor information, location information, and camera image information. The flow then returns to S301.

[0177] If the robot 20x is not outside the elevator (NO in S303), the monitoring server 30 determines whether the robot 20x is on an intermediate floor (S306).

[0178] If the robot 20x is in an intermediate floor state (YES in S306), the monitoring server 30 determines whether the number of people getting on and off the elevator is greater than a predetermined number (S307).

[0179] If the number of people getting on and off the elevator is not greater than a predetermined number (NO in S307), the monitoring server 30 determines whether or not it has received display information from the base station 10 (S308).

[0180] If no display information has been received (NO in S308), the flow returns to S301.

[0181] If display information is received (YES in S308), the monitoring server 30 updates the display on the display unit 313 based on the display information (S309). The display information displayed in S309 includes location information but does not need to include camera image information. The flow then returns to S301.

[0182] If the number of people getting on and off the elevator is greater than a predetermined number (YES in S307), the monitoring server 30 determines whether or not it has received display information from the base station 10 (S310).

[0183] If no display information has been received (NO in S310), the flow returns to S301.

[0184] If display information is received (YES in S310), the monitoring server 30 updates the display on the display unit 313 based on the display information (S311). The display information displayed in S311 may include location information and camera image information. The flow then returns to S301.

[0185] If the robot 20x is not in an intermediate floor state (NO in S306), the monitoring server 30 determines whether the robot 20x is in an elevator boarding state (S312).

[0186] If the robot 20x is in the elevator riding state (YES in S312), the monitoring server 30 determines whether or not it has received display information from the base station 10 (S313).

[0187] If no display information has been received (NO in S313), the flow returns to S301.

[0188] If display information is received (YES in S313), the monitoring server 30 updates the display on the display unit 313 based on the display information (S314). The display information displayed in S314 may include location information but may not include camera image information. The flow then returns to S301.

[0189] If the robot 20x is not in the elevator boarding state (NO in S312), the monitoring server 30 determines whether the robot 20x is in the elevator disembarking state (S315).

[0190] If the robot 20x is in the elevator disembarking state (YES in S315), the monitoring server 30 determines whether or not it has received display information from the base station 10 (S316).

[0191] If no display information has been received (NO in S316), the flow returns to S301.

[0192] If display information is received (YES in S316), the monitoring server 30 updates the display on the display unit 313 based on the display information (S317). The display information displayed in S317 may include location information but may not include camera image information. The flow then returns to S301.

[0193] If the robot 20x is not in the elevator disembarking state (NO in S315), in other words, if the robot 20x is in the elevator door closed state, the monitoring server 30 does not update the display unit 313, and the flow returns to S301.

[0194] <Example 5> In the examples of the embodiments described above, we explained systems in which a robot monitors its surroundings based on satellite positioning, positioning using cameras such as TOF cameras, and positioning through indoor communication. In these positioning technologies, the accuracy of positioning may deteriorate depending on the positioning method, the surrounding environment of the object being positioned, etc.

[0195] For example, with satellite positioning, the accuracy of positioning may deteriorate indoors where radio waves from satellites are difficult to receive.

[0196] Furthermore, for example, in positioning via indoor communication, the positioning accuracy may deteriorate due to significant changes in the radio wave environment surrounding the target device.

[0197] Furthermore, in positioning using cameras such as TOF cameras, the position of the object to be positioned (e.g., robot 20x) is detected by comparing a camera image taken at a certain location by a camera mounted on the object to be positioned with a reference image (which may be called a map) that has been previously taken and registered at that location. Therefore, if the camera image changes significantly, the positioning accuracy may deteriorate.

[0198] For example, in a case where the camera mounted on robot 20x captures an area that includes the elevator door, the camera image may change significantly when the door opens or closes, potentially degrading positioning accuracy. The case where the camera mounted on robot 20x captures an area that includes the elevator door is, for example, a case where robot 20x's positioning is performed near the elevator.

[0199] Therefore, the following section will explain, as an example, the location information that robot 20x notifies when the camera mounted on robot 20x is in a state where it can capture a range including the elevator door.

[0200] Figure 18 shows an example of notification information in the fifth example. Figure 18 shows examples of notification information for the following states: outside the elevator, inside the elevator, with the elevator doors closed, and after exiting the elevator. In Figure 18, the horizontal axis represents the time axis.

[0201] In the fifth example, the "elevator boarding state" may refer to the state from when the elevator carrying robot 20x begins to open its doors until robot 20x has boarded and the elevator has finished closing its doors. Similarly, the "elevator disembarking state" may refer to the state from when the elevator arrives at the floor where robot 20x is disembarking and begins to open its doors until robot 20x disembarks from the elevator. In the fifth example, the "elevator boarding state" and the "elevator disembarking state" may correspond to the state where the elevator doors are open, or to the state where the elevator doors are recognized as open within the camera's field of view of robot 20x.

[0202] As shown in Figure 18, in the fifth example, when outside the elevator, robot 20x sends notification information. Robot 20x also sends notification information when inside and outside the elevator. On the other hand, when the elevator doors are closed, robot 20x does not send notification information.

[0203] In the fifth example, the notification information provided when outside the elevator differs from the notification information provided when inside the elevator or when exiting the elevator.

[0204] The notification information format A, which is notified when the elevator is outside as shown in Figure 18, includes a "Terminal ID" field, a "Floor Information" field, an "Identification Information" field, a "Location Information" field, and a "Camera Image Information" field, similar to the example shown in Figure 5, etc. The location information included in the "Location Information" field of format A indicates the location detected by robot 20x.

[0205] The notification information format C, shown in Figure 18 for elevator boarding and elevator disembarking states, includes the same fields as format A: "Terminal ID," "Floor Information," "Identification Information," and "Camera Image Information." The "Elevator Location Information" field included in format C contains elevator location information indicating the location of the elevator that robot 20x is riding in. In other words, the location information in format A is replaced with elevator location information in format C.

[0206] The elevator location information may be included in the information of the floor to which the robot 20x moves. In other words, the elevator location information is known in advance and may be stored in the robot 20x. The elevator location information may also indicate the position of a single point (hereinafter referred to as the representative point) that represents the area of ​​the elevator. The elevator location information may also be coordinate information indicating the position of the representative point of the elevator. For example, the position of the representative point of the elevator may be the center of the area of ​​the elevator floor, or it may be any other position. Furthermore, since the elevator's position is also known to the monitoring server 30, the elevator location information may also be information (for example, a 1-bit flag) indicating that the position detected by the robot 20x should be replaced with the elevator's position.

[0207] Note that while Figure 18 shows an example of format C that includes the "camera image information" field, the "camera image information" field does not necessarily have to be included in format C.

[0208] In the fifth example described above, when the robot 20x is on or off the elevator, instead of notifying the elevator's position, the robot 20x notifies the elevator's position. This avoids the need to notify the degraded positioning results in situations where positioning accuracy deteriorates, thus preventing degradation of the notified position information. Therefore, the degradation of monitoring quality can be suppressed, and monitoring can be performed efficiently.

[0209] In the fifth example described above, the notification information provided in the elevator boarding state and the elevator alighting state was shown to be the same as in the state outside the elevator, but this disclosure is not limited to this. Below, another example of notification information provided in the elevator boarding state and the elevator alighting state will be described.

[0210] <Example 6> Figure 19 shows an example of notification information in the sixth example. Similar to Figure 18, Figure 19 shows examples of notification information for the following states: outside the elevator, inside the elevator, with the elevator doors closed, and after exiting the elevator. In Figure 19, the horizontal axis represents the time axis.

[0211] The difference between the fifth example and the sixth example is that, as shown in Figure 18, in the fifth example, the same notification information is sent when the user is disembarking from the elevator as when the user is boarding the elevator, whereas in the sixth example, different notification information is sent when the user is disembarking from the elevator compared to when the user is boarding the elevator.

[0212] In Figure 19, the same notification information is sent when the user is outside the elevator as when the user is outside the elevator. For example, format A of the notification information sent when the user is outside the elevator includes a "Terminal ID" field, a "Floor Information" field, an "Identification Information" field, a "Location Information" field, and a "Camera Image Information" field. The location information included in the "Location Information" field of format A indicates the location detected by robot 20x.

[0213] In the sixth example described above, when the robot is riding in an elevator, the elevator's position is reported instead of the position detected by the robot 20x. This notification avoids the need to report degraded positioning results in situations where positioning accuracy deteriorates, thus preventing degradation of the reported position information. Therefore, the degradation of monitoring quality can be suppressed, and monitoring can be performed efficiently.

[0214] Furthermore, when the elevator is disembarked, the positioning accuracy does not deteriorate as much when the elevator doors are open. Therefore, in the sixth example described above, when the elevator is disembarked, the position detected by robot 20x is notified. This allows positioning to start quickly from the elevator disembarked state, thus reducing positioning delay.

[0215] <Example 7> The fifth and sixth examples described above illustrate cases where elevator location information indicates the location of a representative point of the elevator. The seventh example describes variations in the elevator location information to be notified.

[0216] Figure 20 shows an example of elevator positions. Figure 20 shows three examples of elevator positions, (a) to (c).

[0217] Figure 20(a) shows a case where no person and / or other moving object is in the elevator. In this case, robot 20x may enter the elevator and move to the back of the elevator as viewed from outside the elevator. Therefore, in this case, the position at the back of the elevator as viewed from outside the elevator, on the elevator floor, is indicated by the elevator position information.

[0218] Figure 20(b) shows a case where a person and / or other moving object is in the elevator, but the inside of the elevator is relatively empty. In this case, the robot 20x may ride in the elevator and move to near the center of the elevator as seen from outside the elevator. Therefore, in this case, the position of the center of the elevator on the elevator floor is indicated by the elevator position information.

[0219] Figure 20(c) shows a case where a person and / or other moving object is in the elevator and the inside of the elevator is relatively crowded. In this case, robot 20x may enter the elevator and move to the front of the elevator as seen from outside the elevator. Therefore, in this case, the position on the elevator floor that is the front of the elevator as seen from outside the elevator is indicated by the elevator position information.

[0220] The multiple elevator positions described above may be associated with the congestion status and stored in the robot 20x. Alternatively, each of the multiple elevator positions may be associated with an index, and the correspondence between the congestion status, the elevator position, and the index may be stored in the robot 20x and the monitoring server 30.

[0221] Furthermore, when the elevator doors are open, or when robot 20x is inside the elevator, etc., robot 20x may detect the congestion level inside the elevator and select elevator position information based on the detected congestion level and the stored correspondence. In the example in Figure 20, if robot 20 detects that there are no people and / or other moving objects inside the elevator, it selects elevator position information that indicates the position at the back of the elevator as seen from outside the elevator, as illustrated in Figure 20(a). Similarly, if robot 20 detects that there are people and / or other moving objects inside the elevator and it is relatively crowded, it may select elevator position information that indicates the position at the front of the elevator as seen from outside the elevator, as illustrated in Figure 20(c).

[0222] The method for detecting the congestion level inside the elevator is not particularly limited. For example, robot 20x may perform image analysis on images it has captured to detect the number of people and / or moving objects. Alternatively, robot 20x may acquire sensing information about people and / or moving objects in the elevator from sensors mounted on the elevator and detect the congestion level based on the sensing information. For example, the sensing information may include at least one of the presence or absence of people in the elevator obtained from a camera and / or motion sensor inside the elevator, the temperature inside the elevator obtained from a temperature sensor, and the weight of objects inside the elevator obtained from a weight sensor.

[0223] Furthermore, while Figure 20 shows three possible elevator position options, this disclosure is not limited thereto. There may be two or more selectable elevator positions. The selectable elevator positions may be determined according to the size and shape of the elevator, the size and shape of the robot 20x, and the floor to be disembarked from.

[0224] Figure 21 shows an example of notification information in the seventh example. Similar to Figure 19, Figure 21 shows examples of notification information for the following states: outside the elevator, inside the elevator, with the elevator doors closed, and after exiting the elevator. In Figure 21, the horizontal axis represents the time axis.

[0225] The difference between the 7th example and the 6th example is that in the 6th example, as shown in Figure 19, elevator position information indicating the location of a representative point of the elevator is notified when the user is riding in the elevator, whereas in the 7th example, elevator position information indicating the location of the elevator selected by the robot 20x is notified when the user is riding in the elevator.

[0226] The notification information format D, displayed when the user is riding in the elevator as shown in Figure 21, includes the same fields as format C: "Terminal ID," "Floor Information," "Identification Information," "Elevator Location Information," and "Camera Image Information." However, the "Elevator Location Information" field in format D includes elevator location information indicating the location of the elevator selected by robot 20x. In other words, while the elevator location information in format C indicates the location of a representative point of the elevator, the elevator location information in format D indicates the location of the elevator selected by robot 20x.

[0227] In the seventh example described above, when the robot is riding in an elevator, the elevator's position is reported instead of the position detected by the robot 20x. This notification avoids the need to report degraded positioning results in situations where positioning accuracy deteriorates, thus preventing degradation of the reported position information. Therefore, the deterioration of monitoring quality can be suppressed, and monitoring can be performed efficiently.

[0228] In the seventh example described above, the elevator's location is selected from multiple patterns based on the congestion level. Therefore, when notifying the elevator's location instead of the robot 20x's location, a more preferable location can be notified as the robot 20x's location. By notifying the elevator's location selected based on the congestion level, the receiving base station and / or monitoring server can simultaneously understand the elevator's congestion level in addition to the robot 20x's location. For example, knowing the elevator's congestion level makes it possible to guide the robot 20x along a less crowded route in places like train stations. Furthermore, in cases where a specific infectious disease (e.g., COVID-19) is prevalent in crowded places (e.g., when there are many infected individuals), understanding the elevator's congestion level makes it possible to determine the likelihood of infection for people inside the elevator (e.g., the risk of infection).

[0229] In the seventh example described above, while the robot 20x notified the location of the selected elevator instead of the location it detected when boarding the elevator, the robot could also notify the location of the selected elevator when disembarking from the elevator, similar to the elevator boarding state.

[0230] Furthermore, in the seventh example described above, we showed an example where, while the robot 20x is in the elevator, it notifies the selected elevator's position instead of the position it detected. However, before the elevator doors open, the position of a representative point of the elevator may also be notified.

[0231] Furthermore, in the seventh example described above, the location of the elevator's representative point and the location of the selected elevator may be switched between being notified. For example, before the congestion status inside the elevator (e.g., presence or absence of people / moving objects, and their number, etc.) is detected, the location of the elevator's representative point may be notified, and after the congestion status is detected, the location of the elevator selected according to the congestion status may be notified. Alternatively, before the congestion status inside the elevator is detected, location information (e.g., elevator location information) may not be notified, and only the location of the elevator's representative point may be notified, and after the congestion status is detected, the location of the elevator selected according to the congestion status may be notified. Note that the stage before the congestion status inside the elevator is detected may include, for example, the stage before the elevator doors open, or the stage after the elevator doors open but before the detection results are obtained (e.g., before image analysis is completed).

[0232] <Example 8> In the eighth example, similar to the fourth example described above, we will explain an example in which, when the robot 20x notifies notification information in an "intermediate floor state," the notification information includes elevator location information.

[0233] Figure 22 shows an example of notification information in the eighth example. Similar to Figures 8 and 9, Figure 22 shows examples of notification information for each of the following states: outside the elevator, inside the elevator, with the elevator doors closed, on an intermediate floor, and after exiting the elevator. In Figure 22, the horizontal axis represents the time axis.

[0234] As shown in Figure 22, in the eighth example, the robot 20x notifies notification information when the elevator is outside, inside, outside, and on an intermediate floor. On the other hand, the robot 20x does not notify notification information when the elevator doors are closed.

[0235] The format A of the notification information displayed in the elevator outside state and elevator disembark state shown in Figure 22 includes a "Terminal ID" field, a "Floor Information" field, an "Identification Information" field, a "Location Information" field, and a "Camera Image Information" field, similar to the example shown in Figure 5, etc. The location information included in the "Location Information" field of format A indicates the location detected by robot 20x.

[0236] The format D of the notification information displayed in Figure 22 for the elevator boarding state and intermediate floor states includes, similar to the example shown in Figure 21, a "Terminal ID" field, a "Floor Information" field, an "Identification Information" field, an "Elevator Location Information" field, and a "Camera Image Information" field. However, the "Elevator Location Information" field included in format D includes elevator location information indicating the location of the elevator that robot 20x has selected and is boarding.

[0237] In the eighth example described above, during an intermediate floor state, elevator location information indicating the location of the elevator that robot 20x is riding in, selected by robot 20x, is notified. This notification avoids the need to notify the robot 20x of degraded positioning results even if it temporarily disembarks from the elevator on an intermediate floor, thus preventing degradation of the notified location information. Therefore, degradation of monitoring quality can be suppressed, and monitoring can be performed efficiently.

[0238] In the eighth example, elevator location information was shown to be notified even when the elevator is on an intermediate floor. However, even when the elevator is on an intermediate floor, it is not necessary to notify the elevator location information depending on the situation. For example, even when the elevator is on an intermediate floor, if the number of passengers getting on or off is relatively small, the robot 20x does not need to temporarily get off the elevator. Therefore, even when the elevator is on an intermediate floor, elevator location information may be notified when the number of passengers getting on or off is relatively large, and not when the number of passengers is relatively small.

[0239] Examples 5 through 8 above illustrate how location information included in notification information can be replaced with elevator location information depending on the state. Below, we will explain the correspondence between the robot's state, the content of the notification information, and the content displayed on the monitoring server, based on these examples.

[0240] Figure 23 shows another example of the correspondence between the robot's state, notification information, and display content. Figure 23 shows the content of the notification information and the display content corresponding to each state: "Outside the elevator," "Elevator doors closed," "Elevator on board," "Elevator off the elevator," "Intermediate floor (few boarding / alighting)," and "Intermediate floor (frequent boarding / alighting)." Note that explanations for the same information as in Figure 14 are omitted in Figure 23.

[0241] The "Elevator Occupancy State" and "Intermediate Floor State (Frequent Boarding / Alighting)" in Figure 23 differ from the "Elevator Occupancy State" and "Intermediate Floor State (Frequent Boarding / Alighting)" in Figure 14. These two states will be explained below.

[0242] In the "Elevator Riding Status" state, floor information, location information, and camera image information are notified. However, the location information notified in the "Elevator Riding Status" state is not the location detected by robot 20x, but rather the location of the elevator that robot 20x is riding in (the "Elv Location" in Figure 23). In the "Elevator Riding Status" state, the monitoring server displays location information indicating that the robot is inside the elevator, and an additional message indicating "Currently riding in the elevator" is displayed.

[0243] "Intermediate Floor State (High Number of Boarding / Alighting)" corresponds to a state where robot 20x is in an intermediate floor state and the number of people boarding or alighting exceeds a predetermined number. In the "Intermediate Floor State (High Number of Boarding / Alighting)" state, floor information, location information, and camera image information are notified. However, the location information notified in the "Elevator Boarding State" state is not the location detected by robot 20x, but rather the location of the elevator that robot 20x is boarding (the "Elv Location" in Figure 23). In the "Intermediate Floor State (High Number of Boarding / Alighting)" state, the monitoring server displays location information and camera images indicating that the robot is inside the elevator, and an additional message indicating that "the robot is stopped on floor X (the floor where it is currently stopped)" is displayed. In this state, an additional message indicating whether or not the robot is moving outside the elevator may also be displayed. Furthermore, if the number of people boarding or alighting exceeds a predetermined number, an additional message indicating that the number of people boarding or alighting exceeds a predetermined number may also be displayed. In this case, the number of people boarding and alighting may be displayed separately. For example, additional messages such as "High Number of Boarders," "High Number of Alighters," or "High Number of Boarding / Alighting" may be displayed.

[0244] In the examples 5 through 8 described above, the robot 20x primarily demonstrated an example where the elevator's location was reported instead of the location detected by the robot 20x when the elevator doors were closed. However, this disclosure is not limited to these examples. For instance, even if the elevator doors are closed while the robot 20x is waiting to board the elevator, the robot 20x may, depending on the situation, report the elevator's location instead of the location detected by the robot 20x. For example, if there are many people and / or moving objects waiting to board the elevator, the positioning accuracy may deteriorate, regardless of whether satellite positioning, camera positioning, or communication positioning is used. For example, in the case of camera positioning, the camera image may contain many people and / or moving objects that are not included in the reference image, potentially degrading the positioning accuracy. Therefore, if there are many people and / or moving objects waiting to board the elevator, the robot 20x may report the elevator's location instead of the location detected by the robot 20x. In this case, the robot 20x may also report the camera image along with the elevator's location.

[0245] Furthermore, while the above-mentioned examples 6 to 8 mainly show examples where the robot 20x notifies the position it detected when the elevator doors are open as the robot 20x disembarks, this disclosure is not limited to these examples. When the robot 20x disembarks from an elevator, even if the elevator doors are open, it may, depending on the situation, notify the elevator's position instead of the position it detected. For example, if there are many people and / or moving objects waiting to board the elevator, the positioning accuracy may deteriorate, whether using satellite positioning, camera positioning, or communication positioning. For example, in the case of camera positioning, the positioning accuracy may deteriorate because many people and / or moving objects not included in the reference image may be included in the camera image. Therefore, if there are many people and / or moving objects waiting to board the elevator, when the robot 20x disembarks, even if the elevator doors are open, it may notify the elevator's position instead of the position it detected. In this case, robot 20x may also start positioning using the camera after robot 20x has disembarked and the person and / or moving object waiting to board the elevator has moved out of the camera's field of view. The method for determining whether or not the person and / or moving object waiting to board the elevator has moved out of the camera's field of view is not particularly limited, but for example, it may be determined based on the results of image analysis of the captured camera image.

[0246] Furthermore, for example, robot 20x may notify the elevator's location instead of the location it detected, depending on the camera's field of view. For example, when robot 20x is getting off, if the elevator doors are open, and the camera's field of view of robot 20x includes the doors of another elevator, and those doors are open, robot 20x may notify the elevator's location instead of the location it detected. In this case, robot 20x may also start positioning using its camera if the camera's field of view includes the doors of another elevator, and those doors close. Alternatively, robot 20x may start positioning using its camera if the camera's field of view no longer includes the doors of another elevator. The camera's field of view of robot 20x no longer includes the door of an elevator other than the one robot 20x was riding in, for example, when robot 20x moves more than a predetermined distance away from the elevator, or when the orientation of robot 20x's camera changes.

[0247] Furthermore, when the robot 20x disembarks from the elevator, if it notifies the elevator's location instead of the location detected by the robot 20x, it may also notify the camera image along with the elevator's location.

[0248] <Variations of the processing flow in this embodiment> Next, we will explain variations in the processing flow based on the 5th to 8th examples described above. In the following, we will illustrate, as shown in the 8th example, how the robot 20x notifies notification information including elevator location information in the "elevator occupancy state" and the "intermediate floor state" when there are many passengers getting on or off. As with the above explanation, the robot 20x may be replaced with the terminal 20 in the following explanation as well.

[0249] In the variation shown in the eighth example, the location information included in the notification information sent from terminal 20 is changed. The processing excluding the processing related to this change in location information may be the same as the processing flow explained using Figures 15 to 17. For example, the processing of base station 10 and monitoring server 30 may be the same as in Figures 16 and 17, respectively. Below, we will explain the differences between the processing of terminal 20 based on the eighth example and Figure 15.

[0250] Figure 24 is a flowchart showing variations in the processing flow of terminal 20 in this embodiment. For example, the flowchart shown in Figure 24 may start when terminal 20 is powered on, or when the robot 20x on which terminal 20 is mounted starts operating. Also, the flowchart shown in Figure 24 may end when the operation of terminal 20 stops, or when the operation of the robot 20x on which terminal 20 is mounted stops. Alternatively, the start and / or end of the flowchart may be controlled by user operation. Note that in Figure 24, the same reference numerals are used for processes similar to those in Figure 15, and their explanations may be omitted.

[0251] As shown in Figure 24, if the determined state is an intermediate floor state (YES in S205), terminal 20 determines whether or not the notification timing has arrived (S207).

[0252] If the notification timing has not yet arrived (NO in S207), the flow returns to S201.

[0253] When the notification timing arrives (YES in S207), terminal 20 generates notification information and sends the generated notification information to base station 10 (S401). The processing in S401 will be described later. After processing in S401, the flow returns to S201.

[0254] If the determined state is elevator boarding state (YES in S209), terminal 20 determines whether or not the notification timing has arrived (S210).

[0255] If the notification timing has not yet arrived (NO in S210), the flow returns to S201.

[0256] When the notification timing arrives (YES in S210), terminal 20 generates notification information and sends the generated notification information to base station 10 (S402). The processing in S402 will be described later. After the processing in S402, the flow returns to S201.

[0257] Next, we will explain the processes in S402 and S401.

[0258] Figure 25 is a flowchart showing the processing flow for S402 in Figure 24. The flowchart shown in Figure 25 may be started if the answer to S210 in Figure 24 is YES. As shown in Figure 24, the processing in S402 corresponds to the processing when it is determined that the elevator is occupied.

[0259] Terminal 20 determines whether there are any other people and / or moving objects inside the elevator besides Terminal 20 (robot 20x) (S501).

[0260] If there are no other people and / or moving objects inside the elevator besides terminal 20 (robot 20x) (NO in S501), for example, if the inside of the elevator is empty, terminal 20 selects the position at the back of the elevator as the position of terminal 20 (robot 20x) (S502). In other words, in this case, terminal 20 selects elevator position information indicating the position at the back of the elevator instead of position information indicating the position of terminal 20. The flow then proceeds to S506.

[0261] If there are other people and / or moving objects inside the elevator besides terminal 20 (robot 20x) (YES in S501), terminal 20 determines whether the inside of the elevator is a crowded environment or not (S503). For example, terminal 20 may determine that the inside of the elevator is a crowded environment if there are a predetermined number or more people and / or moving objects besides terminal 20 (robot 20x), and that the inside of the elevator is not a crowded environment if there are no predetermined number or more people and / or moving objects besides terminal 20 (robot 20x).

[0262] If the elevator interior is not a crowded environment (NO in S503), terminal 20 selects the center of the elevator as the location of terminal 20 (robot 20x) (S504). In other words, in this case, terminal 20 selects elevator location information indicating the center of the elevator instead of location information indicating the location of terminal 20. The flow then proceeds to S506.

[0263] If the elevator interior is crowded (YES in S503), terminal 20 selects the position in front of the elevator as the location of terminal 20 (robot 20x) (S505). In other words, in this case, terminal 20 selects elevator location information indicating the position in front of the elevator instead of location information indicating the location of terminal 20. The flow then proceeds to S506.

[0264] Terminal 20 generates information other than location information and sends notification information including the generated information and the elevator location information selected in processing S502, S504, or S505 (S506). Then the flow shown in Figure 25 is completed, and processing S201 in Figure 24 may be executed.

[0265] FIG. 26 is a flowchart showing the process flow of S401 in FIG. 24. In FIG. 26, for the same processes as in FIG. 25, the same reference numerals may be assigned and the description may be omitted. The flowchart shown in FIG. 26 may start when YES in S207 of FIG. 24. As shown in FIG. 24, the process of S401 corresponds to the process when it is determined to be in the intermediate floor state.

[0266] The terminal 20 determines whether the number of passengers getting on and off the elevator on the intermediate floor is less than a predetermined number (S601). The method for estimating the number of passengers getting on and off is not particularly limited, and the number of passengers getting on and off may be estimated using image processing of camera images (for example, face authentication processing), or may be estimated using sensing technologies such as infrared sensors and weight sensors.

[0267] If the number of passengers getting on and off is not less than the predetermined number (NO in S601), for example, if the number of passengers getting on and off is greater than or equal to the predetermined number, the flow shifts to S501. The case where the number of passengers getting on and off is not less than the predetermined number is the case where the robot 20x gets off temporarily. In this case, elevator position information may be notified as the position information.

[0268] If the number of passengers getting on and off is less than the predetermined number (YES in S601), the terminal 20 generates and transmits notification information that does not include position information (S602). In this case, the terminal 20 may generate notification information that does not include image information and position information. Then, the flow shown in FIG. 26 ends, and the process of S201 in FIG. 24 is executed.

[0269] Through the process flow described above, the notification information of the variation shown in the eighth example is notified from the terminal 20.

[0270] <Examples of each configuration> Examples of each configuration of the communication system according to the above-described embodiment will now be explained. The communication system according to this embodiment includes a base station 10 shown in Figure 27, a terminal 20 shown in Figure 28, and a monitoring server 30 shown in Figure 29. The wireless communication method (wireless communication system) according to this embodiment is not particularly limited. For example, the wireless communication method according to this embodiment may conform to the 5th generation mobile communication system (5G) standard, or it may conform to other wireless communication standards.

[0271] The configurations shown below are examples only, and this disclosure is not limited thereto. Some of the configurations shown in Figures 27 to 29 may be omitted, and configurations not shown in each figure may be added.

[0272] <Configuration of base station 10> Figure 27 is a block diagram showing an example configuration of a base station 10 according to this embodiment. The base station 10 includes a receiving unit 101, a receiving control unit 102, a status determination unit 103, a display information generation unit 104, a server communication unit 105, a transmission control unit 106, and a transmission unit 107. The receiving control unit 102, the status determination unit 103, the display information generation unit 104, and the transmission control unit 106 may be included in the control unit 108 of the base station 10.

[0273] The receiving unit 101 receives an uplink (UL) signal destined for the base station 1010 based on timing control in the receiving control unit 102. The receiving unit 101 outputs the UL signal to the receiving control unit 102. The UL signal includes, for example, notification information from the terminal 20.

[0274] The receiving control unit 102 controls the reception timing in the receiving unit 101. The receiving control unit 102 also performs predetermined reception processing on the UL signal. For example, predetermined reception processing includes frequency conversion processing (downconversion). Predetermined reception processing may also include baseband signal processing such as demodulation processing and decoding processing.

[0275] The reception control unit 102 extracts received data and / or notification information from the UL signal that has undergone predetermined reception processing. The reception control unit 102 outputs the received data to the data processing unit (omitted in Figure 14) which performs signal processing at a higher layer. If the reception control unit 102 has extracted notification information from terminal 20, it outputs the extracted notification information to the status determination unit 103.

[0276] The state determination unit 103 determines the state of the robot 20x on which the terminal 20 is mounted, based on location information and other data included in the notification information of the terminal 20. The state of the robot 20x determined here is based on the position of the robot 20x and the behavior of the elevator used by the robot 20x (for example, opening and closing of the doors).

[0277] The display information generation unit 104 determines the content to be displayed based on the robot 20x state determined by the state determination unit 103, and generates display information that includes the determined content.

[0278] The server communication unit 105 is a communication interface with the monitoring server 30. The server communication unit 105 transmits and receives signals with the monitoring server 30. For example, the server communication unit 105 transmits display information generated by the display information generation unit 104 to the monitoring server 30.

[0279] The base station 10 may be directly connected to the monitoring server 30 by wired or wireless connection. Alternatively, the base station 10 may be connected to the monitoring server 30 via a network.

[0280] The transmission control unit 106 controls the transmission timing of the transmission data acquired from the data processing unit (omitted in Figure 27) and / or the control information to be transmitted to the terminal 20. The transmission control unit 106 also performs a predetermined transmission process on at least one of the transmission data and the control information to generate a DL signal. The predetermined transmission process may include baseband signal processing such as encoding and modulation.

[0281] The transmitting unit 107 transmits a DL signal to the terminal 20 based on the transmission timing control in the transmission control unit 106.

[0282] <Configuration of Terminal 20> Figure 28 is a block diagram showing an example configuration of terminal 20 according to this embodiment. Terminal 20 comprises a wireless receiving unit 201, a receiving control unit 202, a satellite signal receiving unit 203, an indoor positioning signal receiving unit 204, a position detection unit 205, a state determination unit 206, a notification information generation unit 207, a transmission control unit 208, and a wireless transmission unit 209. The receiving control unit 202, the position detection unit 205, the state determination unit 206, the notification information generation unit 207, and the transmission control unit 208 may be included in the control unit 210 of terminal 20.

[0283] The wireless receiver 201 receives a DL signal destined for terminal 20 based on timing control in the receiver control unit 202. The wireless receiver 201 outputs the DL signal destined for terminal 20 to the receiver control unit 202.

[0284] The reception control unit 202 controls the reception timing in the wireless reception unit 201. The reception control unit 202 also performs predetermined reception processing on DL signals destined for terminal 20. For example, predetermined reception processing includes frequency conversion processing (downconversion). Predetermined reception processing may also include baseband signal processing such as demodulation and decoding.

[0285] The reception control unit 202 extracts warning information, received data, and / or control information transmitted by the base station 10 from the signal after it has undergone predetermined reception processing. The reception control unit 202 outputs the received data to the data processing unit (omitted in Figure 28), which performs signal processing at a higher layer.

[0286] The satellite signal receiving unit 203 receives positioning signals from satellites such as GPS satellites.

[0287] The indoor positioning signal receiving unit 204 receives indoor positioning signals from the indoor positioning AP40.

[0288] The position detection unit 205 detects the position of the terminal 20 based on at least one of the signals received by the satellite signal reception unit 203, the signals received by the indoor positioning signal reception unit 204, and the camera images (omitted in FIG. 28) obtained from the camera mounted on the robot 20x. Further, the position detection unit 205 may determine whether to notify, as the position of the terminal 20, the position of the elevator where boarding (or alighting) instead of the detected position of the terminal 20, based on the state determined by the state determination unit 206. Further, when the position detection unit 205 determines to notify the position of the elevator where boarding (or alighting) instead of the detected position of the terminal 20, it may output the position of one representative point of the elevator instead of the detected position of the terminal 20, or may output the position of the elevator selected from a plurality of candidates instead of the detected position of the terminal 20.

[0289] The state determination unit 206 determines the state of the robot 20x based on the information indicating the position of the terminal 20.

[0290] The notification information generation unit 207 determines whether to notify the notification information based on the state determined by the state determination unit 206, and determines the content (for example, format) of the notification information when notifying. When the notification information generation unit 207 notifies the notification information, it generates the notification information having the determined content.

[0291] The transmission control unit 208 controls the signal transmission timing in the wireless transmission unit 209. Further, the transmission control unit 208 controls the transmission timing of the transmission data acquired from the data processing unit (omitted in FIG. 28) and / or the control information or notification information transmitted to the base station 10. Further, the transmission control unit 208 performs a predetermined transmission process on at least one of the notification information, the transmission data, and the control information to generate a UL signal. The predetermined transmission process may include baseband signal processes such as encoding and modulation.

[0292] The wireless transmission unit 209 transmits a UL signal to the base station 10 based on the transmission timing control in the transmission control unit 208.

[0293] In the terminal 20 described above, the control unit 210 determines the state of the robot 20x (an example of a mobile body) from among several candidates representing the state of the robot 20x, based on the position of the robot 20x and the state of the elevator (an example of a transport mechanism) used by the robot 20x, and controls the transmission of notification information according to the state of the robot 20x. In addition, the wireless transmission unit 209 (an example of a transmission unit) transmits notification information to the monitoring server 30 (an example of an external device) that monitors the robot 20x, in accordance with the transmission control.

[0294] This configuration allows for the transmission of appropriate notification information according to the state of the robot 20x, thereby enabling proper control of the information notified by the robot 20x.

[0295] Furthermore, at least a portion of the configuration of the base station 10 described above may be included in the terminal 20. Alternatively, at least a portion of the configuration of the terminal 20 described above may be included in the base station 10.

[0296] <Configuration of monitoring server 30> Figure 29 is a block diagram showing an example configuration of the monitoring server 30 according to this embodiment. The monitoring server 30 comprises a receiving unit 301, a receiving control unit 302, an information classification unit 303, an elevator outside state display control unit 304, an elevator door closed state display control unit 305, an elevator boarding state display control unit 306, an elevator disembarking state display control unit 307, an intermediate floor state first display control unit 308, an intermediate floor state second display control unit 309, a transmission control unit 310, a transmission unit 311, and a display unit 313. The receiving control unit 302, the information classification unit 303, the elevator outside state display control unit 304, the elevator door closed state display control unit 305, the elevator boarding state display control unit 306, the elevator disembarking state display control unit 307, the intermediate floor state first display control unit 308, the intermediate floor state second display control unit 309, and the transmission control unit 310 may be included in the control unit 312 of the monitoring server 30.

[0297] The receiving unit 301 receives a signal from the base station 10 based on timing control in the receiving control unit 302. The receiving unit 301 outputs the received signal to the receiving control unit 302. The received signal includes, for example, display information.

[0298] The receiving control unit 302 controls the reception timing in the receiving unit 301. The receiving control unit 302 also performs predetermined reception processing on the received signal. For example, predetermined reception processing includes frequency conversion processing (downconversion). Predetermined reception processing may also include baseband signal processing such as demodulation processing and decoding processing.

[0299] The reception control unit 302 extracts received data and / or display information from the signal that has undergone predetermined reception processing. The reception control unit 302 outputs the received data to the data processing unit (omitted in Figure 29) which performs signal processing in a higher layer. If the reception control unit 302 has extracted display information, it outputs the extracted display information to the information classification unit 303.

[0300] The information classification unit 303 classifies the display information according to the state of the robot 20x. The information classification unit 303 outputs the classified information to the display control unit for each state of the robot 20x.

[0301] The elevator-outside-the-elevator-state display control unit 304 acquires display information when the robot 20x is in an elevator-outside-the-elevator state, and the display unit 313 controls the display according to the elevator-outside-the-elevator state.

[0302] The elevator door closed state display control unit 305 acquires display information when the robot 20x is in the elevator door closed state, and the display unit 313 controls the display according to the elevator door closed state.

[0303] The elevator occupancy status display control unit 306 acquires display information when the robot 20x is in the elevator occupancy state, and the display unit 313 controls the display according to the elevator occupancy state.

[0304] The elevator disembarkation status display control unit 307 acquires display information when the robot 20x is in the elevator disembarkation state, and the display unit 313 controls the display according to the elevator disembarkation state.

[0305] The intermediate floor status first display control unit 308 acquires display information when the robot 20x is in an intermediate floor state and the number of people getting on and off the elevator is greater than a predetermined number, and the display unit 313 controls the display according to this case.

[0306] The intermediate floor status second display control unit 309 acquires display information when the robot 20x is in an intermediate floor state and the number of people getting on and off the elevator is less than or equal to a predetermined number, and the display unit 313 controls the display according to this case.

[0307] The transmission control unit 310 controls the transmission timing of the transmission data acquired from the data processing unit (omitted in Figure 29) and / or the control information to be transmitted to the base station 10. The transmission control unit 310 also performs a predetermined transmission process on at least one of the transmission data and the control information to generate a signal. The predetermined transmission process may include baseband signal processing such as encoding and modulation.

[0308] The transmitting unit 311 transmits a signal to the base station 10 based on the transmission timing control in the transmitting control unit 310.

[0309] The embodiments have been described above.

[0310] The embodiments described above exemplify a system in which information acquired by a robot 20x while moving indoors and / or outdoors is monitored by a monitoring server 30, but the disclosure is not limited thereto. For example, the disclosure may be applied to a system in which a robot transports objects while moving indoors and / or outdoors, or a system in which a robot transports visitors while moving indoors and / or outdoors. In this case, the robot can be appropriately controlled by the server operating the robot by checking the robot's position information and camera images.

[0311] The above-described embodiment is applicable to both cases where the robot is operated by a user and where the robot operates autonomously. It is also applicable when the user operates the robot for some operations and / or some states, and the robot operates autonomously otherwise. Here, the operations and / or some states may include, for example, the operation when the elevator stops at an intermediate floor. Alternatively, the operation may be selected when the elevator stops at an intermediate floor and the number of people getting on or off at that intermediate floor exceeds a predetermined number. In this case, for example, if the number of people getting on or off exceeds a predetermined number, the robot may disembark from the elevator and then re-board the elevator, which may be performed by the user's instruction (or operation).

[0312] In addition, the "...part" in the above embodiment may be a "...circuitry", a "...device", a "...unit", or a "...module".

[0313] Furthermore, the term "channel" in the above embodiment may be replaced with other terms such as "frequency," "frequency channel," "bandwidth," "band," "carrier," "subcarrier," or "(frequency) resource."

[0314] This disclosure can be implemented using software, hardware, or software integrated with hardware.

[0315] Each functional block used in the description of the above embodiments may be implemented partially or entirely as an integrated circuit (LSI), and each process described in the above embodiments may be controlled partially or entirely by a single LSI or a combination of LSIs. An LSI may consist of individual chips, or it may consist of a single chip that includes some or all of the functional blocks. An LSI may have data inputs and outputs. Depending on the degree of integration, LSIs may be referred to as ICs, system LSIs, super LSIs, or ultra LSIs.

[0316] The method of integration is not limited to LSIs; it may also be implemented using dedicated circuits, general-purpose processors, or dedicated processors. Furthermore, FPGAs (Field Programmable Gate Arrays) that can be programmed after LSI manufacturing, or reconfigurable processors that allow for the reconfiguration of the connections and settings of circuit cells within the LSI, may also be used. This disclosure may be implemented as digital or analog processing.

[0317] Furthermore, if advancements in semiconductor technology or related technologies lead to the emergence of integrated circuit technologies that replace LSIs, then naturally, these technologies can be used to integrate functional blocks. The application of biotechnology, for example, is a possible possibility.

[0318] This disclosure is applicable to all types of devices, systems, and equipment with communication capabilities (collectively referred to as communication devices). Non-exclusive examples of communication devices include telephones (mobile phones, smartphones, etc.), tablets, personal computers (PCs) (laptops, desktops, notebooks, etc.), cameras (digital still / video cameras, etc.), digital players (digital audio / video players, etc.), wearable devices (wearable cameras, smartwatches, tracking devices, etc.), game consoles, digital book readers, telehealth and telemedicine devices, vehicles or mobile transport with communication capabilities (automobiles, airplanes, ships, etc.), and combinations of the above-mentioned devices.

[0319] Communication devices are not limited to portable or movable devices, but also include all kinds of non-portable or fixed devices, devices, and systems, such as smart home devices (appliances, lighting equipment, smart meters or measuring instruments, control panels, etc.), vending machines, and any other "things" that may exist on an IoT (Internet of Things) network.

[0320] Communication includes data communication via cellular systems, wireless LAN systems, and communication satellite systems, as well as data communication using combinations of these.

[0321] Furthermore, the communication device also includes devices such as controllers and sensors that are connected to or linked to a communication device that performs the communication functions described in this disclosure. For example, this includes controllers and sensors that generate control signals and data signals used by the communication device that performs the communication functions of the communication device.

[0322] Furthermore, communication equipment includes infrastructure facilities such as base stations, access points, and any other devices, devices, and systems that communicate with or control the aforementioned non-limited types of equipment.

[0323] Although various embodiments have been described above with reference to the drawings, it goes without saying that this disclosure is not limited to such examples. It is clear to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of this disclosure. Furthermore, the components in the above embodiments may be combined in any way without departing from the spirit of the disclosure.

[0324] The specific examples of this disclosure have been described in detail above, but these are merely illustrative and do not limit the scope of the claims. The technologies described in the claims include various modifications and changes to the specific examples described above. [Industrial applicability]

[0325] This disclosure is preferred for wireless communication systems. [Explanation of symbols]

[0326] 10 base station 20 devices 30 monitoring servers 101, 301 Receiver 102, 202, 302 Receiving Control Unit 103, 206 State determination unit 104 Display information generation unit 105 Server Communication Unit 106, 208, 310 Transmitting control unit 107, 311 Transmitter 108, 210, 312 Control Unit 201 Wireless Receiver 203 Satellite signal receiving unit 204 Indoor positioning signal receiving unit 205 Position detection unit 207 Notification information generation unit 209 Wireless Transmitter 303 Information Classification Department 304 Elevator External Status Display Control Unit 305 Elevator Door Closed State Indicator Control Unit 306 Elevator occupancy status display control unit 307 Elevator exit status display control unit 308 Intermediate Floor Status First Display Control Unit 309 Intermediate Floor Status Second Display Control Unit 313 Display section

Claims

1. A terminal mounted on a robot, A control unit that determines the state of the robot from among a plurality of candidates representing the state of the robot based on the position of the robot and the state of the elevator used by the robot, and controls the transmission of notification information according to the state of the robot, A transmission unit that transmits the notification information to an external device that monitors the robot in accordance with the transmission control, A terminal equipped with the following features.

2. The transmission control includes at least one of determining whether or not to transmit the notification information, and determining the information to be included in the notification information. The terminal according to claim 1.

3. The aforementioned plurality of candidates include a first state in which the robot is outside the elevator and a second state in which the robot is inside the elevator. The control unit, when the robot is in the second state, performs transmission control to prevent the notification information from being transmitted. The terminal according to claim 1.

4. The aforementioned plurality of candidates include a third state in which the robot is outside the elevator, a fourth state in which the robot is inside the elevator and the elevator doors are closed, and a fifth state in which the robot is inside the elevator and the elevator doors are open. The control unit, when the robot is in the fourth state, performs transmission control to prevent the notification information from being transmitted. The terminal according to claim 1.

5. The control unit generates notification information that includes at least one of image information showing an image taken by the robot and position information showing the robot's position, when the robot's state is the fifth state or the third state. The terminal according to claim 4.

6. The control unit, If the state of the robot is the third state, notification information is generated which includes image information showing an image taken by the robot and location information showing the robot's position. If the state of the robot is the fifth state, the system generates notification information that does not include the image information and the position information. The terminal according to claim 4.

7. The aforementioned plurality of candidates include a sixth state in which the robot is outside the elevator; a seventh state in which the robot is inside the elevator and the elevator doors are closed; an eighth state in which the robot is inside the elevator, the elevator doors are open and the elevator is stopped on the first floor on which the robot boarded; a ninth state in which the robot is inside the elevator, the elevator doors are open and the elevator is stopped on the second floor on which the robot disembarks; and a tenth state in which the robot is inside the elevator, the elevator doors are open and the elevator is stopped on a third floor different from the first and second floors on which the robot boarded. The terminal according to claim 1.

8. The control unit, when the state of the robot is in the tenth state, performs transmission control that does not transmit the notification information. The terminal according to claim 7.

9. The control unit generates, when the state of the robot is in the tenth state, notification information that does not include image information showing an image taken by the robot and location information showing the robot's position. The terminal according to claim 7.

10. The control unit generates notification information including image information showing an image taken by the robot and location information showing the robot's position, when the robot's state is in state 9. The terminal according to claim 7.

11. When the state of the robot is in the tenth state, the control unit generates either first notification information that does not include image information showing an image taken by the robot and location information showing the robot's position, or second notification information that includes the image information and the location information, based on the congestion status of the elevator. The terminal according to claim 7.

12. A terminal mounted on a robot, Based on the position of the robot and the state of the elevator used by the robot, the state of the robot is determined from among a plurality of candidates representing the state of the robot. Depending on the state of the robot, control is performed to send notification information. In accordance with the transmission control, the notification information is transmitted to an external device that monitors the robot. Communication method.

13. A device comprising a terminal mounted on a robot and a base station, The aforementioned terminal is A first control unit determines the state of the robot from among a plurality of candidates representing the state of the robot based on the position of the robot and the state of the elevator used by the robot, and controls the transmission of notification information according to the state of the robot. A first transmission unit that transmits the notification information to the base station in accordance with the transmission control, Equipped with, The aforementioned base station is A second receiving unit that receives the aforementioned notification information, A second control unit generates display information to be displayed on a display device that monitors the robot, based on the notification information, A second transmission unit that transmits the display information to the display device, Equipped with, Communication system.

Images