Mobile robot operation system

Abstract

To provide a mobile robot operation system that enables an operator to surely continue to observe an object to be photographed with video of an appropriate size.SOLUTION: A mobile robot operation system includes: a facility A in which an object 3 to be photographed and a mobile robot 1 are arranged; an operator's room B in which a controller 5 for moving and operating the mobile robot by an operator and an operation monitor 4 that displays video or the like of the object to be photographed are arranged; and a communication network 6 that provides bidirectional communication between the facility and the operator's room.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Embodiments of the present invention relate to a mobile robot operation system equipped with an automatic photographing device that can automatically change the orientation of a photographing device and continue photographing a specified photographing object.

Background Art

[0002] For example, consider a case where a viewer located at a remote location (hereinafter also referred to as an "operator") remotely operates a mobile robot equipped with a photographing device to view the video of an exhibit installed in a facility such as an exhibition hall from various positions and angles in detail.

[0003] FIG. 1 is a schematic diagram of a facility A where an exhibit and a mobile robot are installed and an operator's room B at a remote location. In facility A, there is a mobile robot 1 equipped with an exhibit 3 (hereinafter also referred to as a "photographing object"), a facility staff member 22 who explains the photographing object 3, and an automatic photographing device 11 that photographs the photographing object 3. In the operator's room B at a remote location, there is an operator 51 who operates the mobile robot 1 with a controller 5, and an operation monitor 4 that displays the video of the photographing object 3 and the like.

[0004] The operator 51 moves the mobile robot 1 from the operator's room B at a remote location with a controller 5, observes the video of the photographing object 3 taken at a predetermined position and angle on the operation monitor 4, and listens to the explanation of the facility staff member 22.

[0005] However, for example, when it is desired to observe the photographing object 3 at various positions and angles according to the explanation of the facility staff member 22, the operator 51 needs to move the mobile robot 1. However, there are cases where the photographing object 3 deviates from the photographing area, or the operation of the mobile robot 1 and the adjustment of the direction of the automatic photographing device 11 mounted on the mobile robot 1 become complicated.

[0006] Therefore, a method has been proposed in which a moving route and the position of a shooting object 3 are registered in advance in a mobile robot 1 equipped with an automatic shooting device 11, and when the mobile robot 1 moves along the moving route, the shooting direction of the automatic shooting device 11 is adjusted to the shooting object 3 to continue shooting.

Prior Art Documents

Patent Documents

[0007]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0008] In the conventional mobile robot 1 described above, since it is necessary to register the moving route and the position of the shooting object 3 in advance, there is a problem that the shooting location of the shooting object 3 cannot be changed flexibly. Also, when the shooting object 3 deviates from the shooting area as the mobile robot 1 moves, or when the magnification rate or direction adjustment of the automatic shooting device is inappropriate, there is a problem that it becomes difficult to accurately observe the desired area.

[0009] An embodiment of the present invention has been made to solve the above problems, and an object is to provide a mobile robot operation system equipped with an automatic shooting device that enables an operator to continuously and reliably observe a shooting object in an image of an appropriate size even when the mobile robot is moving.

Means for Solving the Problems

[0010] In order to solve the above problems, the mobile robot operation system according to the present embodiment includes a facility where a shooting target and a mobile robot are arranged, a controller for moving the mobile robot by an operator, and an operation monitor for displaying an image or the like of the shooting target, which are arranged in an operator's room, and a communication network for performing two-way communication between the facility and the operator's room. The mobile robot includes an operation shooting device for shooting the surroundings of the mobile robot, a direction detection unit for detecting the direction of an instruction point of the shooting target instructed by a facility staff member of the facility, a distance measurement unit for measuring the distance from the mobile robot to the instruction point, an automatic shooting device for shooting the shooting target, and an arithmetic processing device. The arithmetic processing device includes an instruction point direction calculation means for calculating the direction of the instruction point, an instruction point distance calculation means for calculating the distance to the instruction point, an optimal magnification adjustment means for adjusting an optimal magnification according to the distance to the instruction point, and when moving the mobile robot, a shooting direction adjustment means for adjusting the shooting direction of the automatic shooting device based on the initial relative position information, the moving amount, and the moving direction of the mobile robot. to continue photographing the indicated point It has.

Effects of the Invention

[0011] According to the present embodiment, even when operating the mobile robot, it is possible to automatically adjust the shooting direction and magnification of the automatic shooting device 11 mounted on the mobile robot 1 in accordance with the movement of the mobile robot 1, and it is possible to continuously observe the shooting target 3 with an image of an appropriate size.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Best Mode for Carrying Out the Invention

[0013] Hereinafter, an embodiment of a mobile robot operation system equipped with an automatic photographing device according to the present invention will be described with reference to the drawings.

[0014] [Overall Configuration] As shown in FIG. 1, the mobile robot operation system according to the present embodiment includes a mobile robot 1 equipped with an automatic photographing device 11 and the like, a photographing object 3 to be observed, a communication device including acoustic devices such as a microphone and a speaker, a facility A in which a facility staff member 22 who explains the photographing object 3 is arranged, and an operator's room B at a remote location. In the operator's room B, an operation monitor 4 for displaying a photographed image of the photographing object 3 and the like, a controller 5 for the operator 51 to move the mobile robot 1, and a communication device 52 including acoustic devices such as a microphone and a speaker are arranged. In addition, a communication network 6 capable of two-way communication is provided between the facility A and the operator's room B.

[0015] The facility staff member 22 in the facility A explains the photographing object 3 and has an indicator 2 for causing the mobile robot 1 to recognize the photographing location of the photographing object 3. As the indicator 2, for example, a laser pointer is used, but an indicating rod with a colored tip may also be used. The operator 51 in the operator's room B remotely operates the mobile robot 1 based on the surrounding video photographed by the operation photographing device 14 mounted on the mobile robot 1, and observes the photographing object 3 photographed by the automatic photographing device 11 with the operation monitor 4. Note that there may be a plurality of operators 51, but in this embodiment, an example in which there is one operator 51 will be described.

[0016] [Mobile Robot] As shown in Fig. 1, the mobile robot 1 includes an automatic photographing device 11 for continuously photographing the object to be photographed 3, a distance measuring unit 12 for measuring the distance to the designated point 21 instructed by the facility staff 22, an azimuth detecting unit 13 for detecting the azimuth (direction, height) of the designated point 21, an operating photographing device 14 used when operating the movement of the mobile robot 1, a communication unit 15, a communication device such as acoustic devices of a microphone 16 and a speaker 17, a monitor 18, and a peripheral obstacle detecting unit 19 for detecting obstacles around the mobile robot 1.

[0017] In this embodiment, the automatic photographing device 11 and the operating photographing device 14 have a pan-tilt mechanism and a zoom function. The distance measuring unit 12 is a distance measuring sensor capable of changing the measuring direction of the distance, and the azimuth detecting unit 13 is a fish-eye camera capable of photographing in a 360° direction. However, the present invention is not limited thereto, and various modifications are possible. For example, as the distance measuring unit 12, a system for measuring the distance to the designated point 21 based on the video of the automatic photographing device 11 may be used. Further, the peripheral obstacle detecting unit 19 uses, for example, a laser range finder to detect the position and shape information of obstacles around the mobile robot 1.

[0018] [Communication System] The communication unit 15 provided in the mobile robot 1 is, for example, a slave unit of a wireless LAN. The communication network 6 enables bidirectional communication between the facility A and the operator's room B at a remote location via the master unit of the wireless LAN arranged at both locations through the Internet. The videos taken by the automatic photographing device 11 and the operating photographing device 14 are transmitted from the communication unit 15 and sent to the operator's room B at a remote location via the communication network 6.

[0019] In addition, the controller 5 with a communication function arranged in the operator's room B is, for example, a personal computer equipped with a WiFi function. The videos of the automatic photographing device 11 and the operating photographing device 14 are received by the controller 5 and displayed as the video 41 of the automatic photographing device 11 and the video 42 of the operating photographing device 14 on the operation monitor 4 connected to the controller 5.

[0020] While observing these videos 41 and 42, the operator 51 performs the movement operation of the mobile robot 1 via the controller 5. The command sent from the controller 5 to the mobile robot 1 is sent to the communication unit 15 provided in the mobile robot 1 via the communication network 6. In addition, a camera 53 may be installed at the upper part of the operation monitor 4, and the operator 51 in the operator room B and the facility staff 22 in the facility A may also communicate with each other by images using an image display device (not shown).

[0021] [Information Processing System] The information processing system of the mobile robot operation system according to this embodiment will be described with reference to FIG. 2. The mobile robot 1 is provided with an information processing device 7 that comprehensively controls the mobile robot, and performs transmission and reception of various signals with the operator room B via the communication network 6.

[0022] The information processing device 7 includes a shooting direction adjustment means 71 that includes a shooting permission determination means 77, an optimal magnification adjustment means 72, an indication point distance calculation means 73, an initial relative position calculation means 74, an indication point azimuth calculation means 75, and a movement position calculation means 76.

[0023] The indication point distance calculation means 73 is a means for calculating the distance from the mobile robot 1 to the indication point 21 based on the measurement value output from the distance measurement unit 12.

[0024] The indication point azimuth calculation means 75 is a means for calculating the azimuth of the indication point 21 as seen from the mobile robot 1 from the signal sent from the azimuth detection unit 13.

[0025] The initial relative position calculation means 74 is a means for calculating the relative position between the initial position before the mobile robot 1 moves and the indication point 21 from the distance to the indication point 21 calculated by the indication point distance calculation means 73 and the azimuth of the indication point 21 calculated by the indication point azimuth calculation means 75.

[0026] The moving position calculation means 76 is a means for calculating the direction and distance that the mobile robot 1 has moved from the initial position from the rotation direction and rotation amount of the wheels in the drive unit of the mobile robot 1, and calculating the current position of the mobile robot 1.

[0027] The shooting direction adjustment means 71 calculates the rotation amount of the pan-tilt of the automatic shooting device 11 for continuously shooting the instruction point 21 from the relative position between the initial position of the mobile robot 1 calculated by the initial relative position calculation means 74 and the instruction point 21, and the current position of the mobile robot 1 calculated by the moving position calculation means 76, and issues a command for the automatic shooting device 11 to move by the calculated rotation amount.

[0028] The optimal magnification adjustment means 72 optimally adjusts the magnification of the zoom function of the automatic shooting device 11 by setting the optimal magnification of the automatic shooting device 11 according to the distance from the mobile robot 1 to the instruction point 21, and observes the shooting object 3 in an appropriate size. In this embodiment, a set value of the distance is provided as 1 or more, and the optimal magnification is determined according to the set value. Furthermore, by receiving the information of the current position of the mobile robot 1 from the moving position calculation means 76, the zoom magnification of the automatic shooting device 11 can be automatically and optimally changed even when the mobile robot 1 moves.

[0029] The shooting permission determination means 77 is included in the shooting direction adjustment means 71, and determines whether it is possible to continue shooting the instruction point 21 when the mobile robot 1 continues to move in the moving direction from the current position of the mobile robot 1, the moving direction of the mobile robot 1, the movable area and the current shooting direction of the automatic shooting device 11, and the body information of the mobile robot 1. If it is determined that shooting cannot be continued, it is a means for displaying that fact on the operation monitor 4.

[0030] In addition, the shooting permission determination means 77 can determine whether shooting is possible in consideration of the surrounding situation by utilizing the obstacle information around the mobile robot 1 detected by the surrounding obstacle detection unit 19.

[0031] [Operation Flow of Mobile Robot] In the mobile robot operation system configured as described above, when operating the mobile robot 1, an example of the operation flow when continuously photographing the photographing object 3 while automatically adjusting the direction of the automatic photographing device 11 will be described with reference to FIGS. 3 and 4.

[0032] (ST1) First, the operator 51 uses the controller 5 to move the mobile robot 1 in the standby position to a predetermined area (the photographing location of the mobile robot) where the photographing object 3 in facility A is installed (not shown). (ST2) Next, the facility staff 22 of facility A who explains the photographing object 3 indicates the photographing object 3 with the indicator 2, thereby forming an indication point 21.

[0033] (ST3) Next, based on the video signal being photographed by the azimuth detection unit 13 mounted on the mobile robot 1, the azimuth calculation means 75 for the indication point calculates the azimuth (direction, height) of the indication point 21, and sets the position of the mobile robot 1 at this time as the initial position. (ST4) Based on the azimuth of the indication point 21 calculated in ST3, the distance measurement unit 12 is directed toward the indication point 21, the distance to the indication point 21 is measured, and based on the measured value, the indication point distance calculation means 73 calculates the distance to the indication point 21. (ST5) Next, based on the azimuth of the indication point 21, the photographing direction is adjusted so that the automatic photographing device 11 faces the indication point 21.

[0034] (ST6) The distance to the indication point 21 calculated in ST4 is output to the optimal magnification adjustment means 72, and it is determined whether the distance to the indication point 21 is within the set value. If it is greater than or equal to the set value, a zoom-in or zoom-out operation of the automatic photographing device 11 is performed according to the distance to the indication point 21. That is, photographing is performed by adjusting the optimal magnification according to the distance so that the photographing object 3 has an easily visible size. Note that there may be a plurality of set values, and the optimal magnification is adjusted according to the set value. On the other hand, if it is within the set value, since the optimal magnification is appropriate, the process proceeds to ST7.

[0035] (ST7) Next, based on the azimuth information of the indication point 21 calculated in ST3 and the distance to the indication point 21 calculated in ST4, the initial relative position calculation means 74 calculates the initial relative position between the mobile robot 1 and the indication point 21 with the current position of the mobile robot 1 as the initial position. (ST8) Notify or display to the facility staff 22 of facility A and the operator 51 in the operator's room B that the calculation of the initial relative position has been completed.

[0036] (ST9) Based on the end information of ST8, the facility staff 22 starts to explain the object to be photographed 3 as necessary, and the operator 51 can perform the movement operation of the mobile robot 1. (ST10) In ST10, it is determined whether there is a movement command from the controller 5. If there is a movement command, that is, when the operator 51 attempts to move the mobile robot 1 with the controller 5, it proceeds to ST11. If there is no movement command, it proceeds to ST14, and the mobile robot 1 remains in a standby state until the explanation by the facility staff 22 ends.

[0037] (ST11) In ST11, the photographing availability determination means 77 determines whether the automatic photographing device 11 can continue to photograph the object to be photographed 3 when the mobile robot 1 moves according to the movement command received in ST10. If photographing can be continued, it proceeds to ST12. If photographing cannot be continued, a warning indicating that it cannot be continued is displayed on the operation monitor 4.

[0038] (ST12) In ST12, when the mobile robot 1 moves based on the movement command received in ST10, the movement amount, movement direction information, and distance information of the mobile robot 1 calculated by the movement position calculation means 76, as well as the initial relative position information calculated in ST7, are output to the photographing direction adjustment means 71. The adjustment amount of the photographing direction for the automatic photographing device 11 to continue photographing the indication point 21 is calculated, and a command is issued to the automatic photographing device 11.

[0039] (ST13) In ST13, various information obtained in ST12 is output to the optimal magnification rate adjustment means 72, the distance from the current position of the mobile robot 1 to the position of the instruction point 21 is calculated, and it is determined whether the distance to the instruction point 21 is within the set value. If it is greater than or equal to the set value, a zoom-in or zoom-out operation of the automatic imaging device 11 is performed according to the distance to the instruction point 21. That is, the optimal magnification rate is adjusted according to the distance so that the imaging object 3 is of a size that is easy to observe, and the process proceeds to ST14. On the other hand, if it is within the set value, since the optimal magnification rate is appropriate, the process proceeds to ST14.

[0040] (ST14) In ST14, it is determined whether the explanation by the facility staff 22 has ended. If the explanation has ended, the imaging of the imaging object 3 is terminated and the process proceeds to ST15. If not, the process returns to ST10 and the imaging of the imaging object 3 by the automatic imaging device 11 continues. Here, the determination of whether the explanation by the facility staff 22 has ended is performed by the facility staff 22 or the operator 51, and the output to the mobile robot 1 is, for example, performed by the operator 51 using the controller 5.

[0041] (ST15) In ST15, a two-way communication device including acoustic devices such as microphones and speakers and monitors respectively arranged in the mobile robot 1 and the operator's room is used, and a step in which the facility staff 22 and the operator 51 appropriately ask and answer questions about the imaging object 3. Note that if there is no question and answer, this step can be omitted.

[0042] (ST16) In ST16, it is determined whether there is a next imaging object. If it has ended, the process proceeds to ST17, and the operator 51 moves the mobile robot 1 to the waiting place to end the imaging. On the other hand, if imaging continues, the process returns to ST1, and the operator 51 moves the mobile robot 1 to the place where the next imaging object 3 is installed.

[0043] [Effect] As described above, according to the present embodiment, even when the facility staff 22 of facility A changes the imaging object 3 or the imaging location by the indicator 2, the imaging direction and the optimal magnification of the automatic imaging device 11 mounted on the mobile robot 1 can be automatically adjusted according to the movement of the mobile robot 1, and the imaging object 3 can be continuously observed in an image of an appropriate size.

[0044] As described above, some embodiments of the present invention have been described. However, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, changes, and combinations can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalent scope thereof.

Explanation of Reference Numerals

[0045] A... Facility, B... Operator's room, 1... Mobile robot, 11... Automatic imaging device, 12... Distance measurement unit, 13... Azimuth detection unit, 14... Imaging device for operation, 15... Communication unit, 16... Microphone, 17... Speaker, 18... Monitor, 19... Peripheral obstacle detection unit, 2... Indicator, 21... Indication point, 22... Facility staff, 3... Exhibition object (imaging object), 4... Operation monitor, 41, 42... Images, 5... Controller, 51... Operator, 52... Communication equipment, 53... Camera, 6... Communication network, 7... Arithmetic processing device, 71... Imaging direction adjustment means, 72... Optimal magnification adjustment means, 73... Indication point distance calculation means, 74... Initial relative position calculation means, 75... Indication point azimuth calculation means, 76... Movement position calculation means, 77... Imaging permission determination means

Claims

1. A mobile robot operation system comprising a facility where a subject to be photographed and a mobile robot are arranged, an operator's room where a controller for moving the mobile robot by an operator and an operation monitor for displaying an image or the like of the subject to be photographed are arranged, and a communication network for performing two-way communication between the facility and the operator's room, wherein the mobile robot includes an operation imaging device for photographing the surroundings of the mobile robot, an azimuth detection unit for detecting the azimuth of an instruction point of the subject to be photographed instructed by a facility staff member of the facility, a distance measurement unit for measuring the distance from the mobile robot to the instruction point, an automatic imaging device for imaging the subject to be photographed, and an arithmetic processing device, and the arithmetic processing device includes an instruction point azimuth calculation means for calculating the azimuth of the instruction point, an instruction point distance calculation means for calculating the distance to the instruction point, an optimal magnification adjustment means for adjusting an optimal magnification according to the distance to the instruction point, and a photographing direction adjustment means for adjusting the photographing direction so that the automatic imaging device continuously photographs the instruction point based on the initial relative position information, the movement amount, and the movement direction of the mobile robot when moving the mobile robot. A mobile robot operation system.

2. The mobile robot operation system according to claim 1, wherein the photographing direction adjustment means includes a photographing possibility determination means for determining whether or not the automatic imaging device can continue to photograph the subject to be photographed when moving the mobile robot.

3. The mobile robot operation system according to claim 1 or 2, wherein the mobile robot includes a peripheral obstacle detection unit for detecting obstacles around the mobile robot.

4. The mobile robot operation system according to any one of claims 1 to 3, wherein the mobile robot and the operator's room each have a two-way communicable communication device including an acoustic device and a monitor or the like.

Images