Video control device, video control system, video control method, and program

Abstract

The present invention makes it possible to acquire video for assisting safe travel of a mobile robot. This video control device includes: a position information acquisition unit that acquires position information of a mobile robot; a movement destination information acquisition unit that acquires movement destination information of the mobile robot; and a video control unit that controls at least an imaging device that images the mobile robot or a video acquired by the imaging device, on the basis of the acquired position information and the movement destination information.

Description

Video control device, video control system, video control method, and program

[0001] The present disclosure relates to a video control device, a video control system, a video control method, and a program.

[0002] As a related technique, a mobile robot control system is disclosed. In the mobile robot control system described in Patent Document 1, a mobile robot moves autonomously within a predetermined facility. A camera captures images of the area in which the robot moves. The camera is installed on the ceiling of the area in which the mobile robot passes. A server acquires image data from the camera. The server detects the presence or absence of stationary objects included in the image data.

[0003] The server updates the map information by reflecting information about stationary objects in the map information. The server supplies the updated map information to the mobile robot. The mobile robot uses the updated map information to change its travel route. For example, if a stationary object is placed in a passageway, the mobile robot generates a route that avoids the stationary object and places it in the passageway. The mobile robot travels along the generated route, passing by the stationary object.

[0004] JP 2022-38520 A

[0005] In Patent Document 1, a camera that captures images of the area in which a mobile robot moves is used to detect stationary objects on the path of the mobile robot. Although Patent Document 1 describes reflecting the positions of detected stationary objects in map information, it does not describe using camera images to monitor the mobile robot. Therefore, in Patent Document 1, the images captured by the camera are not necessarily suitable for remote monitoring of a mobile robot.

[0006] One object of the present disclosure is to provide a video control system, a video control device, a video control method, and a program that can acquire video to assist the safe driving of a mobile robot.

[0007] The video control device according to the first aspect of the present disclosure includes a location information acquisition unit that acquires location information of a mobile robot, a destination information acquisition unit that acquires destination information of the mobile robot, and a video control unit that controls at least one of the video acquired by an imaging device that photographs the mobile robot and the imaging device based on the acquired location information and destination information.

[0008] A video control system according to a second aspect of the present disclosure includes an imaging device that captures an image of a mobile robot and the video control device.

[0009] A video control method according to a third aspect of the present disclosure includes acquiring position information of a mobile robot, acquiring destination information of the mobile robot, and controlling at least one of the video captured by an imaging device that captures the mobile robot and the imaging device based on the acquired position information and destination information.

[0010] A program according to a fourth aspect of the present disclosure includes acquiring position information of a mobile robot, acquiring destination information of the mobile robot, and controlling at least one of an image captured by an imaging device that captures the mobile robot and the imaging device based on the acquired position information and destination information.

[0011] The video control system, video control device, video control method, and program according to the present disclosure can acquire video to assist the safe running of a mobile robot.

[0012] Fig. 1 is a block diagram showing a schematic configuration example of a video control system according to the present disclosure; Fig. 2 is a block diagram showing a configuration example of a video control system according to the present disclosure; Fig. 3 is a schematic diagram showing an example of an intersection where a mobile robot crosses; Fig. 4 is a flowchart showing an operation procedure of a video control device; Fig. 5 is a block diagram showing an example of a hardware configuration of an electronic control device;

[0013] 1 is a block diagram showing a schematic configuration example of a video control system according to the present disclosure. The video control system 10 includes a video control device 20 and an imaging device 30.

[0014] The imaging device 30 captures an image of the mobile robot. The position information acquisition unit 21 acquires position information of the mobile robot. The destination information acquisition unit 22 acquires destination information of the mobile robot. The image control unit 23 controls at least one of the image captured by the imaging device 30 and the imaging device 30 itself, based on the acquired position information and destination information.

[0015] In the present disclosure, the image control unit 23 controls at least one of the image captured by the imaging device 30 and the imaging device 30, according to the position information of the mobile robot and the destination information of the mobile robot. A monitor of the mobile robot can monitor the mobile robot using the image controlled by the image control unit 23. As an example, the image control unit 23 controls the image or the imaging device 30 so that both the mobile robot and the destination of the mobile robot are captured in a single image. In this case, the monitor can easily check the presence or absence of obstacles at the destination of the mobile robot using the image. In this way, the image control system 10 according to the present disclosure can acquire images to support the safe travel of the mobile robot.

[0016] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the following description and drawings have been omitted and simplified as appropriate for clarity of explanation. In addition, in the following drawings, the same or similar elements are designated by the same reference numerals, and duplicate explanations are omitted as necessary.

[0017] Fig. 2 is a block diagram showing an example configuration of a video control system according to the present disclosure. One embodiment of the present disclosure will be described using Fig. 2. The video control system 100 shown in Fig. 2 includes a video control device 101 and a camera 210. In this embodiment, the video control system 100 is used to support the safe running of a mobile robot 220. The video control system 100 corresponds to the video control system 10 shown in Fig. 1.

[0018] In this embodiment, the mobile robot 220 is equipped with sensors such as a camera, radar, or LiDAR (Light Detection and Ranging), and an electronic control device. The mobile robot 220 is configured to be capable of autonomous driving by the sensors and electronic control device installed in the mobile robot 220. The mobile robot 220 autonomously drives from a starting point to a final destination according to a driving route plan for autonomous driving. The mobile robot 220 may be remotely operated or controlled by a person such as a remote monitor.

[0019] The mobile robot 220 can travel on a sidewalk at, for example, a speed equivalent to a human walking speed. The mobile robot 220 may be configured as, for example, a delivery robot that delivers packages. The mobile robot 220 may also be configured as a security robot used for security purposes. The mobile robot 220 stores three-dimensional high-precision map information. The mobile robot 220 is assumed to be able to recognize intersections, the positions of traffic lights, and the boundaries between sidewalks and roadways using the three-dimensional high-precision map information.

[0020] The camera 210 is installed on a road and captures an image of the location on the road where the mobile robot 220 is traveling. The camera 210 acquires an image of a crosswalk, for example. When the mobile robot 220 is crossing a road, the camera 210 acquires an image of the mobile robot 220 traveling on the crosswalk. The camera 210 may be installed on a traffic light pole of a traffic light installed at an intersection where the mobile robot 220 passes. The number of cameras 210 at one intersection is not limited to one. Multiple cameras 210 may be installed at one intersection. The camera 210 corresponds to the imaging device 30 shown in FIG. 1 .

[0021] The image control device 101 is a device that controls the image captured by the camera 210. The image control device 101 is placed, for example, in a monitoring center that remotely monitors the mobile robot 220. The image control device 101 receives images from the camera 210 via a network. The image control device 101 displays the images from the camera 210 on the monitoring screen display device 120. The monitoring center may receive various information including images from the mobile robot 220 via a wireless communication network and display the received information on the monitoring screen display device 120.

[0022] The monitoring screen display device 120 is configured as a display device such as a liquid crystal display device, for example. The monitoring screen display device 120 may include a plurality of display devices. A remote monitor who remotely monitors the mobile robot 220 can use the video captured by the camera 210 to monitor whether the mobile robot 220 is traveling safely.

[0023] The mobile robot 220 has a position information transmission unit 221 and a destination information transmission unit 222. The mobile robot 220 includes a device having one or more memories and one or more processors. At least a part of the function of each unit in the mobile robot 220 can be realized by the one or more processors performing processing in accordance with instructions read from one or more memories. The mobile robot 220 also includes drive units such as motors and wheels, an autonomous driving control unit, and a wireless communication device, although these are not shown in FIG. 2 .

[0024] The mobile robot 220 is equipped with a Global Navigation Satellite System (GNSS) and measures its position information using the GNSS. The position information transmission unit 221 transmits the position information to the video control device 101 via a wireless communication network. The position information transmission unit 221 periodically transmits the position information to the video control device 101, for example, at predetermined time intervals.

[0025] Based on the travel route plan for autonomous driving, the mobile robot 220 transmits destination information, i.e., information about the destination of the mobile robot 220, to the image control device 101. The destination information includes, for example, the direction of movement of the mobile robot 220 and position information of the destination of the mobile robot 220.

[0026] The video control device 101 has a location information acquisition unit 111, a destination information acquisition unit 112, and a video control unit 113. The video control device 101 can be physically configured as a device having one or more memories and one or more processors. At least a portion of the functions of each unit in the video control device 101 can be realized by one or more processors performing processing in accordance with instructions read from one or more memories. The video control device 101 corresponds to the video control device 20 shown in FIG. 1.

[0027] The position information acquisition unit 111 acquires position information from the mobile robot 220. The destination information acquisition unit 112 acquires destination information of the mobile robot 220. If the mobile robot 220 is remotely controlled, the destination information acquisition unit 112 may acquire the destination information from a remote control device not shown in FIG. 2. The position information acquisition unit 111 corresponds to the position information acquisition unit 21 shown in FIG. 1. The destination information acquisition unit 112 corresponds to the destination information acquisition unit 22 shown in FIG. 1.

[0028] The image control unit 113 controls the image captured by the camera 210 capturing the moving robot 220 based on the position information acquired by the position information acquisition unit 111 and the destination information acquired by the destination information acquisition unit 112. The image control unit 113 controls the image captured by the camera 210 so that, for example, both the moving robot 220 and the destination of the moving robot 220 are captured in a single image. The image control unit 113 may control the image captured by controlling the angle of view of the camera 210, for example.

[0029] The image control unit 113 identifies an intersection that the mobile robot 220 has reached, for example, based on the position information of the mobile robot 220. The image control unit 113 identifies the direction in which the mobile robot 220 is heading from its current position, based on the destination information of the mobile robot 220. The camera 210 is configured, for example, as a PTZ (panoramic, tilt, zoom) camera whose pan, tilt, and zoom can each be controlled. The image control unit 113 may control the pan, tilt, and zoom of the camera 210 so that the mobile robot 220 and its destination fit within a single image range.

[0030] The image control unit 113 may trim a portion of the image captured using the camera 210. In this case, the image control unit 113 may control the image displayed on the monitoring screen display device 120 by controlling the area to be trimmed from the image according to the position information and the destination information. For example, the image control unit 113 may cut out an area including the mobile robot 220 and its destination from the image captured using the camera 210 based on the position information and the destination information of the mobile robot 220.

[0031] When multiple cameras 210 are placed at an intersection, the image control unit 113 may select, from the multiple cameras 210, a camera 210 to be used to capture the moving robot 220 based on the position information and the destination information. For example, the image control unit 113 selects, from the multiple cameras 210, a camera 210 that is suitable for capturing the moving robot 220 and its destination. The image control unit 113 may control the pan, tilt, and zoom of the selected camera 210 so that the moving robot 220 and its destination fit within a single image range. The image control unit 113 corresponds to the image control unit 23 shown in FIG. 1 .

[0032] 3 is a schematic diagram showing an example of an intersection where a mobile robot 220 crosses. In this example, cameras placed at the intersection include two cameras 210A and 210B. The mobile robot 220 stops temporarily before the crosswalk. At the intersection, the mobile robot 220 transmits destination information to the image control device 101. When the traffic light 200 is green, the mobile robot 220 starts crossing the crosswalk. The mobile robot 220 uses sensor information from the sensor 103, such as a camera or LiDAR, to autonomously cross the crosswalk while avoiding pedestrians and obstacles.

[0033] Camera 210A and camera 210B each capture images of mobile robot 220 traveling on the crosswalk and the surrounding area. When mobile robot 220 crosses the crosswalk and heads toward point P on the side where traffic light 200 is installed, image control unit 113 selects camera 210B from the two cameras. Image control unit 113 controls the orientation and angle of view of camera 210B so that both mobile robot 220 and point P are included in the image captured by camera 210B.

[0034] When mobile robot 220 moves out of the shooting range of camera 210, image control unit 113 hands over the camera used to shoot images of mobile robot 220 to another camera. For example, consider a case where mobile robot 220 changes course to the right at point P, crosses a crosswalk not shown in Fig. 3, and heads toward another point. In this case, image control unit 113 hands over the camera used to shoot images of mobile robot 220 from camera 210B to another camera.

[0035] Next, the operation procedure will be described. Fig. 4 is a flowchart showing the operation procedure of the video control device 101. The operation procedure of the video control device 101 corresponds to a video control method. The position information acquisition unit 111 acquires position information of the mobile robot 220 (step S1). The destination information acquisition unit 112 acquires destination information of the mobile robot 220 (step S2). The video control unit 113 controls the video captured by the camera 210 based on the position information acquired in step S1 and the destination information acquired in step S2 (step S3). In step S3, the video control unit 113 may control the video captured by the camera 210 by controlling the orientation and zoom of the camera 210.

[0036] In order to ensure that the mobile robot 220 travels safely, it is conceivable to obtain images from a camera mounted on the mobile robot 220 and remotely monitor the mobile robot 220. However, the images captured by the camera mounted on the mobile robot 220 have wide blind spots, and it may not be possible to safely remotely monitor the mobile robot 220. In this embodiment, the image control device 101 displays images of the mobile robot 220 traveling on a road, captured by a camera 210 installed at a traffic light or the like, on the monitoring screen display device 120. In this case, a remote monitor can remotely monitor the mobile robot using the images from the camera 210.

[0037] In this embodiment, the image control unit 113 controls the image captured by the camera 210 based on the position information of the mobile robot 220 and the destination information of the mobile robot 220. For example, the image control unit 113 controls the image captured by the camera 210 so that both the mobile robot 220 and the destination of the mobile robot 220 are captured in a single image. In this case, a monitor can use the controlled image to check whether there is an obstacle at the destination of the mobile robot. Therefore, the image control device 101 according to this embodiment can capture an image for supporting the safe traveling of the mobile robot 220 in remote monitoring using the image captured by the camera 210.

[0038] Next, the hardware configuration of the video control device 101 will be described. The video control device 101 can be configured as an electronic control device or a computer device. FIG. 5 shows an example of the hardware configuration of an electronic control device that can be used as the video control device 101. The electronic control device 500 has a processor 501 such as a CPU (Central Processing Unit), a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503. In the electronic control device 500, the processor 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. Although not shown, the electronic control device 500 can also include other circuits such as peripheral circuits, communication circuits, and interface circuits.

[0039] The ROM 502 is a non-volatile storage device. For example, a semiconductor storage device with a relatively small capacity, such as a flash memory, is used as the ROM 502. The ROM 502 stores the programs executed by the processor 501.

[0040] The program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include RAM, ROM, flash memory, SSD or other memory technologies, Compact Disc (CD), digital versatile disc (DVD), Blu-ray (registered trademark) disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0041] The RAM 503 is a volatile storage device. Various semiconductor memory devices such as a dynamic random access memory (DRAM) or a static random access memory (SRAM) are used for the RAM 503. The RAM 503 can be used as an internal buffer for temporarily storing data and the like.

[0042] The processor 501 loads a program stored in the ROM 502 into the RAM 503 and executes the program. When the CPU 501 executes the program, at least some of the functions of each unit in the video control device 101 can be realized.

[0043] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0044] Each drawing is merely an example for describing one or more embodiments. Each drawing may not relate to only one particular embodiment, but may also relate to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0045] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0046] [Supplementary Note 1] An image control device comprising: a location information acquisition unit that acquires location information of a mobile robot; a destination information acquisition unit that acquires destination information of the mobile robot; and an image control unit that controls at least one of an image acquired by an imaging device that images the mobile robot and the imaging device based on the acquired location information and the destination information.

[0047] [Supplementary Note 2] The video control device according to Supplementary Note 1, wherein the video control unit controls at least one of the video and the imaging device so that both the mobile robot and a destination of the mobile robot are included in a single video.

[0048] [Supplementary Note 3] The video control device according to Supplementary Note 1 or 2, wherein the imaging device is placed on a road along which the mobile robot passes.

[0049] [Appendix 4] The image control device described in Appendix 3, wherein a plurality of imaging devices are arranged on the road, and the image control unit selects an imaging device to be used to photograph the mobile robot from among the plurality of imaging devices based on the location information and the destination information.

[0050] [Supplementary Note 5] The video control device according to any one of Supplementary Notes 1 to 4, wherein the mobile robot is a mobile robot that travels on a sidewalk.

[0051] [Supplementary Note 6] The video control device according to any one of Supplementary Notes 1 to 5, wherein the video control unit controls the video by controlling an angle of view of the imaging device.

[0052] [Supplementary Note 7] The video control device according to any one of Supplementary Notes 1 to 6, wherein the video control unit controls the video by controlling an area to be trimmed from the video.

[0053] [Supplementary Note 8] A video control system comprising: an imaging device that captures an image of a mobile robot; and the video control device according to any one of Supplementary Notes 1 to 7.

[0054] [Supplementary Note 9] An image control method comprising: acquiring position information of a mobile robot; acquiring destination information of the mobile robot; and controlling at least one of an image captured by an imaging device that captures an image of the mobile robot and the imaging device based on the acquired position information and destination information.

[0055] [Supplementary Note 10] The video control method according to Supplementary Note 9, further comprising controlling at least one of the video and the imaging device so that both the mobile robot and a destination of the mobile robot are included in a single video.

[0056] [Supplementary Note 11] The video control method according to Supplementary Note 9 or 10, wherein the imaging device is placed on a road along which the mobile robot passes.

[0057] [Supplementary Note 12] The video control method according to Supplementary Note 11, wherein a plurality of imaging devices are arranged on the road, and an imaging device to be used to photograph the mobile robot is selected from the plurality of imaging devices based on the position information and the destination information.

[0058] [Supplementary Note 13] The video control method according to any one of Supplementary Notes 9 to 12, wherein the mobile robot is a mobile robot that travels on a sidewalk.

[0059] [Supplementary Note 14] The image control method according to any one of Supplementary Notes 9 to 13, wherein the image is controlled by controlling an angle of view of the imaging device.

[0060] [Supplementary Note 15] The video control method according to any one of Supplementary Notes 9 to 14, wherein the video is controlled by controlling an area to be trimmed from the video.

[0061] [Supplementary Note 16] A program comprising: acquiring position information of a mobile robot; acquiring destination information of the mobile robot; and controlling at least one of an image captured by an imaging device that captures an image of the mobile robot and the imaging device based on the acquired position information and destination information.

[0062] [Supplementary Note 17] The program according to Supplementary Note 16, which controls at least one of the video and the imaging device so that both the mobile robot and a destination of the mobile robot are included in one video.

[0063] [Supplementary Note 18] The program according to Supplementary Note 16 or 17, wherein the imaging device is placed on a road on which the mobile robot passes.

[0064] [Supplementary Note 19] The program according to Supplementary Note 18, wherein a plurality of imaging devices are arranged on the road, and an imaging device to be used to photograph the mobile robot is selected from the plurality of imaging devices based on the position information and the destination information.

[0065] [Supplementary Note 20] The program according to any one of Supplementary Notes 16 to 19, wherein the mobile robot is a mobile robot that travels on a sidewalk.

[0066] [Supplementary Note 21] The program according to any one of Supplementary Notes 16 to 20, wherein the image is controlled by controlling an angle of view of the imaging device.

[0067] [Supplementary Note 22] The program according to any one of Supplementary Notes 16 to 21, wherein the video is controlled by controlling an area to be trimmed from the video.

[0068] 10: Video control system 20: Video control device 21: Position information acquisition unit 22: Destination information acquisition unit 23: Video control unit 30: Imaging device 100: Video control system 101: Video control device 111: Position information acquisition unit 112: Destination information acquisition unit 113: Video control unit 120: Monitoring screen display device 210: Camera 220: Mobile robot 221: Position information transmission unit 222: Destination information transmission unit

Claims

1. An image control device comprising: a location information acquisition unit that acquires location information of a mobile robot; a destination information acquisition unit that acquires destination information of the mobile robot; and an image control unit that controls at least one of the image acquired by an imaging device that photographs the mobile robot and the imaging device based on the acquired location information and destination information.

2. The video control device according to claim 1, wherein the video control unit controls at least one of the video and the imaging device so that both the mobile robot and the destination of the mobile robot are included in a single video.

3. The video control device according to claim 1 or 2, wherein the imaging device is placed on a road along which the mobile robot passes.

4. The image control device described in claim 3, wherein a plurality of imaging devices are arranged on the road, and the image control unit selects an imaging device from the plurality of imaging devices to be used to photograph the mobile robot based on the location information and the destination information.

5. The video control device according to any one of claims 1 to 4, wherein the mobile robot is a mobile robot that travels on a sidewalk.

6. The video control device according to any one of claims 1 to 5, wherein the video control unit controls the video by controlling the angle of view of the imaging device.

7. The video control device according to any one of claims 1 to 6, wherein the video control unit controls the video by controlling an area to be trimmed from the video.

8. A video control system comprising: an imaging device for photographing a mobile robot; and a video control device according to any one of claims 1 to 7.

9. An image control method comprising: acquiring position information of a mobile robot; acquiring destination information of the mobile robot; and controlling at least one of an image captured by an imaging device that captures an image of the mobile robot and the imaging device based on the acquired position information and destination information.

10. The image control method according to claim 9, wherein the images are controlled so that both the mobile robot and the destination of the mobile robot are included in one image.

11. The video control method according to claim 9 or 10, wherein the imaging device is placed on a road along which the mobile robot passes.

12. The video control method described in claim 11, wherein a plurality of imaging devices are arranged on the road, and an imaging device to be used to photograph the mobile robot is selected from the plurality of imaging devices based on the location information and the destination information.

13. A video control method according to any one of claims 9 to 12, wherein the mobile robot is a mobile robot that travels on a sidewalk.

14. The image control method according to any one of claims 9 to 13, wherein the image is controlled by controlling the angle of view of the imaging device.

15. A video control method according to any one of claims 9 to 14, wherein the video is controlled by controlling an area to be trimmed from the video.

16. A program comprising: acquiring position information of a mobile robot; acquiring destination information of the mobile robot; and controlling at least one of an image captured by an imaging device that captures an image of the mobile robot and the imaging device based on the acquired position information and destination information.

17. The program according to claim 16, which controls at least one of the video and the imaging device so that both the mobile robot and the destination of the mobile robot are captured in a single video.

18. The program according to claim 16 or 17, wherein the imaging device is placed on a road along which the mobile robot passes.

19. The program described in claim 18, wherein a plurality of imaging devices are arranged on the road, and an imaging device to be used to photograph the mobile robot is selected from the plurality of imaging devices based on the location information and the destination information.

20. The program according to any one of claims 16 to 19, wherein the mobile robot is a mobile robot that travels on a sidewalk.

Images