Display method, display system, image transmission device, display control device, and computer program

Abstract

To accurately provide information required for remotely monitoring or remotely controlling a mobile to a worker who performs at least one of the remote monitor work and the remote control work of the mobile.SOLUTION: A display method is provided for displaying an image for performing at least one of remote monitor work and remote control work using wireless communications to an autonomously travelable mobile on a screen that a worker who performs the work can browse. The display method includes the steps of: detecting a position and a type of a target existing in a region including the direction of travel of the mobile, which is imaged by a camera loaded on the mobile, based on the image of the region; detecting a distance from the mobile to the target; and displaying on the screen the image on which an overlay region including the type of the target and the distance to the target is superposed in a mode corresponding to the target based on the detected position and type of the target and the detected distance to the target.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present disclosure relates to a display method, a display system, an image transmission device, a display control device, and a computer program. [Background technology]

[0002] Non-patent document 1 discloses a system that detects other moving objects or objects that have fallen on the road around an autonomously running robot from images captured by a camera mounted on the robot, and notifies the detection results to a worker who is performing at least one of the tasks of remotely monitoring and remotely operating the robot.

[0003] Non-Patent Document 2 discloses a system in which an image captured by a camera mounted on an autonomous vehicle is transmitted to a server in a control center, and the server detects objects from the image. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] “Panasonic obtains permission to use fully remote delivery robots on roads”, [online], April 18, 2022, IoTNEWS, R-Gene IoTNEWS Division, [Retrieved August 23, 2023], Internet<URL: https: / / iotnews.jp / smart-city / 200568 / > [Non-Patent Document 2] "NEC develops 'learning-based media transmission control technology' to enhance 'remote monitoring' of cars and other devices using AI", [online], January 12, 2021, DIGITAL SHIFT TIMES, Digital Holdings Inc., [Retrieved August 23, 2023], Internet<URL: https: / / digital-shift.jp / flash_news / FN210112_8> Summary of the Invention [Problem to be solved by the invention]

[0005] However, although the system of Non-Patent Document 1 can notify the worker of the detection result, it is difficult for the worker to instantly determine which object in the video the notified detection result corresponds to. For this reason, it is difficult for one worker to monitor multiple moving objects in parallel.

[0006] In the system of Non-Patent Document 2, when the communication bandwidth of the transmission path between the autonomous vehicle and the server becomes low, the autonomous vehicle must transmit the video to the server after processing such as reducing the image quality. Therefore, the server must perform object detection from low-image-quality video, which may result in missed detection or false detection of objects.

[0007] The present disclosure has been made in consideration of the above circumstances, and aims to provide a display method, display system, image transmission device, display control device, and computer program that can accurately provide information necessary for remote monitoring or remote operation of a mobile object to a worker who performs at least one of the tasks of remotely monitoring or remotely operating a mobile object. [Means for solving the problem]

[0008] A display method according to one aspect of the present disclosure is a display method for displaying images for performing at least one of remote monitoring and remote operation via wireless communication of an autonomously movable body on a screen viewable by a worker performing the work, and includes the steps of detecting the position and type of an object present within an image of the area including the direction of travel of the movable body, based on the image of the area photographed by a camera mounted on the movable body, detecting the distance from the movable body to the object, and displaying on the screen the image in which an overlay area including the type of the object and the distance to the object is superimposed in a manner that can be associated with the object, based on the detected position and type of the object, and the distance to the object. Effect of the Invention

[0009] According to the present disclosure, it is possible to accurately provide information required for remote monitoring or remote operation of a moving object to a worker who performs at least one of the tasks of remotely monitoring or remotely operating a moving object. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an overall configuration of a video display system according to the first embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram showing an example of an image included in a video captured by a camera. [Diagram 3] FIG. 3 is a block diagram illustrating an example of a functional configuration of the video transmission device according to the first embodiment of the present disclosure. [Figure 4] FIG. 4 is a diagram showing an example of the type and position of an object detected by the object detection unit. [Diagram 5] FIG. 5 is a diagram illustrating an example of the priority determination table. [Figure 6] FIG. 6 is a diagram showing an example of an image on which an overlay area is superimposed. [Figure 7] FIG. 7 is a diagram showing an example of an image on which an overlay area is superimposed. [Figure 8] FIG. 8 is a block diagram illustrating an example of a functional configuration of the video receiving device according to the first embodiment of the present disclosure. [Figure 9] FIG. 9 is a flowchart illustrating an example of the operation of the video transmission device according to the first embodiment of the present disclosure. [Figure 10] FIG. 10 is a flowchart illustrating an example of the operation of the video receiving device according to the first embodiment of the present disclosure. [Figure 11] FIG. 11 is a block diagram illustrating an example of a functional configuration of a video transmission device according to the second embodiment of the present disclosure. [Figure 12] FIG. 12 is a block diagram illustrating an example of a functional configuration of a video receiving device according to the second embodiment of the present disclosure. [Figure 13] FIG. 13 is a flowchart illustrating an example of the operation of the video transmission device according to the second embodiment of the present disclosure. [Figure 14] FIG. 14 is a flowchart illustrating an example of the operation of the video receiving device according to the second embodiment of the present disclosure. [Figure 15] FIG. 15 is a diagram showing an example of an image on which an overlay area according to the first modification is superimposed. [Figure 16] FIG. 16 is a diagram showing an example of an image on which an overlay area according to the second modification is superimposed. [Figure 17] FIG. 17 is a diagram showing an example of an image on which a circumscribing rectangle according to the third modification is superimposed. [Figure 18] FIG. 18 is a diagram showing an example of an image on which a circumscribing rectangle according to the fourth modification is superimposed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] [Overview of the embodiment of the present disclosure] First, an overview of the embodiments of the present disclosure will be listed and described. (1) A display method according to one embodiment of the present disclosure is a display method for displaying, on a screen viewable by a worker performing at least one of remote monitoring and remote operation via wireless communication of an autonomously travelling mobile body, an image for performing the at least one of remote monitoring and remote operation of the autonomously travelling mobile body, the display method including the steps of: detecting a position and type of an object present within an image of the area including the direction of travel of the mobile body, based on the image of the area photographed by a camera mounted on the mobile body; detecting a distance from the mobile body to the object; and displaying on the screen the image in which an overlay area including the type of the object and the distance to the object is superimposed in a manner that can be associated with the object, based on the detected position and type of the object, and the distance to the object.

[0012] According to this configuration, the position and type of the object are detected from the image captured by the camera. Therefore, the object can be accurately detected. Also, the overlay area is superimposed in a manner that is associated with the object in the image. Therefore, the worker can instantly determine which moving object the overlay area corresponds to. Therefore, the information required for remote monitoring or remote operation of the moving object can be accurately provided to the worker.

[0013] (2) In the above (1), the step of displaying on the screen may further include displaying on the screen the image whose image quality is controlled based on an available bandwidth of a wireless transmission path of the image.

[0014] According to this configuration, for example, when the available bandwidth is small, the resolution of the portion other than the overlay area can be reduced, and the resolution of the background area other than the area of ​​the target object can be reduced. As a result, even if the available bandwidth is reduced, the overlay area can be displayed on the screen without reducing its resolution. Therefore, the information required for remote monitoring or remote operation of the moving object can be accurately provided to the worker.

[0015] (3) In the above (1) or (2), the display method further includes a step of determining a priority of the object based on at least one of the degree of impact on the object in the event of a collision by the moving body and the danger avoidance capability of the object, and the step of displaying on the screen may further control the superimposition of the overlay area on the image based on the determined priority.

[0016] According to this configuration, for example, the area or number of overlay areas superimposed on an image can be controlled according to the priority, and the display or non-display of the overlay areas can be controlled. Therefore, it is possible to display preferentially the overlay areas of moving objects that have a high degree of impact on the target object, or the overlay areas of moving objects that have a low risk of avoiding the target object. Therefore, the worker can remotely monitor or remotely operate these moving objects so as not to collide with them.

[0017] (4) In any of (1) to (3) above, in the step of displaying on the screen, the display manner of the overlay area may further be controlled based on at least one of the type of the object and the distance to the object.

[0018] With this configuration, for example, the overlay area of ​​a person and the overlay area of ​​an object can be displayed in different colors, or the overlay area of ​​a nearby object and the overlay area of ​​a distant object can be displayed in different colors. Therefore, the worker can instantly determine whether or not a collision between the moving body and the object must be avoided.

[0019] (5) In any of (1) to (4) above, the step of detecting the position and type of the object may further include detecting an area of ​​the object and displaying the area on the screen, and the step of displaying the area of ​​the object on the screen may further include displaying the area of ​​the object on the screen in a manner distinguishable from areas other than the area of ​​the object.

[0020] With this configuration, the operator can instantly determine the area of ​​the object.

[0021] (6) In any one of (1) to (5) above, in the step of displaying on the screen, display of the overlay area may be controlled based on a distance to the object.

[0022] With this configuration, for example, the overlay area can be hidden when the distance to the object is far away, and can be displayed when the distance is within a certain distance. Therefore, the display of the overlay area can be controlled depending on whether or not the object needs to be avoided.

[0023] (7) A display system according to another embodiment of the present disclosure includes an image transmission device and a display control device connected to the image transmission device via a network and configured to display an image on a screen viewable by an operator performing at least one of remote monitoring and remote operation of an autonomously movable body, wherein the image transmission device includes an object detection unit that detects a position and type of an object present within an image of an area including a direction of travel of the moving body, based on the image of the area captured by a camera mounted on the moving body, a distance detection unit that detects a distance from the moving body to the object, an image generation unit that generates an image in which an overlay area including the type of the object and the distance to the object is superimposed in a manner that corresponds to the object, based on the detected position and type of the object and the distance to the object, and a transmission unit that transmits the generated image to the display control device, and the display control device includes a receiving unit that receives the image from the image transmission device and a display control unit that displays the received image on a screen.

[0024] According to this configuration, the position and type of the object are detected from the image captured by the camera. Therefore, the object can be accurately detected. Also, the overlay area is superimposed in a manner that is associated with the object in the image. Therefore, the worker can instantly determine which moving object the overlay area corresponds to. Therefore, the information required for remote monitoring or remote operation of the moving object can be accurately provided to the worker.

[0025] (8) A display system according to another embodiment of the present disclosure includes an image transmitting device and a display control device connected to the image transmitting device via a network and configured to display an image on a screen viewable by an operator performing at least one of remote monitoring and remote operation of an autonomously movable body, wherein the image transmitting device includes an object detection unit that detects a position and type of an object present within an area including a direction of travel of the moving body based on an image of the area captured by a camera mounted on the moving body, a distance detection unit that detects a distance from the moving body to the object, and a transmission unit that transmits the image and metadata to the display control device, wherein the metadata includes a position of the object, a type of the object, and a distance to the object, and the display control device includes a receiving unit that receives the image and the metadata from the image transmitting device, and a display control unit that displays on the screen the image in which an overlay area including the type of the object and the distance to the object is superimposed in a manner that corresponds to the object based on the received image and metadata.

[0026] According to this configuration, the position and type of the object are detected from the image captured by the camera. Therefore, the object can be accurately detected. Also, the overlay area is superimposed in a manner that is associated with the object in the image. Therefore, the worker can instantly determine which moving object the overlay area corresponds to. Therefore, the worker can be accurately provided with information required for remote monitoring or remote operation of the moving object. Also, the data volume of the metadata is very small compared to the image. Therefore, even if the available bandwidth of the network is reduced and the image quality of the image must be reduced, the metadata can be correctly transmitted to the display control device.

[0027] (9) An image transmission device according to another embodiment of the present disclosure is an image transmission device that transmits an image to be displayed on a screen viewable by an operator performing at least one of remote monitoring and remote operation of an autonomously capable moving body to the image display control device, and includes an object detection unit that detects the position and type of an object present in an area including the direction of travel of the moving body based on an image of the area captured by a camera mounted on the moving body, a distance detection unit that detects the distance from the moving body to the object, an image generation unit that generates the image in which an overlay area including the type of the object and the distance to the object is superimposed in a manner that can be associated with the object based on the detected position and type of the object and the distance to the object, and a transmission unit that transmits the generated image to the display control device.

[0028] According to this configuration, the position and type of the object are detected from the image captured by the camera. Therefore, the object can be accurately detected. Also, the overlay area is superimposed in a manner that is associated with the object in the image. Therefore, the worker can instantly determine which moving object the overlay area corresponds to. Therefore, the information required for remote monitoring or remote operation of the moving object can be accurately provided to the worker.

[0029] (10) An image transmission device according to another embodiment of the present disclosure is an image transmission device that transmits an image to be displayed on a screen viewable by an operator performing at least one of remote monitoring and remote operation of an autonomously capable moving body to the image display control device, and is equipped with an object detection unit that detects the position and type of an object present within an area including the direction of travel of the moving body based on an image of the area captured by a camera mounted on the moving body, a distance detection unit that detects the distance from the moving body to the object, and a transmission unit that transmits the image and metadata to the display control device, wherein the metadata includes the position of the object, the type of the object, and the distance to the object.

[0030] According to this configuration, the position and type of the object are detected from the image captured by the camera. Therefore, the object can be accurately detected. Also, the overlay area is superimposed in a manner that is associated with the object in the image. Therefore, the worker can instantly determine which moving object the overlay area corresponds to. Therefore, the worker can be accurately provided with information required for remote monitoring or remote operation of the moving object. Also, the data volume of the metadata is very small compared to the image. Therefore, even if the available bandwidth of the network is reduced and the image quality of the image must be reduced, the metadata can be correctly transmitted to the display control device.

[0031] (11) A display control device according to another embodiment of the present disclosure is a display control device that displays an image on a screen that can be viewed by a worker performing at least one of remote monitoring and remote operation of an autonomously capable moving body, and includes a receiving unit that receives from an image transmitting device an image of an area including the direction of travel of the moving body captured by a camera mounted on the moving body and metadata including the position of an object present within the area, the type of the object, and the distance to the object, and a display control unit that displays on the screen the image in which an overlay area including the type of the object and the distance to the object is superimposed in a manner that corresponds to the object based on the received image and metadata.

[0032] According to this configuration, the overlay area is superimposed in a manner that corresponds to the object in the image based on the image and metadata received from the image transmission device. Therefore, the worker can instantly determine which moving object the overlay area corresponds to. Therefore, the worker can be provided with accurate information required for remote monitoring or remote operation of the moving object.

[0033] (12) A computer program according to another embodiment of the present disclosure causes a computer to function as the image transmission device described in (9) or (10), or the display control device described in (11).

[0034] According to this configuration, the computer can function as the above-mentioned image transmission device or the above-mentioned display control device, and therefore, the same effects and advantages as those of the above-mentioned image transmission device or the above-mentioned display control device are achieved.

[0035] [Details of the embodiment of the present disclosure] Hereinafter, the embodiments of the present disclosure will be described with reference to the drawings. Note that each of the embodiments described below shows a specific example of the present disclosure. The numerical values, shapes, materials, components, the arrangement and connection forms of the components, steps, and the order of steps shown in the following embodiments are examples and do not limit the present disclosure. In addition, among the components in the following embodiments, components that are not described in the independent claims are components that can be added arbitrarily. In addition, each figure is a schematic diagram and is not necessarily illustrated strictly.

[0036] In addition, the same components are denoted by the same reference numerals, and since their functions and names are also the same, their explanations will be omitted as appropriate.

[0037] <Embodiment 1> [Overall configuration of the video display system] FIG. 1 is a diagram showing an overall configuration of a video display system according to the first embodiment of the present disclosure. The video display system 100 includes a plurality of robots 1 that run autonomously on a road 4, and a video receiving device 9 that is connected to each robot 1 via a network 7.

[0038] The robot 1 is an example of an autonomous mobile object, for example, a delivery robot that delivers packages to a destination. The robot 1 includes a camera 2 that captures the direction in which the robot 1 travels, and a video transmission device 3. Note that the robot 1 may be replaced by another autonomous mobile object. For example, the robot 1 may be replaced by an autonomous drone.

[0039] For example, while a robot 1 travels on a road 4 in the direction of the arrow in Fig. 1, a camera 2 captures an image of the direction of travel of the robot 1. In the direction of travel of the robot 1, there are an object 5 and an adult 6B.

[0040] Fig. 2 is a diagram showing an example of an image included in a video captured by the camera 2. Fig. 2 shows one of a time series of images included in the video. The image includes, in order from closest to the robot 1, an object 5, an adult 6B, and an adult 6A. Note that the adult 6A is not shown in Fig. 1.

[0041] Based on the image captured by the camera 2, the robot 1 autonomously travels to the destination while avoiding collisions with obstacles such as the object 5, the adult 6B, and the adult 6A.

[0042] Referring back to FIG. 1, video transmitting device 3 compresses the video captured by camera 2, and transmits the compressed video to video receiving device 9 via wireless base station 8 and network 7 by wireless communication.

[0043] The network 7 transfers data in accordance with a communication standard such as LTE (Long Term Evolution) or 5G (fifth generation mobile communication system), etc. The wireless base station 8 is connected to the network 7 and performs wireless communication.

[0044] The video receiving device 9 is connected to the network 7 and receives compressed video from the video transmitting device 3 via the network 7. The video receiving device 9 expands the received compressed video and displays it on the display device 10. The video receiving device 9 receives and expands compressed video from each of the video transmitting devices 3 of the multiple robots 1, and displays the multiple expanded videos side by side. For example, the screen of the display device 10 is assumed to be divided into nine partial screens. The video receiving device 9 displays the videos from the video transmitting devices 3 of the nine robots 1 on the nine partial screens of the display device 10, respectively. Furthermore, when nine display devices 10 are prepared, the video receiving device 9 may display the videos from the video transmitting devices 3 of the nine robots 1 on the screens of the nine display devices 10, respectively. Note that the number of robots 1 and the number of display devices 10 are merely an example, and other numbers of each may be used.

[0045] The display device 10 can be viewed by a worker who performs at least one of remote monitoring and remote operation of the robot 1. That is, the worker may only remotely monitor the robot 1, may only remotely operate the robot 1, or may perform both remote monitoring and remote operation of the robot 1. Which operation the worker performs may change over time. For example, in a certain time phase, the worker may only remotely monitor the robot 1, and in the next time phase, the worker may only remotely operate the robot 1. Furthermore, in the next time phase, the worker may simultaneously perform remote monitoring and remote operation of the robot 1 in parallel. As a result, while looking at the screen of the display device 10, the worker judges whether or not it is an emergency situation in which the robot 1 is expected to collide with an obstacle, for each of the multiple robots 1. If the worker judges that it is an emergency situation, he or she performs manual intervention and remotely operates the robot 1 by wireless communication via the network 7 and the wireless base station 8. As a result, the worker manually remotely controls the moving direction, speed, etc. of the robot 1.

[0046] [Configuration of video transmission device 3] FIG. 3 is a block diagram showing an example of a functional configuration of the video transmission device 3 according to the first embodiment of the present disclosure. The video transmission device 3 includes a video acquisition unit 301, an object detection unit 302, a distance detection unit 303, a speed detection unit 304, a priority determination unit 305, a bandwidth observation unit 306, an area calculation unit 307, an overlay processing unit 308, a video compression unit 309, and a video transmission unit 310.

[0047] The image acquisition unit 301 acquires an image of the area in front of the robot 1 from the camera 2 . The object detection unit 302 detects the type and position of an object (object) included in the image by image processing of the image acquired by the image acquisition unit 301. The types of objects include, for example, "child / elderly", "adult", and "object". The position of the object is indicated, for example, by the position of a circumscribing rectangle surrounding the object (for example, the coordinates of the upper left corner and the lower right corner of the circumscribing rectangle). For example, the object detection unit 302 inputs each image constituting the input image to a learning model, and acquires information on the position of the circumscribing rectangle of the object and the type of the object from the learning model. The learning model is, for example, a convolution neural network (CNN), a recurrent neural network (RNN), an autoencoder, or the like. Using images including various types of objects as teacher data, learning of the type and position of the object progresses by a machine learning method such as deep learning, and the parameters of the learning model are determined.

[0048] 4 is a diagram showing an example of the types and positions of objects detected by object detection unit 302. For example, object detection unit 302 detects an adult 6A and a circumscribing rectangle 20A, an adult 6B and a circumscribing rectangle 20B, and an object 5 and a circumscribing rectangle 20C as the types and positions of objects.

[0049] The distance detection unit 303 acquires each image and the circumscribing rectangle position of the object in the image from the object detection unit 302. The distance detection unit 303 calculates the distance R from the robot 1 to the object based on the following formula 1. R = F × H / (yd) ... (Equation 1) Here, F is the focal length of camera 2. H is the height to the mounting position of camera 2. y is the Y coordinate of the vanishing point of the object in the image. d is the Y coordinate of the bottom edge position of the circumscribing rectangle in the image.

[0050] The speed detection unit 304 acquires each image and the circumscribing rectangle position of the object in the image from the object detection unit 302. The speed detection unit 304 also acquires the distance to the object present at the circumscribing rectangle position from the distance detection unit 303 based on the circumscribing rectangle position. The speed detection unit 304 calculates the amount of change in the distance to the object between images for each object, and calculates the relative speed of the object based on the calculated amount of change. The speed detection unit 304 acquires the moving speed of the robot 1 from the robot 1, and calculates the absolute speed of the object based on the moving speed of the robot 1 and the relative speed of the object.

[0051] The priority determination unit 305 acquires information on the type, position, and distance of an object from the distance detection unit 303. The priority determination unit 305 acquires information on the type, position, and absolute speed of an object from the speed detection unit 304. The priority determination unit 305 determines a priority for each object based on the information acquired from the distance detection unit 303 and the speed detection unit 304.

[0052] The priority determination unit 305 determines the priority of an object based on at least one of the degree of effect on the object in the event of a collision with the robot 1 and the object's danger avoidance capability.

[0053] Specifically, the priority determination unit 305 acquires the movement speed of the robot 1 from the robot 1, and calculates the relative speed of the object with respect to the robot 1 based on the movement speed of the robot 1 and the absolute speed of the object. The priority determination unit 305 calculates the predicted collision time between the robot 1 and the object based on the relative speed of the object and the distance from the robot 1 to the object, using the following formula 2. Predicted collision time = distance from robot 1 to the object / relative speed of the object ... (Equation 2)

[0054] The priority determination unit 305 may use the relative speed of the object detected by the speed detection unit 304 as the relative speed of the object in Equation 2.

[0055] The priority determination unit 305 refers to a priority determination table, which will be described later, and determines the priority of the object based on the type of the object and the predicted collision time of the object.

[0056] FIG. 5 is a diagram showing an example of a priority determination table. The priority determination table specifies the priority of an object for each combination of the object type and the predicted collision time. Here, the smaller the priority value, the higher the priority. For example, if the object type is an "adult" and the predicted collision time is "less than 1 minute," the priority is "2." Also, if the object is surrounded by "objects" and the predicted collision time is "less than 3 minutes," the priority is "4."

[0057] The priority for the same predicted collision time is "child / elderly", "adult", and "object" in that order. This is because, when the robot 1 collides with an object, the degree of the impact on "child / elderly" or "adult" is greater than the impact on "object", and "child / elderly" has a lower danger avoidance ability than "adult". In other words, the higher the impact on the object when the robot 1 collides with it and the lower the object's danger avoidance ability, the higher the priority.

[0058] The bandwidth observing unit 306 observes the available bandwidth of the transmission path through which the video transmitting device 3 performs wireless communication. For example, the bandwidth observing unit 306 may estimate the available bandwidth by transmitting a test stream to a predetermined server, receiving the returned test stream, measuring quality information such as a round trip time and a packet loss rate, and inputting the quality information into a pre-defined analysis model.

[0059] The area calculation unit 307 calculates the total area of ​​the overlay area to be superimposed on the image based on the available bandwidth observed by the bandwidth observation unit 306. Here, the overlay area is an area indicating the attributes of an object. Specifically, the overlay area indicates the type of object and the distance from the robot 1 to the object. An overlay area is provided for each object.

[0060] The area calculation unit 307, for example, refers to table information showing the relationship between the available bandwidth and the total area of ​​the overlay area, and calculates the total area of ​​the overlay area for the available bandwidth. Note that the relationship between the available bandwidth and the total area is such that the larger the available bandwidth, the larger the total area.

[0061] The overlay processing unit 308 determines objects on which to overlay the overlay area in descending order of priority determined by the priority determination unit 305, within the total area of ​​the overlay area calculated by the area calculation unit 307. For example, if the total area is 100 and the area of ​​each overlay area is 30, the overlay processing unit 308 determines three objects in descending order of priority.

[0062] The overlay processing unit 308 superimposes an overlay area of ​​the object on the image acquired by the image acquisition unit 301 in a manner that allows the overlay area to be associated with the determined object. In other words, when an operator of the robot 1 sees the overlay area in the image, the overlay area is superimposed in a manner that allows the operator to recognize which object the overlay area corresponds to. For example, the overlay processing unit 308 may superimpose the overlay area near the object, such as the upper part of the circumscribing rectangle of the object. Note that the overlay area and the object may partially overlap. The overlay processing unit 308 may also associate the object with the overlay area by superimposing a line segment connecting the object and the overlay area on the image. In this case, the overlay processing unit 308 may place the overlay area at a position away from the object, such as an end of the image. An example of placing the overlay area near the object will be described below.

[0063] 6 is a diagram showing an example of an image on which an overlay area is superimposed. On the image, overlay areas 30A, 30B, and 30C are displayed near an adult 6A, an adult 6B, and an object 5, respectively. For example, overlay area 30A shows the type of object "adult" and the distance to adult 6A, "40.7 m."

[0064] For example, if the total area of ​​the overlay areas calculated by the area calculation unit 307 is 50 and the area of ​​each overlay area is 30, the overlay processing unit 308 determines one object with the highest priority. For example, among an adult 6A, an adult 6B, and an object 5, the adult 6B has the highest priority.

[0065] 7 is a diagram showing an example of an image on which an overlay area is superimposed. In the image, an overlay area 30B is displayed near an adult 6B, which has the highest priority, and no overlay area is displayed near the other adult 6A and object 5.

[0066] The overlay processor 308 may control the area of ​​the overlay area so that more overlay areas are superimposed. However, it is assumed that the lower limit of the area of ​​the overlay area is set in advance. For example, if the lower limit of the area is 20 and the total area of ​​the overlay areas is 50, the overlay processor 308 can superimpose two overlay areas on the image.

[0067] For example, the overlay processor 308 sets the area of ​​each overlay area to a predetermined value (e.g., 30) and determines whether all the overlay areas can be superimposed on the image within the range of the total area. If the overlay processor 308 determines that all the overlay areas cannot be superimposed on the image, the overlay processor 308 reduces the area of ​​the overlay areas in stages (e.g., by 5) until all the overlay areas can be superimposed on the image or the area of ​​the overlay areas reaches a lower limit value.

[0068] The video compression unit 309 obtains information on the position and priority of an object from the priority determination unit 305. The video compression unit 309 also obtains video on which an overlay area is superimposed from the overlay processing unit 308. Furthermore, the video compression unit 309 obtains information on available bandwidth from the bandwidth observation unit 306.

[0069] The video compression unit 309 controls the image quality of the compressed video based on the available bandwidth. That is, the video compression unit 309 compresses the video at different compression rates for different regions so that the data amount of the compressed video does not exceed the available bandwidth. The video compression method is, for example, H.264 / MPEG-4 AVC, or H.265 / MPEG-H HEVC. For example, the video compression unit 309 may change the compression rate by changing the value of a quantization parameter QP.

[0070] The video compression unit 309 compresses the video so that the compression rate is lower for the circumscribing rectangular area of ​​an object with a higher priority. The video compression unit 309 also compresses the video so that the compression rate of the area other than the object is higher than the compression rate of the circumscribing rectangular area of ​​the object. However, the video compression unit 309 compresses the overlay area at a compression rate that maintains a quality that allows characters to be read by the human eye when decompressed. If the data volume of the compressed video exceeds the available bandwidth, the video compression unit 309 gradually increases the compression rate of the area other than the object up to a predetermined compression rate, and if the data volume still exceeds the available bandwidth, increases the compression rate of the object starting from the object with the lowest priority.

[0071] By the video compressor 309 compressing the video at different compression rates for different regions in this way, the video transmitter 3 can transmit the compressed video to the video receiver 9 within the available bandwidth.

[0072] The video compression unit 309 adds information on the compression rate for each region to the compressed video and outputs the compressed video to the video transmission unit 310.

[0073] The video transmitting unit 310 transmits the compressed video acquired from the video compressing unit 309 to the video receiving device 9 via the wireless base station 8 and the network 7 .

[0074] [Configuration of video receiving device 9] FIG. 8 is a block diagram showing an example of a functional configuration of the video receiving device 9 according to the first embodiment of the present disclosure. The video receiving device 9 includes a video receiving unit 901 , a video decompression unit 902 , and a display control unit 903 . The video receiving unit 901 receives compressed video from the video transmitting device 3 . The video decompression unit 902 decompresses the compressed video received by the video reception unit 901 for each region, based on the information on the compression rate for each region added to the compressed video.

[0075] The display control unit 903 causes the decompressed image to be displayed on the display device 10. As a result, the image on which the overlay area is superimposed as shown in FIG.

[0076] [Operation of video transmission device 3] FIG. 9 is a flowchart showing an example of the operation of the video transmission device 3 according to the first embodiment of the present disclosure. The video transmission device 3 acquires a video from the camera 2 (step S1). The video transmitter 3 detects the type and position of the object from the video (step S2). The video transmitting device 3 detects the distance from the robot 1 to the object based on the video (step S3).

[0077] The video transmission device 3 calculates the absolute speed of the object based on the change in the distance to the object (step S4).

[0078] The video transmitting device 3 determines the priority of the object based on the type, position, distance and absolute speed of the object (step S5). The video transmitting device 3 monitors the available bandwidth of the transmission path through which the video transmitting device 3 performs wireless communication (step S6). The video transmitting device 3 calculates the total area of ​​the overlay region based on the available bandwidth (step S7).

[0079] The video transmission device 3 determines the object on which the overlay area is to be superimposed based on the total area of ​​the overlay area and the priority of the object (step S8).

[0080] The video transmission device 3 superimposes an overlay area in the vicinity of the object determined in step S8 in the video (step S9).

[0081] The video transmitting device 3 compresses the video on which the overlay area is superimposed at different compression rates for each area based on the priority of the object and the available bandwidth (step S10). The video transmitting device 3 transmits the compressed video to the video receiving device 9 (step S11).

[0082] [Operation of video receiving device 9] FIG. 10 is a flowchart showing an example of the operation of the video receiving device 9 according to the first embodiment of the present disclosure. The video receiving device 9 receives the compressed video from the video transmitting device 3 (step S21). The video receiving device 9 expands the received compressed video by region (step S22). The video receiving device 9 causes the display device 10 to display the decompressed video (step S23).

[0083] According to the first embodiment, the position and type of the object are detected from the image captured by the camera 2. Therefore, the object can be accurately detected. Also, the overlay area is superimposed in a manner that is associated with the object in the image. Therefore, the worker of the robot 1 can instantly determine which robot 1 the overlay area is for. Therefore, the worker can be provided with accurate information required for at least one of remote monitoring and remote operation of the robot 1.

[0084] Also, for example, when the available bandwidth is small, the resolution of the portion other than the overlay area can be reduced, or the resolution of the background area other than the area of ​​the target object can be reduced. As a result, even if the available bandwidth is reduced, the overlay area can be displayed on the screen without reducing its resolution. Therefore, it is possible to accurately provide the operator of the robot 1 with information necessary for at least one of remote monitoring and remote operation of the robot 1.

[0085] Also, for example, the area or number of overlay areas superimposed on an image can be controlled according to the priority, and the display or non-display of the overlay areas can be controlled. Therefore, it is possible to preferentially display the overlay areas of robots 1 that have a high degree of influence on the object, or the overlay areas of robots 1 that have a low ability to avoid danger to the object. Therefore, the operator of the robot 1 can remotely monitor or remotely operate the robot 1 so as not to collide with these robots 1.

[0086] Furthermore, the video receiving device 9 can display the video with the overlay area superimposed on it on the display device 10 without performing any special processing when decompressing the video. Therefore, the video receiving device 9 can decompress the video using a normal decoder.

[0087] <Embodiment 2> In the first embodiment, the video transmission device 3 superimposes the overlay area on the video. In contrast, in the second embodiment, the video reception device 9 superimposes the overlay area on the video.

[0088] The following description will focus on the differences from the first embodiment, and will not repeat the description of the parts in common with the first embodiment.

[0089] FIG. 11 is a block diagram showing an example of a functional configuration of the video transmission device 3 according to the second embodiment of the present disclosure. The video transmission device 3 includes a video acquisition unit 301, an object detection unit 302, a distance detection unit 303, a speed detection unit 304, a priority determination unit 305, a bandwidth observation unit 306, a video compression unit 321, a metadata multiplexing unit 322, and a video transmission unit 310.

[0090] The video compression unit 321 obtains information on the position and priority of the object from the priority determination unit 305. In addition, the video compression unit 321 obtains information on the available bandwidth from the bandwidth observation unit 306.

[0091] The video compression unit 321 compresses the video acquired by the video acquisition unit 301 at different compression rates for different regions so that the data amount of the compressed video does not exceed the available bandwidth. The video compression method is, for example, H.264 / MPEG-4 AVC, H.265 / MPEG-H HEVC, etc. For example, the video compression unit 321 may change the compression rate by changing the value of a quantization parameter QP.

[0092] The video compression unit 321 compresses the video so that the compression rate is lower for the circumscribing rectangular area of ​​an object with a higher priority. The video compression unit 321 also compresses the video so that the compression rate of the area other than the object is higher than the compression rate of the circumscribing rectangular area of ​​the object. If the data volume of the compressed video exceeds the available bandwidth, the video compression unit 309 increases the compression rate of the area other than the object in stages up to a predetermined compression rate, and if the data volume still exceeds the available bandwidth, increases the compression rate of the object starting from the object with the lowest priority.

[0093] By the video compressor 321 compressing the video at different compression rates for different regions in this way, the video transmitter 3 can transmit the compressed video to the video receiver 9 at or below the available bandwidth.

[0094] The video compression unit 321 adds information on the compression rate for each region to the compressed video and outputs the compressed video to the metadata multiplexing unit 322.

[0095] The metadata multiplexing unit 322 acquires information on the type and position of an object from the object detection unit 302. The metadata multiplexing unit 322 acquires information on the distance of the object from the distance detection unit 303. The metadata multiplexing unit 322 generates metadata including the position, type, and distance of the object for each object detected by the object detection unit 302. The metadata multiplexing unit 322 multiplexes the generated metadata onto the compressed video acquired from the video compression unit 321, and outputs the metadata to the video transmission unit 310.

[0096] The video transmitting unit 310 transmits the compressed video onto which the metadata is multiplexed to the video receiving device 9 via the wireless base station 8 and the network 7 .

[0097] FIG. 12 is a block diagram showing an example of a functional configuration of the video receiving device 9 according to the second embodiment of the present disclosure. The video receiving device 9 includes a video receiving unit 901 , a metadata extraction unit 911 , a video decompression unit 902 , an overlay processing unit 912 , and a display control unit 903 .

[0098] The video receiving unit 901 receives compressed video on which metadata is superimposed from the video transmitting device 3 . The metadata extraction unit 911 extracts metadata from the compressed video received by the video receiving unit 901 . The video decompression unit 902 decompresses the compressed video received by the video reception unit 901 for each region.

[0099] The overlay processing unit 912 superimposes an overlay area on the image after decompression by the image decompression unit 902 based on the metadata extracted by the metadata extraction unit 911. That is, the overlay processing unit 912 superimposes an overlay area including the type and distance of an object indicated by the metadata in the image in a manner that is associated with the object indicated by the metadata in the image. In other words, when an operator of the robot 1 sees the overlay area in the image, the overlay area is superimposed in a manner that allows the operator to recognize which object the overlay area is for. For example, the overlay processing unit 912 may superimpose the overlay area near the object, such as the upper part of the circumscribing rectangle of the object. The overlay area and the object may partially overlap each other. The overlay processing unit 912 may also associate the object with the overlay area by superimposing a line segment connecting the object and the overlay area on the image. In this case, the overlay processor 912 may place the overlay area at a position away from the target object, such as an edge of the image. The overlay processor 912 outputs the image on which the overlay area is superimposed to the display controller 903.

[0100] The display control unit 903 acquires the image on which the overlay area is superimposed from the overlay processing unit 912, and causes the image to be displayed on the screen of the display device 10.

[0101] FIG. 13 is a flowchart showing an example of the operation of the video transmission device 3 according to the second embodiment of the present disclosure. The video transmitting device 3 executes the processes from step S31 to step S36. These processes are similar to the processes from step S1 to step S6.

[0102] The video transmitting device 3 compresses the video acquired from the camera 2 at different compression rates for different areas based on the priority of the object and the available bandwidth (step S37). The video transmitting device 3 multiplexes the metadata onto the compressed video (step S38).

[0103] The video transmitting device 3 transmits the compressed video with the metadata multiplexed therein to the video receiving device 9 (step S39).

[0104] FIG. 14 is a flowchart showing an example of the operation of the video receiving device 9 according to the second embodiment of the present disclosure. The video receiving device 9 receives the compressed video onto which metadata is multiplexed from the video transmitting device 3 (step S41).

[0105] The video receiving device 9 extracts the metadata from the received compressed video (step S42). The video receiving device 9 decompresses the received compressed video (step S43).

[0106] The video receiving device 9 superimposes the overlay area on the decompressed video based on the extracted metadata (step S44).

[0107] The video receiving device 9 causes the display device 10 to display the video on which the overlay area is superimposed (step S45).

[0108] According to the second embodiment, the position and type of the object are detected from the image captured by the camera 2. Therefore, the object can be accurately detected. Also, the overlay area is superimposed in a manner that allows it to be associated with the object in the image. Therefore, the worker of the robot 1 can instantly determine which robot 1 the overlay area is for. Therefore, the worker can be accurately provided with information necessary for at least one of remote monitoring and remote operation of the robot 1. Also, the amount of data of the metadata is very small compared to the image. Therefore, even if the available bandwidth of the network is reduced and the image quality of the image must be reduced, the metadata can be correctly transmitted to the video receiving device 9.

[0109] <Variation 1> In the first and second embodiments, the display mode of the overlay area superimposed on the video image is the same for all of the images. However, the display mode of the overlay area may be different for each object.

[0110] For example, the overlay processing unit 308 of the video transmitting device 3 or the overlay processing unit 912 of the video receiving device 9 may control the display mode of the overlay area based on the type of the object.

[0111] 15 is a diagram showing an example of an image on which an overlay area according to Modification 1 is superimposed. For example, the video transmitting device 3 or the video receiving device 9 may surround the overlay areas 31A and 31B for the adults 6A and 6B with a thick frame to highlight them more than the overlay area 30C for the object 5.

[0112] Note that video transmitting device 3 or video receiving device 9 may change the display mode of the overlay area based on the type of object by differentiating the color of overlay areas 31A, 31B from the color of overlay area 30C.

[0113] According to the first modification, for example, the overlay area of ​​an adult and the overlay area of ​​an object can be displayed in a distinguished manner. Therefore, the operator of the robot 1 can instantly determine whether or not a collision between the robot 1 and the object must be avoided.

[0114] <Variation 2> The overlay processing unit 308 of the video transmitting device 3 or the overlay processing unit 912 of the video receiving device 9 may control the display mode of the overlay area based on the distance from the robot 1 to the target object.

[0115] 16 is a diagram showing an example of an image on which an overlay area according to Modification 2 is superimposed. For example, the video transmitting device 3 or the video receiving device 9 may surround with a thick frame overlay areas 31B, 31C for an adult 6B and an object 5 whose distance from the robot 1 is less than a predetermined threshold (e.g., 20 m) to highlight them more than overlay area 30A whose distance from the robot 1 is equal to or greater than the threshold.

[0116] Note that video transmitting device 3 or video receiving device 9 may change the display mode of the overlay area based on the type of object by differentiating the color of overlay areas 31B and 31C from the color of overlay area 30A.

[0117] According to the second modification, for example, the overlay area of ​​a nearby object and the overlay area of ​​a distant object can be displayed with distinction, so that the operator of the robot 1 can instantly determine whether or not a collision between the robot 1 and an object must be avoided.

[0118] <Modification 3> The overlay processing unit 308 of the video transmitting device 3 or the overlay processing unit 912 of the video receiving device 9 may display an area of ​​an object in a video so as to be distinguishable from areas other than the area of ​​the object. For example, the video transmitting device 3 or the video receiving device 9 may superimpose a rectangular area of ​​the object on the video.

[0119] 17 is a diagram showing an example of an image on which a circumscribing rectangle according to Modification 3 is superimposed. The video transmitting device 3 or the video receiving device 9 superimposes a circumscribing rectangle 20A of an adult 6A, a circumscribing rectangle 20B of an adult 6B, and a circumscribing rectangle 20C of an object 5 on the image, in addition to an overlay area 30A of an adult 6A, an overlay area 30B of an adult 6B, and an overlay area 30C of an object 5.

[0120] According to the third modification, the operator of the robot 1 can instantly determine the area of ​​the target object.

[0121] <Variation 4> The overlay processing unit 308 of the video transmitting device 3 or the overlay processing unit 912 of the video receiving device 9 may control the display of the overlay area based on the distance from the robot 1 to the target object. For example, the video transmitting device 3 or the video receiving device 9 may superimpose the overlay area on the video when the distance from the robot 1 to the target object becomes equal to or less than a predetermined threshold (e.g., 20 m).

[0122] 18 is a diagram showing an example of an image on which a circumscribing rectangle according to Modification 4 is superimposed. When the distance from the robot 1 to the object becomes 20 m or less, the video transmitting device 3 or the video receiving device 9 superimposes an overlay area for the object on the image. As a result, for example, an overlay area 30C for an object 5 that is 2.0 m away from the robot 1 and an overlay area 30B for an adult 6B that is 11.1 m away from the robot 1 are displayed. On the other hand, the overlay area for an adult 6A that is 40.7 m away from the robot 1 is not displayed.

[0123] According to the fourth modification, for example, when the distance to the object is far, the overlay area can be hidden, and when the distance is within a certain distance, the overlay area can be displayed. Therefore, the display of the overlay area can be controlled depending on whether or not the object needs to be avoided.

[0124] <Variation 5> The video transmission device 3 may detect the distance from the robot 1 to an object and the absolute speed of the robot 1 using a millimeter wave radar. The millimeter wave radar is installed at a position on the robot 1 capable of emitting millimeter waves in the direction in which the robot 1 is moving.

[0125] 3, for example, the distance detection unit 303 acquires each image and a circumscribing rectangle position of an object in the image from the object detection unit 302. The distance detection unit 303 calculates the direction in which the object exists from the circumscribing rectangle position, and acquires the distance from the millimeter wave radar to the object located in the direction in which the object exists.

[0126] The speed detection unit 304 obtains each image and the circumscribing rectangular position of the object in the image from the object detection unit 302. The speed detection unit 304 calculates the direction of the object from the circumscribing rectangular position, and obtains the relative speed of the object located in the direction of the object from the millimeter wave radar. The speed detection unit 304 obtains the movement speed of the robot 1 from the robot 1, and calculates the absolute speed of the object based on the movement speed of the robot 1 and the relative speed of the object.

[0127] [Note] Each process (each function) of the above-mentioned embodiment is realized by a processing circuit including one or more processors. The processing circuit may be configured by an integrated circuit or the like in which one or more memories, various analog circuits, and various digital circuits are combined in addition to the one or more processors. The one or more memories store programs (instructions) that cause the one or more processors to execute each of the above processes. The one or more processors may execute each of the above processes according to the programs read from the one or more memories, or may execute each of the above processes according to a logic circuit designed in advance to execute each of the above processes. The processor may be various processors suitable for computer control, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit). The physically separated processors may execute each of the above processes in cooperation with each other. For example, the processors mounted on each of a plurality of physically separated computers may execute the above processes in cooperation with each other via a network such as a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, etc. The program may be installed in the memory from an external server device or the like via the network, or may be distributed in a state stored in a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a semiconductor memory, and may be installed in the memory from the recording medium. Furthermore, at least some of the above-described embodiments and modifications may be combined in any manner.

[0128] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims. [Explanation of symbols]

[0129] 1. Robot (mobile object) 2 Camera 3. Video transmission device (image transmission device) 4 road 5. Things 6A Adult 6B Adult 7 Network 8 Radio base stations 9. Video receiving device (display control device) 10 Display device 20A circumscribed rectangle 20B circumscribed rectangle 20C circumscribed rectangle 30A Overlay Area 30B Overlay Area 30C Overlay Area 31A Overlay Area 31B Overlay Area 31C Overlay Area 100 Video display system (display system) 301 Video Acquisition Unit 302 Object detection unit 303 Distance detection unit 304 Speed ​​detection unit 305 Priority determination section 306 Bandwidth Observation Unit 307 Area calculation part 308 Overlay processing unit (image generation unit) 309 Video Compression Section 310 Video transmission unit (transmission unit) 321 Video Compression Section 322 Metadata Multiplexing Unit 901 Video receiving unit (receiving unit) 902 Video expansion unit 903 Display control unit 911 Metadata Extraction Unit 912 Overlay Processing Unit

Claims

1. A display method for displaying an image for performing at least one of remote monitoring and remote operation of an autonomously traveling mobile body by wireless communication on a screen viewable by a worker performing the operation, comprising: detecting a position and a type of an object present within an area including a moving direction of the moving body based on an image of the area captured by a camera mounted on the moving body; Detecting a distance from the moving body to the target object; and displaying on the screen the image in which an overlay area including the type of object and the distance to the object is superimposed in a manner that corresponds to the object based on the position and type of the detected object, as well as the distance to the object.

2. The display method according to claim 1 , wherein the step of displaying on the screen further comprises displaying on the screen the image whose image quality has been controlled based on an available bandwidth of a wireless transmission path of the image.

3. The display method further includes a step of determining a priority of the object based on at least one of a degree of an effect on the object in the event of a collision of the moving body and a danger avoidance capability of the object; 3. The display method according to claim 1, wherein the step of displaying on the screen further comprises controlling superimposition of the overlay area on the image based on the determined priority.

4. 3. The display method according to claim 1, wherein the step of displaying on the screen further comprises controlling a display mode of the overlay area based on at least one of a type of the object and a distance to the object.

5. The step of detecting the position and type of the object further includes detecting an area of ​​the object; 3. The display method according to claim 1, wherein the step of displaying on the screen further comprises displaying the area of ​​the object on the screen so as to be distinguishable from areas other than the area of ​​the object.

6. 3. The display method according to claim 1, wherein in the step of displaying on the screen, display of the overlay area is controlled based on a distance to the object.

7. an image transmitting device; and a display control device that is connected to the image transmitting device via a network and displays an image on a screen that can be viewed by a worker performing at least one of remote monitoring and remote operation of a mobile body capable of autonomous travel; The image transmitting device includes: an object detection unit that detects the position and type of an object present within an area including a moving direction of the moving body based on the image of the area captured by a camera mounted on the moving body; a distance detection unit that detects a distance from the moving body to the target object; an image generating unit that generates the image in which an overlay area including the type of the object and the distance to the object is superimposed on the object in a manner that the overlay area can be associated with the object, based on the position and type of the object detected and the distance to the object; a transmission unit that transmits the generated image to the display control device, The display control device includes: a receiving unit for receiving the image from the image transmitting device; a display control unit that displays the received image on a screen.

8. an image transmitting device; and a display control device that is connected to the image transmitting device via a network and displays an image on a screen that can be viewed by a worker performing at least one of remote monitoring and remote operation of a mobile body capable of autonomous travel; The image transmitting device includes: an object detection unit that detects the position and type of an object present within an area including a moving direction of the moving body based on the image of the area captured by a camera mounted on the moving body; a distance detection unit that detects a distance from the moving body to the target object; a transmission unit that transmits the image and metadata to the display control device, the metadata includes a location of the object, a type of the object, and a distance to the object; The display control device includes: a receiving unit for receiving the image and the metadata from the image transmitting device; and a display control unit that displays on the screen the image in which an overlay area including the type of the object and the distance to the object is superimposed in a manner that corresponds to the object based on the received image and metadata.

9. An image transmission device that transmits an image to be displayed on a screen viewable by a worker performing at least one of remote monitoring and remote operation of an autonomously traveling moving body to a display control device of the image, an object detection unit that detects the position and type of an object present within an area including a moving direction of the moving body based on the image of the area captured by a camera mounted on the moving body; a distance detection unit that detects a distance from the moving body to the target object; an image generating unit that generates the image in which an overlay area including the type of the object and the distance to the object is superimposed on the object in a manner that the overlay area can be associated with the object, based on the position and type of the object detected and the distance to the object; a transmission unit that transmits the generated image to the display control device.

10. An image transmission device that transmits an image to be displayed on a screen viewable by a worker performing at least one of remote monitoring and remote operation of an autonomously traveling moving body to a display control device of the image, an object detection unit that detects the position and type of an object present within an area including a moving direction of the moving body based on the image of the area captured by a camera mounted on the moving body; a distance detection unit that detects a distance from the moving body to the target object; a transmission unit that transmits the image and metadata to the display control device, The metadata includes a location of the object, a type of the object, and a distance to the object.

11. A display control device that displays an image on a screen that can be viewed by a worker performing at least one of remote monitoring and remote operation of an autonomously traveling mobile body, a receiving unit that receives from an image transmitting device the image of an area including the moving direction of the moving object captured by a camera mounted on the moving object, and metadata including the position of an object present in the area, the type of the object, and the distance to the object; a display control unit that displays on the screen the image in which an overlay area including the type of object and the distance to the object is superimposed in a manner that corresponds to the object based on the received image and metadata.

12. Computer, A computer program for causing the image transmitting device according to claim 9 or 10 to function as the display control device according to claim 11.

Images