Video aggregation computer, observation device, video aggregation method, and program

Abstract

This video aggregation computer (1) is a computer for displaying, in a terminal (3), videos from a plurality of observation devices (2) that are dispersed. The video aggregation computer (1) estimates network available bands for a terminal (3), allocates a communication band to each of the plurality of observation devices (2) according to resources of the terminal or the network available bands estimated by the estimation unit, receives videos from the plurality of observation devices (2) in the communication bands allocated to the plurality of observation devices (2), and transmits the received videos to the terminal (3).

Description

Video aggregation computer, observation device, video aggregation method, and program

[0001] The present invention relates to a video aggregating computer, an observation device, a video aggregating method, and a program.

[0002] Conventionally, at particularly dangerous sites such as construction sites, work machines and the like have been remotely operated. At such sites, imaging devices such as fixed cameras installed at multiple locations, cameras mounted on drones, and mobile cameras installed on portable devices, etc., are used to provide work support to workers, managers, and other related parties in various remote locations with information about the site situation (e.g., Patent Documents 1 and 2).

[0003] International Publication No. 2021 / 070214 Pamphlet Patent No. 7029586 Specification

[0004] The object of the present invention is to provide an image aggregation computer, observation device, image aggregation method, and program that, when displaying images from multiple observation devices on a terminal, can cause the terminal to display images appropriate to the environment, even in an environment where there is a limit to the amount of data transmission that can be used for the images.

[0005] The provided video aggregation computer is a computer for displaying videos from multiple distributed observation devices on a terminal, and includes an estimation unit that estimates a network available bandwidth for the terminal, an allocation unit that allocates communication bandwidth to the multiple observation devices in accordance with the network available bandwidth estimated by the estimation unit or the resources of the terminal, and a transmission unit that receives videos from the multiple observation devices using the communication bandwidth allocated to the multiple observation devices and transmits the received videos to the terminal.

[0006] FIG. 1 is a diagram for explaining an overview of a system including a video aggregation computer 1, an observation device 2, and a terminal 3 according to one embodiment of the present invention. FIG. 2 is a configuration diagram of a system including a video aggregation computer 1, an observation device 2, and a terminal 3 according to this embodiment. FIG. 3 is a flowchart of video switching processing executed by the video aggregation computer 1 according to this embodiment. FIG. 4 is a diagram for explaining an example of video switching that the video aggregation computer 1 according to this embodiment causes to be displayed (output) on the terminal 3. FIG. 5 is a diagram for explaining another example of video switching that the video aggregation computer 1 according to this embodiment causes to be displayed (output) on the terminal 3. FIG. 6 is a flowchart for explaining video transmission processing executed by the video aggregation computer 1 according to this embodiment. FIG. 7 is a flowchart for explaining video distribution processing executed by the observation device 2 constituting this embodiment.

[0007] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same elements are designated by the same numbers or symbols throughout the description of the embodiments.

[0008] [Basic Concept / Basic Configuration] Fig. 1 is a diagram for explaining an overview of a video aggregation system that is a system including a video aggregation computer 1 according to one embodiment of the present invention, a plurality of observation devices 2, and a terminal (information terminal) 3. Hereinafter, the video aggregation computer 1 may be referred to simply as computer 1.

[0009] As shown in FIG. 1 , the video aggregation computer 1 may function as, for example, a media server. The computer 1 is connected to an observation device 2 and a terminal 3 for data communication via the Internet, a network such as a mobile phone network, a network constructed by software virtualization, or the like. The number of each observation device 2 and terminal 3 may be one or more. Two or more observation devices 2 may be connected to the computer 1 simultaneously. The observation device 2 and terminal 3 may be connected to the computer 1 simultaneously. The system shown in FIG. 1 includes multiple observation devices 2. The multiple observation devices 2 may be installed at a single site or may be installed separately at multiple sites. The observation device 2 may be, for example, an imaging device such as a video camera that observes the remote operation status of a remotely controlled work machine such as a robot arm, shovel, crane, bulldozer, or transporter. The multiple observation devices 2 may include an observation device 2 mounted on a mobile object. The mobile object may be a mobile work machine such as a shovel or crane, a vehicle such as an automobile, or an aerial vehicle such as a drone. The terminal 3 acquires and displays the images captured by the observation device 2 via the image aggregation computer 1.

[0010] The video aggregation computer 1 may be an on-premise computer or computing system such as an on-premise server or on-premise computing system, or may be a cloud computer or computing system such as a cloud server or cloud computing system. In this embodiment, the video aggregation computer 1 is a cloud computing system. As will be described later, the computer 1 may be a personal computer, a computer installed in a mobile terminal, or a computer installed in a wearable terminal.

[0011] The observation device 2 may be a device such as a camera, a temperature sensor, or a metal detector. The video captured by the observation device 2 may be video data, image data, numerical data, or other data. The video aggregation computer 1 may acquire the data from the observation device 2 via the network, or may acquire the data via a system (not shown) that controls remote operations other than the observation device 2.

[0012] The terminal 3 is a terminal capable of transmitting and receiving data between the terminal 3 and the video aggregation computer 1. The terminal 3 may be, for example, an electronic device such as a laptop computer, a desktop computer, a smartphone, or a tablet terminal.

[0013] As shown in Fig. 1, the computer 1 has at least one video channel 4. The video channel 4 includes a plurality of ports 5. Each of the plurality of ports 5 may be assigned a unique port number (identifier). The video channel 4 may be, for example, a data channel in a peer-to-peer communication connection method. An example of communication based on a peer-to-peer communication connection method is WebRTC (Web Real-Time Communication).

[0014] The video aggregation computer 1 estimates the network available bandwidth for the remote user's terminal 3. The network available bandwidth is the bandwidth (unit: "bps" (bits per second)) available on the communication path between the video aggregation computer 1 and the terminal 3. The method by which the video aggregation computer 1 estimates the network available bandwidth is not particularly limited, and any known method can be adopted. Specifically, the network available bandwidth estimation method may be an active measurement method, a passive measurement method, or another method. The active measurement estimation method is a method in which a test packet is sent from a sending host to a receiving host and the available bandwidth is estimated based on changes in the probe interval between sending and receiving due to queuing delay. The passive measurement estimation method is a method in which packet information is collected at a specified measurement point on the communication path and the available bandwidth is estimated based on the packet information.

[0015] In addition, the video aggregation computer 1 may estimate the network available bandwidth based on information from the terminal 3, may estimate the network available bandwidth based on input by a user to the input section of the terminal 3, may estimate the network available bandwidth using a trained model constructed by machine learning, or may estimate the network available bandwidth using an existing tool for estimating the network available bandwidth.

[0016] The video aggregation computer 1 allocates a communication bandwidth to each of the multiple observation devices 2 according to the estimated available network bandwidth or the resources of the terminal 3. The multiple observation devices 2 to which the communication bandwidth is allocated may be a portion of the large number of observation devices 2, or may be all of the large number of observation devices 2. The unit of communication bandwidth is "bps" (bits per second). Here, "allocating a communication bandwidth to an observation device 2" may mean allocating a communication bandwidth to data (e.g., video data) transmitted by the observation device 2.

[0017] For example, if the resource performance of terminal 3 is high, specifically, if terminal 3 can process data with a data volume corresponding to the available network bandwidth, the video aggregation computer 1 may allocate a communication bandwidth to each of the multiple observation devices 2 according to the available network bandwidth. On the other hand, if the resource performance of terminal 3 is low, specifically, if terminal 3 cannot process data with a data volume corresponding to the available network bandwidth, the video aggregation computer 1 may allocate a communication bandwidth to each of the multiple observation devices 2 according to the resource performance of terminal 3. The data with a data volume corresponding to the available network bandwidth is received data transmitted from the video aggregation computer 1 and received at terminal 3. The received data may be actual data (real data) actually transmitted from computer 1 to terminal 3, or may be virtual data (virtual data) assumed to be transmitted from the video aggregation computer 1 to terminal 3. The received data may be data with the same data volume (bps) as the network available bandwidth (bps), or may be data with a data volume (bps) set based on the data volume (bps) of the network available bandwidth but smaller than the network available bandwidth, or may be data with a data volume (bps) set based on the data volume (bps) of the network available bandwidth but larger than the network available bandwidth.

[0018] The resource performance of terminal 3 may be, for example, the performance of the arithmetic processing unit of terminal 3. The performance of the arithmetic processing unit may be, for example, the processing speed of the arithmetic processing unit. The arithmetic processing unit may include one or both of a CPU and a GPU, which will be described later, or may include other arithmetic processing units. Furthermore, the resource performance of terminal 3 may include, for example, the performance of the arithmetic processing unit of terminal 3 and the memory capacity of terminal 3. The resource performance of terminal 3 may include, for example, the performance of a graphics board equipped with a GPU. The resource performance of terminal 3 may include, for example, the screen size of the display device (monitor) of terminal 3. The resource performance of terminal 3 may include, for example, at least one of the supported frame rate and supported resolution of the display device (monitor) of terminal 3. The resource performance of terminal 3 may include, for example, at least one of the type of browser and the version of the browser installed on terminal 3.

[0019] The amount of data that a terminal 3 can process correlates with the resource performance of the terminal 3. Therefore, the video aggregation computer 1 allocates a communication bandwidth to each of the multiple observation devices 2 according to the available network bandwidth or according to the resource performance of the terminal 3, i.e., the amount of data that can be processed by the terminal 3. In other words, the video aggregation computer 1 allocates a communication bandwidth to each of the multiple observation devices 2 according to the available network bandwidth or according to the resource performance of the terminal 3 that can process the received data.

[0020] Specifically, if the performance of a processing unit such as a CPU (Central Processing Unit) of terminal 3 is low and terminal 3 is unable to process the received data, the video aggregation computer 1 may allocate a communication bandwidth to each of the multiple observation devices 2 in accordance with the performance of the processing unit of terminal 3. On the other hand, if the performance of the processing unit of terminal 3 is high and terminal 3 is able to process the received data, the video aggregation computer 1 may allocate a communication bandwidth to each of the multiple observation devices 2 in accordance with the estimated network available bandwidth.

[0021] The video aggregation computer 1 instructs (notifies) the observation devices 2 corresponding to each video to increase the image quality of the video to which a wide bandwidth communication band (high communication band) is assigned and to decrease the image quality of the video to which a narrow bandwidth communication band (low communication band) is assigned.The observation devices 2 then provide the video aggregation computer 1 with video of the image quality corresponding to the instruction (notification).

[0022] The video aggregation computer 1 may be configured to receive the importance of the video received at each port 5. The importance may be the level of attention to the video by the user of the terminal 3. The computer 1 may set the importance of the video based on information received from the observation device 2, based on input by the user to the input unit of the terminal 3, or may estimate and set the importance using a trained model constructed by machine learning.

[0023] The video aggregation computer 1 may allocate a communication bandwidth to each of the multiple observation devices 2 according to the available network bandwidth and importance, or according to the performance and importance of the resources of the terminal 3 .

[0024] For example, if the resource performance of terminal 3 is high, specifically, if terminal 3 can process the received data, the video aggregation computer 1 may allocate a communication bandwidth to each of the multiple observation devices 2 in accordance with the available network bandwidth and importance. On the other hand, if the resource performance of terminal 3 is low, specifically, if terminal 3 cannot process the received data, the video aggregation computer 1 may allocate a communication bandwidth to each of the multiple observation devices 2 in accordance with the resource performance and importance of terminal 3.

[0025] The video aggregation computer 1 allocates a wide bandwidth communication band (high communication band) to the observation device 2 that captures video of high importance, and allocates a narrow bandwidth communication band (low communication band) to the observation device 2 that captures video of low importance. The video aggregation computer 1 may issue instructions (notifications) to the observation device 2 corresponding to each video to increase the image quality of the video allocated with a wide bandwidth communication band (high communication band) and decrease the image quality of the video allocated with a narrow bandwidth communication band (low communication band). The observation device 2 then provides the video aggregation computer 1 with video of a quality corresponding to the instructions (notifications).

[0026] The video aggregation computer 1 receives, for example, video from multiple observation devices 2 observing a site X on a video channel 4 of the computer 1 via a communication band allocated to the multiple observation devices 2. The computer 1 may have only a single video channel 4, or multiple channels 4 including a first video channel 4 and a second video channel 4. In this case, the computer 1 may acquire video from the multiple observation devices 2 observing the site X on the first video channel 4, and video from the multiple observation devices 2 observing other sites on the second video channel 4. The video channel 4 may be composed of multiple ports 5 set according to the communication band. The video aggregation computer 1 receives video from each observation device 2 on the port 5 via the communication band allocated to that observation device 2.

[0027] The video aggregation computer 1 receives video transmitted from multiple observation devices 2, transmits the received video to terminal 3, and causes terminal 3 to display (output) the received video on terminal 3. Specifically, by opening each port 5 of the video channel 4 to terminal 3, terminal 3 can acquire video with image quality corresponding to the communication bandwidth allocated by the video aggregation computer 1 and display the acquired video on the screen of terminal 3. The video aggregation computer 1 may cause terminal 3 to display (output) multiple videos received from multiple observation devices 2 on the screen of a single display device of terminal 3, i.e., on a single screen, as shown in FIG. 1. Furthermore, if terminal 3 has multiple display devices, the video aggregation computer 1 may cause terminal 3 to display (output) multiple videos received from multiple observation devices 2 across multiple screens of multiple display devices.

[0028] The video aggregation computer 1 may enlarge and display on the terminal 3 the video (A1, E1) received from the observation device 2 assigned the widest bandwidth communication band (highest communication band), as in the display example of the terminal 3 in FIG. 1 . Specifically, the computer 1 may output to the terminal 3 the video received from the observation device 2 assigned the widest bandwidth communication band (highest communication band) at a larger size than other videos received from other observation devices 2. When displaying video of site X on the terminal 3, the computer 1 may display (output) on the terminal 3 the video A1 or video E1 received from the observation device 2 assigned the widest bandwidth communication band (highest communication band) (e.g., 10 Mbps) at a larger size than the video received from the observation device 2 assigned a lower communication band (e.g., 1 Mbps), as shown in FIG. 1 . Furthermore, the computer 1 may not display on the terminal 3 the video received from the observation device 2 assigned a low communication band (narrow communication band). In this embodiment, the video channel 4 may or may not be divided into a video acquisition channel and a video output channel.

[0029] When the computer 1 enlarges and displays the image (A1, E1) received from the observation device 2 to which the widest bandwidth communication band is assigned, the computer 1 may use upscaling technology to convert the resolution of the image (original resolution) to a higher resolution. This can prevent pixelation, block noise, and other problems from occurring in the enlarged and displayed image (A1, E1). This can prevent a decrease in the visibility of the enlarged and displayed image (A1, E1). Note that a known method can be used as the upscaling technology.

[0030] 2 is a configuration diagram of a video aggregation system that includes a video aggregation computer 1 of this embodiment, an observation device 2, and a terminal 3. The video aggregation computer 1 may be realized, for example, by one terminal device or by multiple terminal devices.

[0031] As described above, the computer 1 may be an on-premise computer or computing system, or a cloud-based computer or computing system. The computer 1 may also be a personal computer such as a desktop computer or a laptop computer. The video aggregation computer 1 may also be a computer installed in a mobile terminal such as a handheld terminal, smartphone, or tablet terminal, or a computer installed in a wearable terminal such as smart glasses, a head-mounted display, or a smart watch. In this case, the computer 1 may be equipped with a camera or other imaging device that captures color video and / or still images.

[0032] The video-intensive computer 1 includes an arithmetic processing unit and a memory. Examples of the arithmetic processing unit include a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). Examples of the memory include a RAM (Random Access Memory) and a ROM (Read Only Memory). The computer 1 includes a control unit. The functions of the control unit are realized by the arithmetic processing unit executing a program stored in the memory. As shown in FIG. 2 , the control unit includes a processing unit, a communication unit, a storage unit, an input unit, and an output unit. The control unit issues execution commands to the processing unit, communication unit, input unit, output unit, storage unit, etc., and the processing unit calculates data and determines the calculation results.

[0033] The video aggregation computer 1 includes a communication unit that is a device that enables the computer 1 to communicate with other devices such as the terminal 3 and the observation device 2. The communication method may be wireless or wired.

[0034] The video-intensive computer 1 has, as an input unit, functions necessary for a user to operate the video-intensive computer 1. The computer 1 includes an input device for realizing input. The computer 1 can be equipped with, for example, an LCD display that realizes a touch panel function, a keyboard, a mouse, a pen tablet, hardware buttons on the device, a microphone for voice recognition, and the like as input devices. The input unit of the computer 1 is not limited to the input methods described above.

[0035] The video-intensive computer 1 has, as an output unit, functions necessary for a user to operate the video-intensive computer 1. The computer 1 includes an output device for realizing output. The computer 1 may, for example, be equipped with a display device or an audio output device as the output device. Examples of the display device include a liquid crystal display, a PC display, a projector, a head-mounted display, etc., and examples of the audio output device include a speaker, etc. The output unit of the computer 1 is not limited to the output methods described above.

[0036] The video aggregation computer 1 includes a data storage (recording medium) such as a hard disk, semiconductor memory, or memory card as a memory unit. The data may be stored in a cloud service, a database, or the like. The memory unit may store, for example, some or all of the video acquired from the observation device 2 as data. In this case, the computer 1 may be configured to be able to search for video data stored in the memory unit.

[0037] The processing unit of the control unit includes an estimation unit 10, an allocation unit 11, and a determination unit 12. The communication unit of the control unit includes an importance acceptance unit 20, an instruction unit 21, a reception unit 22, and a transmission unit 23. The memory unit of the control unit includes a video storage unit 30.

[0038] The observation device 2 may be, for example, a camera, a temperature sensor, a metal detector, or the like. The observation device 2 is equipped with a camera unit that captures video. The observation device 2 may also be, for example, a mobile terminal such as a handheld terminal, smartphone, or tablet terminal, or a wearable terminal such as smart glasses, a head-mounted display, or a smartwatch. The multiple observation devices 2 shown in FIG. 1 may include an observation device 2 attached to a mobile body for observing the remote operation status of the work machine, or may include an observation device 2 attached to the work machine itself. Two or more observation devices 2 may be attached to one mobile body, and two or more observation devices 2 may be attached to one work machine. In this case, the two or more observation devices 2 are disposed at positions distant from each other on the one mobile body or the one work machine. The multiple observation devices 2 shown in FIG. 1 may also include one or more observation devices 2 installed on-site for observing the remote operation status of the work machine. The multiple observation devices 2 shown in FIG. 1 may also include one or more observation devices 2 installed on-site for observing the remote operation status of the mobile body. The multiple observation devices 2 shown in Figure 1 include one or more observation devices 2 attached to a work machine, an observation device 2 attached to a mobile body, and one or more observation devices 2 fixed to the work site.

[0039] The observation device 2 includes a processing unit and a memory. Examples of the processing unit include a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). Examples of the memory include a RAM (Random Access Memory) and a ROM (Read Only Memory). The observation device 2 includes a control unit. The functions of the control unit are realized by the processing unit executing a program stored in the memory. As shown in FIG. 2 , the control unit of the observation device 2 includes a processing unit, a communication unit, a memory unit, an imaging unit, an input unit, and an output unit. The control unit issues execution commands to the processing unit, imaging unit, communication unit, input unit, output unit, memory unit, etc., and the processing unit calculates data and determines the calculation results.

[0040] The observation device 2 includes a communication unit that is a device that enables the observation device 2 to communicate with other devices such as the computer 1 and the terminal 3. The communication method may be wireless or wired.

[0041] The observation device 2 has an input unit that is equipped with functions necessary for a user to operate the observation device 2. The observation device 2 includes an input device for realizing input. The observation device 2 can be equipped with, for example, an LCD display that realizes a touch panel function, a keyboard, a mouse, a pen tablet, hardware buttons on the device, a microphone for voice recognition, etc. as input devices. The input unit of the observation device 2 is not limited to the input methods described above.

[0042] The observation device 2 has, as an output unit, functions necessary for a user to operate the observation device 2. The observation device 2 includes an output device for realizing output. The observation device 2 may, for example, be equipped with a display device or an audio output device as the output device. Examples of the display device include a liquid crystal display, a PC display, a projector, etc., and examples of the audio output device include a speaker, etc. The output unit of the observation device 2 is not limited to the output methods described above.

[0043] The observation device 2 includes a storage unit such as a hard disk, a semiconductor memory, a memory card, or other data storage (recording medium). The data may be stored in a cloud service, a database, or the like.

[0044] The processing unit of the control unit of the observation device 2 includes an information detection unit 40, an importance setting unit 41, an image quality setting unit 42, an acceleration detection unit 43, and a learning unit 44. The communication unit of the control unit includes an importance providing unit 50, an instruction receiving unit 51, and an image providing unit 52.

[0045] Terminal 3 is a terminal for a remote user. If the object of remote operation by the remote user is, for example, a work machine, terminal 3 may be located at a location remote from the work site where the work machine is located, and may be equipped with a display device that displays an image of the work site including the work machine. Note that the remote user is not limited to an operator who remotely operates the work machine as described above. For example, the remote user may be a person involved in the work who uses terminal 3 located at a location remote from the work site where the work machine is located to check the status of work performed by the work machine, monitor the work, manage the work, and guard the work.

[0046] The terminal 3 may be, for example, a mobile terminal such as a handheld terminal, a smartphone, or a tablet terminal, or a wearable terminal such as smart glasses, a head-mounted display, or a smart watch, or may be another type of information terminal. The terminal 3 may be equipped with a photographing device such as a camera that captures images such as color moving images and / or still images.

[0047] The terminal 3 includes an arithmetic processing unit and a memory. Examples of the arithmetic processing unit include a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). Examples of the memory include a RAM (Random Access Memory) and a ROM (Read Only Memory). The terminal 3 includes a control unit. The functions of the control unit are realized by the arithmetic processing unit executing a program stored in the memory. As shown in FIG. 2 , the control unit includes a processing unit, a communication unit, an input unit, an output unit, and a storage unit. The control unit issues execution commands to the processing unit, communication unit, input unit, output unit, storage unit, etc., and the processing unit calculates data and determines the calculation results.

[0048] The terminal 3 includes a communication unit that enables the terminal 3 to communicate with other devices such as the computer 1 and the observation device 2. The communication method may be wireless or wired.

[0049] The terminal 3 has, as an input unit, functions necessary for a user to operate the terminal 3. The terminal 3 includes an input device for realizing input. The terminal 3 can be equipped with, for example, an LCD display realizing a touch panel function, a keyboard, a mouse, a pen tablet, hardware buttons on the device, a microphone for voice recognition, etc. as input devices. The input unit of the terminal 3 is not limited to the input methods described above.

[0050] The terminal 3 has, as an output unit, functions necessary for a user to operate the terminal 3. The terminal 3 includes an output device for realizing output. The terminal 3 may have, for example, a display device or an audio output device as the output device. Examples of the display device include a liquid crystal display, a PC display, a projector, etc., and examples of the audio output device include a speaker, etc. The output unit of the terminal 3 is not limited to the output methods described above.

[0051] The terminal 3 includes a storage unit such as a hard disk, a semiconductor memory, a memory card, or other data storage (recording medium). The data may be stored in a cloud service, a database, or the like.

[0052] Furthermore, some or all of the functions of the image-collecting computer 1 may be implemented in the observation device 2 or the terminal 3 by software virtualization.

[0053] The above is the basic concept and basic configuration of the video aggregation system, which is a system including the video aggregation computer 1 (distributed video aggregation computer), the observation device 2, and the terminal 3.

[0054] [Video Switching Process] The video switching process executed by the video aggregation computer 1 of this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart of the video switching process executed by the video aggregation computer 1.

[0055] The estimation unit 10 of the video aggregation computer 1 estimates the network available bandwidth for the terminal 3 of the remote user (step S1). Specifically, the estimation unit 10 may estimate the network available bandwidth, which is the bandwidth available on the communication path between the video aggregation computer 1 and the terminal 3 communicatively connected to the video aggregation computer 1, using an estimation method such as the active measurement method or passive measurement method described above. The estimation unit 10 may also estimate the network available bandwidth based on information about the network to which the video aggregation computer 1 or the terminal 3 is communicatively connected, based on input of information about the network by a user to the input unit of the video aggregation computer 1 or the input unit of the terminal 3, using a trained model constructed by machine learning, or using an existing tool.

[0056] The estimation unit 10 of the video aggregation computer 1 may estimate resource performance of the terminal 3 of a remote user. Specifically, the estimation unit 10 may estimate resource performance based on terminal information of the terminal 3 communicatively connected to the video aggregation computer 1. The terminal information of the terminal 3 may include, for example, information regarding the performance of the CPU of the terminal 3. The terminal information of the terminal 3 may be pre-stored in a storage unit of the computer 1. The video aggregation computer 1 may estimate resource performance of the terminal 3 based on input of the terminal information by a user to an input unit of the video aggregation computer 1 or an input unit of the terminal 3, or may estimate resource performance of the terminal 3 using a trained model constructed by machine learning, or may estimate resource performance of the terminal 3 using an existing tool. Note that if the resource performance of the terminal 3 is pre-stored in the computer 1, the above-described process by the estimation unit 10 to estimate resource performance of the terminal 3 may be omitted.

[0057] The importance level receiving unit 20 of the video aggregation computer 1 receives the importance level of the video (step S2). Specifically, the importance level receiving unit 20 may receive the importance level of the video captured by the observation device 2 from the observation device 2 communicatively connected to the video aggregation computer 1 via the network. The type of data indicating the importance level is not particularly limited and may be various data, but numerical data such as a score representing the importance level of the video is preferable. The importance level receiving unit 20 may receive, as the importance level of the video, a measurement value detected by a sensor (e.g., the observation device 2) or information such as an abnormality detected by the sensor. The importance level receiving unit 20 may also receive, as the importance level of the video, an input by a user to the input unit of the video aggregation computer 1 or the input unit of the terminal 3. Note that the processing of step S2 can be omitted if the allocation unit 11 does not use the importance level when allocating communication bandwidth in step S3 described below.

[0058] Next, the allocation unit 11 of the video aggregation computer 1 allocates a communication bandwidth to each of the observation devices 2 according to the available network bandwidth estimated by the estimation unit 10 in step S1 or the resource performance of the terminal 3 (according to the resource performance capable of processing the received data) (step S3). The allocation unit 11 may allocate the communication bandwidth using an algorithm such as a predetermined allocation algorithm. The type of algorithm is not particularly limited.

[0059] In step S3, the allocation unit 11 determines whether the terminal 3 is capable of processing the received data. The received data may be video data (video data) transmitted from the video aggregation computer 1 and received at the terminal 3. The received data may be actual video data (real data) actually transmitted from the computer 1 to the terminal 3, or may be virtual video data (virtual data) assumed to be transmitted from the video aggregation computer 1 to the terminal 3. In other words, the allocation unit 11 may determine whether the terminal 3 is capable of processing the amount of the actual data, or may determine whether the terminal 3 is capable of processing the amount of the virtual data.

[0060] For example, the allocation unit 11 may determine whether the CPU of the terminal 3 is capable of processing the received data based on the performance of the CPU of the terminal 3. In this embodiment, the allocation unit 11 determines whether the terminal 3 is capable of processing the received data based on the resource performance estimated in step S1. If the allocation unit 11 determines that the terminal 3 is capable of processing the received data, it allocates a communication bandwidth to each of the multiple observation devices 2 in accordance with the network available bandwidth estimated by the estimation unit 10 in step S1. On the other hand, if the allocation unit 11 determines that the terminal 3 is not capable of processing the received data, it allocates a communication bandwidth to each of the multiple observation devices 2 in accordance with the resource performance of the terminal 3 (in accordance with the performance of the resources that can process the received data).

[0061] Specifically, for example, the storage unit of the computer 1 may store in advance a map such as a mathematical formula or a table that represents the relationship between the resource performance of the terminal 3 (e.g., the performance of the arithmetic processing unit of the terminal 3) and the amount of data that can be processed by the terminal 3. In this case, the allocation unit 11 may recognize the amount of data that the terminal 3 can process based on the resource performance of the terminal 3 and the map, and may compare the recognized amount of data with the received data to determine whether the terminal 3 can process the received data.

[0062] The received data may be data with a data volume corresponding to the network available bandwidth. In this case, the received data may be data with the same data volume (unit: bps) as the network available bandwidth (unit: bps), data with a data volume smaller than the network available bandwidth, or data with a data volume larger than the network available bandwidth. The data volume of the received data may be a value calculated by, for example, substituting the value of the network available bandwidth estimated in step S1 into a map such as a predetermined mathematical formula or table.

[0063] If the received data is data with a data volume that corresponds to the network available bandwidth, in step S3, the allocation unit 11 determines whether the terminal 3 can process the received data, which is data with a data volume that corresponds to the network available bandwidth, based on the performance of the resources of the terminal 3.

[0064] When the allocating unit 11 determines that the terminal 3 can process the received data, it allocates a communication bandwidth to each of the multiple observation devices 2 according to the available network bandwidth or according to the network bandwidth and the importance. That is, the allocating unit 11 allocates a communication bandwidth to each of the multiple observation devices 2 within the available network bandwidth. Specifically, for example, the allocating unit 11 may allocate a communication bandwidth to each of the multiple observation devices 2 so that the total bandwidth of the communication bandwidth allocated to the multiple observation devices 2 is the same as the available network bandwidth. Alternatively, the allocating unit 11 may allocate a communication bandwidth to each of the multiple observation devices 2 so that the total bandwidth of the communication bandwidth allocated to the multiple observation devices 2 is smaller than the available network bandwidth.

[0065] On the other hand, if the allocating unit 11 determines that the terminal 3 cannot process the received data, it allocates a communication band to each of the multiple observation devices 2 according to the resource performance of the terminal 3, or according to the resource performance of the terminal 3 and the importance. That is, the allocating unit 11 allocates a communication band to each of the multiple observation devices 2 within the range of the amount of data that the terminal 3 can process. Specifically, for example, the allocating unit 11 may allocate a communication band to each of the multiple observation devices 2 so that the total bandwidth of the communication band allocated to the multiple observation devices 2 is the same as the amount of data that the terminal 3 can process. The allocating unit 11 may allocate a communication band to each of the multiple observation devices 2 so that the total bandwidth of the communication band allocated to the multiple observation devices 2 is smaller than the amount of data that the terminal 3 can process.

[0066] [Case in which Importance is Not Used] A specific example in which importance is not used in allocating communication bands will be described.

[0067] If the network available bandwidth estimated in step S1 is, for example, "16 Mbps" and the amount of data that the terminal 3 can process is, for example, "20 Mbps," the allocating unit 11 determines in step S3 that the terminal 3 can process the received data of a data amount (for example, 16 Mbps) that corresponds to the network available bandwidth, and allocates a communication bandwidth to each of the multiple observation devices 2 in accordance with the network available bandwidth. That is, the allocating unit 11 allocates a communication bandwidth to each of the multiple observation devices 2 within the range of the network available bandwidth (a range of 16 Mbps or less).

[0068] For example, the allocating unit 11 may equally allocate the value (16 Mbps / n) obtained by dividing the available network bandwidth by the number "n" of the plurality of observation devices 2 as the communication bandwidth of each observation device 2. Furthermore, if the priorities of the plurality of observation devices 2 are determined, the allocating unit 11 may allocate the communication bandwidth to each of the plurality of observation devices 2 so that the communication bandwidth allocated to the n observation devices 2 has a size according to the priorities. Specifically, the allocating unit 11 may allocate the communication bandwidth to each of the plurality of observation devices 2 so that the communication bandwidth of the observation device 2 with the highest priority is larger than the communication bandwidths of the other observation devices 2.

[0069] If the network available bandwidth estimated in step S1 is, for example, "16 Mbps" and the amount of data that terminal 3 can process is, for example, "10 Mbps," then in step S3, the allocating unit 11 determines that terminal 3 cannot process the received data of a data amount (for example, 16 Mbps) that corresponds to the network available bandwidth, and allocates a communication bandwidth to each of the multiple observation devices 2 in accordance with the resource performance of terminal 3. That is, the allocating unit 11 allocates a communication bandwidth to each of the multiple observation devices 2 within the range of the amount of data that terminal 3 can process (a range of 10 Mbps or less).

[0070] For example, the allocating unit 11 may equally allocate the value (10 Mbps / n) obtained by dividing the amount of data that the terminal 3 can process by the number "n" of the plurality of observation devices 2 as the communication bandwidth of each observation device 2. Furthermore, if the priorities of the plurality of observation devices 2 are determined, the allocating unit 11 may allocate the communication bandwidth to each of the plurality of observation devices 2 so that the communication bandwidth allocated to the n observation devices 2 has a size according to the priorities. Specifically, the allocating unit 11 may allocate the communication bandwidth to each of the plurality of observation devices 2 so that the communication bandwidth of the observation device 2 with the highest priority is larger than the communication bandwidths of the other observation devices 2.

[0071] [Cases in which Importance is Used] Next, a specific example in which importance is used in allocating a communication band will be described.

[0072] The allocation unit 11 of the video aggregation computer 1 may allocate a communication bandwidth to each of the multiple observation devices 2 according to the available network bandwidth estimated by the estimation unit 10 in step S1 and the importance level accepted by the importance level acceptance unit 20 in step S2, or according to the resource performance of the terminal 3 (the amount of data that the terminal 3 can process) and the importance level. Specifically, the allocation unit 11 may allocate a communication bandwidth using an algorithm such as a predetermined allocation algorithm. Note that the type of algorithm is not particularly important. The allocation unit 11 allocates a wide bandwidth communication bandwidth (high communication bandwidth) to the observation device 2 that captures the accepted video with high importance, and allocates a narrow bandwidth communication bandwidth (low communication bandwidth) to the observation device 2 that captures the accepted video with low importance.

[0073] If the network available bandwidth estimated in step S1 is, for example, "16 Mbps" and the amount of data that the terminal 3 can process is, for example, "20 Mbps," the allocating unit 11 determines in step S3 that the terminal 3 can process the received data of a data amount (for example, 16 Mbps) that corresponds to the network available bandwidth, and allocates a communication bandwidth to each of the multiple observation devices 2 according to the network available bandwidth and the importance. That is, the allocating unit 11 allocates a communication bandwidth to each of the multiple observation devices 2 according to the importance within the range of the network available bandwidth (a range of 16 Mbps or less).

[0074] Specifically, for example, the allocation unit 11 may allocate a communication bandwidth to each of multiple observation devices 2 so that the bandwidth of the communication bandwidth of the observation device 2 that captures the video with the highest importance is wider than the bandwidth of the communication bandwidth of the other observation devices 2 within the range of the network available bandwidth (within a range of 16 Mbps or less).

[0075] If the network available bandwidth estimated in step S1 is, for example, "16 Mbps" and the amount of data that terminal 3 can process is, for example, "10 Mbps," then in step S3, the allocation unit 11 determines that terminal 3 cannot process the received data of a data amount (for example, 16 Mbps) that corresponds to the network available bandwidth, and allocates a communication bandwidth to each of the multiple observation devices 2 according to the importance and the resource performance of terminal 3. That is, the allocation unit 11 allocates a communication bandwidth to each of the multiple observation devices 2 according to the importance within the range of the amount of data that terminal 3 can process (a range of 10 Mbps or less).

[0076] Specifically, for example, the allocation unit 11 may allocate a communication bandwidth to each of multiple observation devices 2 so that the bandwidth of the communication bandwidth of the observation device 2 capturing the video with the highest importance is wider than the bandwidth of the communication bandwidth of the other observation devices 2, within the range of the amount of data that the terminal 3 can process (within a range of 10 Mbps or less).

[0077] [Number of Observation Devices to Which Communication Bandwidth is Allocated] The multiple observation devices 2 to which the allocation unit 11 allocates communication bandwidth in step S3 may be a portion of the multiple observation devices 2 present at one or multiple sites, or may be all of the multiple observation devices 2. For example, if the number of the multiple observation devices 2 is "N" (a natural number equal to or greater than 2) and the number of the multiple observation devices 2 to which the allocation unit 11 allocates communication bandwidth is "n" (a natural number equal to or greater than 2), the number n of the multiple observation devices 2 may be the same as the number N of the multiple observation devices 2 (n=N), or may be smaller than the number N of the multiple observation devices 2 (n<N).

[0078] Specifically, if the number of images displayed on the display device of terminal 3 is "k" (a natural number greater than or equal to 2), in step S3, the allocation unit 11 may allocate a communication bandwidth to each of the numerous observation devices 2 present at one or more sites (n = N), or may allocate a communication bandwidth to each of the multiple observation devices 2 corresponding to the multiple images displayed on the display device of terminal 3, while not allocating a communication bandwidth to the remaining observation devices 2 corresponding to images not displayed on the display device of terminal 3 (n = k, n < N).

[0079] In a case where the importance is not used, the allocation unit 11 may select, from the large number of observation devices 2, observation devices 2 (k observation devices 2) corresponding to the video displayed on the display device of the terminal 3, for example, based on a predetermined priority order for the observation devices 2. Furthermore, in a case where the importance is not used, the allocation unit 11 may select, from the large number of observation devices 2, observation devices 2 (k observation devices 2) corresponding to the video displayed on the display device of the terminal 3, for example, based on an input by a user to an input unit of the terminal 3. In a case where the importance is used, the allocation unit 11 may select, from the large number of observation devices 2, observation devices 2 (k observation devices 2) corresponding to the video displayed on the display device of the terminal 3, for example, based on the importance received by the importance receiving unit 20 in step S2.

[0080] In the display example of terminal 3 depicted in the upper right of FIG. 4 , the number of multiple videos displayed on the display device of terminal 3 is seven (k=7), while in the display example of terminal 3 depicted in the lower right of FIG. 4 , the number of multiple videos displayed on the display device of terminal 3 is six (k=6). In the display example of terminal 3 depicted in the upper right of FIG. 4 , the allocation unit 11 sets the communication bandwidth of the observation device 2 capturing the video A1, which has the highest importance, to "10 Mbps," and sets the communication bandwidth of videos A2, A3, A4, A5, B1, and B2 other than video A1 to "1 Mbps." In the display example of terminal 3 depicted in the lower right of FIG. 4 , the allocation unit 11 sets the communication bandwidth of the observation device 2 capturing the video E1, which has the highest importance, to "10 Mbps," and sets the communication bandwidth of videos E2, E3, E4, B2, and B4 other than video E1 to "1 Mbps."

[0081] As shown in FIG. 4 , the allocating unit 11 may increase the allocation of the communication bandwidth of video from an observation device 2 (nearby observation device 2) located near an observation device 2 capturing video with the highest communication bandwidth. Specifically, the nearby observation device 2 may be an observation device 2 capturing video with the highest communication bandwidth (video A1 or video E1) whose distance from the observation device 2 is equal to or less than a predetermined threshold. The allocating unit 11 may increase the allocation of video captured by the near observation device 2 by widening the bandwidth of the communication bandwidth of the video captured by the near observation device 2 (e.g., video B1 or video C1) by a predetermined value compared to before the change. The allocating unit 11 may increase the allocation of video captured by the near observation device 2 by increasing the proportion of the bandwidth of the communication bandwidth of the video captured by the near observation device 2 to an upper limit value by a predetermined value compared to before the change. The upper limit value is the available bandwidth of the network (e.g., 16 Mbps) or the amount of data that the terminal 3 can process (10 Mbps). Note that the upper limit value is not limited to 16 Mbps. The upper limit may be, for example, 50 Mbps or less, 40 Mbps or less, 30 Mbps or less, 20 Mbps or less, 15 Mbps or less, 10 Mbps or less, 5 Mbps or less, 3 Mbps or less, 2 Mbps or less, 1.5 Mbps or less, or 1 Mbps or less.

[0082] There are no particular restrictions on the method by which the allocation unit 11 allocates the communication bands, as long as the total bandwidth of the communication bands allocated to the multiple observation devices 2 falls within the range of the network available band or the range of the amount of data that the terminal 3 can process.

[0083] Furthermore, as described above, the allocation unit 11 may allocate a communication band to all of the multiple observation devices 2, or may allocate a communication band to some of the multiple observation devices 2. In this case, the allocation unit 11 may not allocate a communication band to observation devices 2 with low importance among the multiple observation devices 2, thereby widening the bandwidth of the communication band that can be allocated to observation devices 2 with high importance.

[0084] Next, the instruction unit 21 of the video aggregation computer 1 instructs the observation devices 2 corresponding to each video to set the image quality of the video based on the communication bandwidth allocated by the allocation unit 11 (step S4). Specifically, the instruction unit 21 may instruct the observation devices 2 capturing each video to increase or decrease the image quality of the video (image quality instruction) depending on the size of the communication bandwidth. Image quality may include resolution and / or frame rate, and the instruction unit 21 may instruct the observation devices 2 to increase or decrease the resolution and / or frame rate of the video as the image quality instruction. Specific aspects of image quality, such as the number of resolution levels and the number of frame rate levels, are not particularly important.

[0085] The observation device 2 sets the image quality of the video based on the image quality instruction acquired from the computer 1. The image quality instruction may include image quality instruction information for instructing the image quality. In this case, the observation device 2 may set the image quality based on the image quality instruction information. The image quality instruction information may include at least one of information instructing the image resolution and information instructing the image frame rate. The image quality instruction may also include communication band information regarding the bandwidth of the communication band. In this case, a map such as a relational equation or table expressing the relationship between the bandwidth of the communication band and the image quality may be set in advance in the observation device 2, and the observation device 2 may set the image quality based on the map and the communication band information. The map may express a relationship such that the larger the communication band bandwidth, the higher the image quality of the video.

[0086] Next, the receiving unit 22 of the video aggregation computer 1 receives, from the plurality of observation devices 2, video of the image quality set by the plurality of observation devices 2 via the communication bands allocated to the plurality of observation devices 2 (step S5). Specifically, the receiving unit 22 receives, from the plurality of observation devices 2, video of the image quality set by the observation devices 2 via the communication bands allocated to each of the plurality of observation devices 2, from the observation devices 2 on the video channel 4 of the video aggregation computer 1. The receiving unit 22 may also acquire signals generated by the observation devices 2 for setting low values ​​of brightness, contrast, etc. The signals generated by the observation devices 2 will be described later. Furthermore, the receiving unit 22 may store the acquired video in the video storage unit 30 based on the importance level received in step S2. Specifically, for example, if the importance level of the video acquired in step S5 is equal to or greater than a predetermined threshold, the receiving unit 22 may store the video in the video storage unit 30.

[0087] In this embodiment, storing video in the video storage unit 30 may mean storing the video in a standby state in a computer 1, such as a server, in which the video can be instantly displayed on the terminal 3 as needed. The standby state is a state in which video is transmitted from the observation device 2 to the computer 1, the computer 1 is not transmitting the video to the terminal 3, and the computer 1 can quickly access the video. The transmitter 23 can instantly display the video stored in the video storage unit 30 (video in the standby state) on the terminal 3 as needed.

[0088] Furthermore, the receiving unit 22 may determine whether there is a lot of movement in the video by recognizing the acquired video (for example, by performing image recognition). Specifically, when the receiving unit 22 determines that the movement in the video satisfies a predetermined condition, the receiving unit 22 may increase the accepted importance (for example, increase the importance to a predetermined threshold or higher) and store the acquired video in the video storage unit 30. The predetermined condition may be, for example, that the moving speed of an object in the video is greater than a predetermined threshold, that the acceleration of an object in the video is greater than a predetermined threshold, or that the moving distance of an object in the video is longer than a predetermined threshold.

[0089] In the example shown in FIG. 4 , the video aggregation computer 1 is provided with a video channel 4, which is composed of a plurality of ports 5 (e.g., 11 ports 5), but the number of ports 5 is not limited to the specific example shown in FIG. 4 . The plurality of ports 5 may be set in advance according to the communication band. That is, a plurality of communication bands assigned to a plurality of observation devices 2 may be associated with a plurality of ports 5. The receiver 22 acquires video at ports 5 corresponding to the plurality of communication bands. For example, the receiver 22 may acquire video with a high communication band at a port 5 associated with that communication band, and acquire video with a low communication band at a port 5 associated with that communication band.

[0090] Next, the transmitter 23 of the video aggregation computer 1 transmits the videos from the multiple observation devices 2 received via the multiple communication bands allocated to the multiple observation devices 2 to the terminal 3 (step S6). Specifically, by the transmitter 23 opening each port 5 of the video channel 4 to the terminal 3, the terminal 3 can acquire video corresponding to the communication band allocated by the allocation unit 11 with image quality corresponding to the communication band, and can display the acquired video on the screen of the terminal 3. As a result, the terminal 3 displays the video captured by each observation device 2 with the amount of data received by the computer 1 from that observation device 2 via the communication band allocated to each observation device 2. The transmitter 23 may display multiple videos from the multiple observation devices 2 on a single screen on one display device of the terminal 3. The transmitter 23 may also display multiple videos from the multiple observation devices 2 on multiple display devices of the terminal 3. That is, the transmitter 23 may display multiple videos from the multiple observation devices 2 on multiple screens. Furthermore, the transmitter 23 may enlarge and display the video (A1 or E1) with the highest communication bandwidth on the terminal 3, as in the display example of the terminal 3 in Fig. 4. Specifically, when the computer 1 causes the terminal 3 to display video of the site X, the computer 1 may output the video A1 with the highest display order to the terminal 3 in a larger size than the other videos other than the video A1, as shown in Fig. 4. The transmitter 23 may retrieve the video from the video storage unit 30 according to the communication bandwidth and display it on the terminal 3.

[0091] FIG. 4 shows an example of video switching, where the display example of terminal 3 depicted in the upper right of FIG. 4 is switched to the display example of terminal 3 depicted in the lower right of FIG. 4. For example, as shown in the display example in the upper right of FIG. 4, on the display device of terminal 3 displaying multiple videos of site X, video A1 is a video captured by observation device 2 assigned the widest bandwidth (highest communication bandwidth) of 10 Mbps. This video A1 (main video A1) is displayed larger than the other videos (enlarged display) at high resolution and / or a high frame rate. Videos A2-A5 (sub-videos A2-A5) related to this main video A1 are displayed on terminal 3 at low resolution and / or a low frame rate near main video A1 as videos in the same group (videos from the same cassette) as main video A1. In addition, videos B1-B2 are displayed depending on the communication bandwidth.

[0092] In the example of video switching shown in FIG. 4 , the display example in the upper right corner of FIG. 4 is switched to the display example in the lower right corner of FIG. 4 . In this display example in the lower right corner, video E1 is video captured by the observation device 2 to which the widest bandwidth (highest communication bandwidth) of 10 Mbps is assigned. This video E1 (main video E1) is displayed larger than the other videos (enlarged display) at high resolution and / or a high frame rate. Videos E2-E4 (sub-videos E2-E4) related to this main video E1 are displayed on the terminal 3 at low resolution and / or a low frame rate near the main video E1 as videos in the same group as the main video E1 (videos from the same cassette). In addition, videos B2 and B4 are displayed depending on the communication bandwidth.

[0093] The secondary images (A2-A5 or E2-E4) may be, for example, images of the same location as the primary image (A1 or E1), or may be images of the same machine as the primary image. Furthermore, the observation devices (five observation devices) that capture images A1-A5 included in the same group may be attached to a work machine. Furthermore, the observation devices (four observation devices) that capture images E1-E4 included in the same group may be attached to an air vehicle such as a drone.

[0094] The vicinity of the primary video (A1 or E1) may be, for example, a peripheral position of the primary video, such as a position next to the primary video on the screen of terminal 3. Specifically, the vicinity of the primary video may be, for example, a position next to and to the side (right or left) of the primary video on the screen of terminal 3, a position next to and above the primary video, or a position next to and below the primary video. Also, as described above, the number of resolution levels is not limited to two, high resolution and low resolution. For example, the number of resolution levels including high resolution and low resolution may be three or more. Similarly, the number of frame rate levels is not limited to two, high frame rate and low frame rate. For example, the number of frame rate levels including high frame rate and low frame rate may be three or more.

[0095] In the display example shown in the upper right of Fig. 4, images A1-A5 captured by the observation device 2 attached to the work machine are primarily displayed on a single screen on the display device of terminal 3. In the display example shown in the lower right of Fig. 4, images E1-E4 (bird's-eye view images) captured by the observation device 2 attached to the aircraft are primarily displayed on a single screen on the display device of terminal 3. In the switching example shown in Fig. 4, it is possible to switch from images A1-A5 captured by the observation device 2 attached to the work machine to the bird's-eye view images (images E1-E4).

[0096] Furthermore, the display example shown in Fig. 4 may be switched from the upper right display example to the display example shown in Fig. 5. In the display example shown in Fig. 5, images E1-E2 are displayed in the area where images B1-B2 were displayed in the upper right display example in Fig. 4. This allows the user of terminal 3 to simultaneously view images A1-A5 captured by observation device 2 attached to the work machine and images E1-E2 captured by observation device 2 attached to the aircraft.

[0097] The video switching shown in the specific example above may be achieved by the computer 1 changing the allocation of communication bandwidth. For example, the computer 1 allocates communication bandwidth to each of the multiple observation devices 2 corresponding to the multiple videos A1-A5, B1-B2 displayed on the terminal 3 shown in the upper right of FIG. 4 . The computer 1 receives the multiple videos A1-A5, B1-B2 from the multiple observation devices 2 using the communication bandwidth allocated to the multiple observation devices 2, and transmits the received videos A1-A5, B1-B2 to the terminal 3. This allows the terminal 3 to display the multiple videos A1-A5, B1-B2, as shown in the display example of the terminal 3 shown in the upper right of FIG. 4 . In this case, the importance receiving unit 20 of the computer 1 may receive the importance of the multiple videos A1-A5, B1-B2. Specifically, if the importance of video A1 is the highest among the multiple videos A1-A5, B1-B2, the computer 1 displays video A1 as the main video, larger than the other videos, as shown in the upper right of FIG. 4 .

[0098] Furthermore, when switching the image from the display example in the upper right of FIG. 4 to the display example in the lower right of FIG. 4, the computer 1 may perform the following process. That is, the computer 1 allocates a communication bandwidth to each of the multiple observation devices 2 corresponding to the multiple images E1-E4, B2, and B4 displayed on the terminal 3 shown in the lower right of FIG. 4, receives the multiple images E1-E4, B2, and B4 from the multiple observation devices 2 using the communication bandwidth allocated to the multiple observation devices 2, and transmits the received images E1-E4, B2, and B4 to the terminal 3. This allows the terminal 3 to display the multiple images E1-E4, B2, and B4 as shown in the display example of the terminal 3 shown in the lower right of FIG. 4. In this case, the importance receiving unit 20 of the computer 1 may receive the importance of the multiple images E1-E4, B2, and B4. Specifically, if the importance of image E1 is the highest among the multiple images E1-E4, B2, and B4, the computer 1 displays image E1 as the main image and larger than the other images, as shown in the lower right of FIG. 4.

[0099] This completes the video switching process.

[0100] 6 is a diagram for explaining the video transmission process executed by the video integration computer 1 of this embodiment. This process is a detailed description of the process performed in step S6 of the video switching process described above.

[0101] The determination unit 12 of the video aggregation computer 1 determines whether the video received in step S5 is an associated video (step S61). Specifically, the determination unit 12 determines whether the video received in step S5 is included in a group (cassette) as a sub-image.

[0102] If it is determined in step S61 above that the received video is associated video (step S61: YES), the determination unit 12 determines whether the received video is associated with video from the observation device 2 to which the widest bandwidth communication band (highest communication band) is assigned (step S62). Specifically, the determination unit 12 determines whether the video received in step S5 is included in a group (cassette) as a sub-image associated with the main image from the observation device 2 to which the widest bandwidth communication band (highest communication band) is assigned.

[0103] On the other hand, if it is determined in step S61 that the video received in step S5 is not an associated video (step S61: NO), the determination unit 12 determines whether the communication bandwidth allocated to the observation device 2 capturing the received video is the highest (step S63). Specifically, the determination unit 12 determines whether the video received in step S5 is a main image from the observation device 2 to which the widest communication bandwidth (highest communication bandwidth) is allocated.

[0104] In the above-mentioned step S62, if it is determined that the video received in step S5 is video associated with video from the observation device 2 to which the widest bandwidth communication band (highest communication band) is assigned (step S62: YES), the transmitter 23 transmits the image to the terminal 3 (step S64) and terminates this process.

[0105] In step S62, if it is determined that the video received in step S5 is not video associated with video from the observation device 2 to which the widest bandwidth (highest bandwidth) is assigned (step S62: NO), the computer 1 ends this process. In this case, the image is not transmitted to the terminal 3, but may be stored in the video storage unit 30 based on the importance level received in step S2.

[0106] If it is determined in step S63 above that the communication bandwidth allocated to the observation device 2 capturing the video received in step S5 is the highest (step S63: YES), the transmitter 23 transmits the video to the terminal 3 (step S64), and the process ends. At this time, the computer 1 may retrieve the video from the video storage unit 30 according to the communication bandwidth, transmit it to the terminal 3, and then end the process.

[0107] On the other hand, if it is determined in step S63 that the communication bandwidth assigned to the observation device 2 capturing the video received in step S5 is not the highest (step S63: NO), the computer 1 ends this process. In this case, the image is not transmitted to the terminal 3, but may be stored in the video storage unit 30 based on the importance level received in step S2.

[0108] This completes the video transmission process.

[0109] The image aggregation computer 1 of this embodiment, when displaying images from multiple observation devices 2 on a remote user's terminal 3, can cause the terminal 3 to display images according to the environment, even in an environment where there is a limit to the amount of data transmission that can be used for the images.

[0110] Specifically, the video aggregation computer 1 can control the amount of data transmitted from a large number of observation devices scattered around the site by receiving video in the communication bandwidth allocated to each of the multiple observation devices 2 and transmitting the received video to the terminal 3. Therefore, in an environment where the amount of data transmission available for video is limited, even with the minimum necessary infrastructure, it is possible to suppress problems such as video delays and degradation of video quality, and for the video aggregation computer 1 to automatically and smoothly switch between videos according to the available network bandwidth or the performance of resources that can process the received data.

[0111] According to the video aggregation computer 1 of this embodiment, video is received in a communication bandwidth allocated to each of the plurality of observation devices 2 in accordance with the available network bandwidth and the importance of the video, or video is received in a communication bandwidth allocated to the plurality of observation devices 2 in accordance with the resources of the terminal 3 and the importance, and the received video is transmitted to the terminal 3. Therefore, in an environment where there is a limit to the amount of data transmission available for video, even with the minimum necessary infrastructure, it is possible to suppress the occurrence of problems such as video delays and degradation of video quality, and to smoothly switch between videos automatically by the video aggregation computer 1 in accordance with the importance of the video, or to smoothly switch between videos in accordance with the importance manually input by the user.

[0112] According to the video aggregation computer 1 of this embodiment, the video from each observation device 2 is displayed on a single screen, so that even in an environment where there is a limit to the amount of data transmission that can be used for video, and with only the minimum necessary infrastructure, the user can closely monitor the locations and / or machines captured by multiple videos.

[0113] Furthermore, according to the video aggregation computer 1 of this embodiment, the video with the highest communication bandwidth is displayed large, so that even in an environment where there is a limit to the amount of data transmission that can be used for video, even with the minimum necessary infrastructure, it is possible to closely monitor the locations and / or machines that show the video that the user is most interested in.

[0114] Furthermore, the image aggregation computer 1 according to this embodiment can display images of machines, places, etc. that are attracting a lot of attention, or images of machines, places, etc. that are close to the images, allowing the user to simultaneously monitor images of machines, places, etc. that are attracting attention and the surrounding environment.

[0115] Furthermore, according to the video consolidation computer 1 of this embodiment, the user of the video consolidation computer 1 or the user of the terminal 3 can input the importance of the video into the input unit, which allows the computer 1 to switch the video displayed on the terminal 3 to another video based on the importance. In other words, the user can increase or decrease the importance of the video showing the monitored object, such as a machine being operated or a monitored site, which allows the computer 1 to switch the video displayed on the terminal 3 based on the importance.

[0116] According to the video aggregation computer 1 of this embodiment, by increasing the resolution of videos of high importance, videos requiring detailed information can be delivered clearly, and by decreasing the resolution of videos of low importance, these videos can be delivered smoothly even in a slow network environment or on a device with limited processing power, allowing users to view these videos. Also, by increasing the frame rate of videos of high importance, videos with a lot of fast movement can be delivered, and by decreasing the frame rate of videos of low importance, the data size of the videos can be reduced, allowing these videos to be delivered and users to view these videos even in a slow network environment or an unstable network environment due to mobility, etc.

[0117] According to the video aggregation computer 1 of this embodiment, for example, by storing video of high importance and keeping the output of that video in a standby state in the computer 1, it becomes possible to smoothly switch between videos without delay even in a site with a poor communication environment based on the importance that is input automatically by the computer or manually by the user. Also, by storing the video in the computer 1 (server), it becomes possible to search for the video of the subject being shot.

[0118] According to the video aggregation computer 1 of this embodiment, stored video, i.e., video that is on standby for output, can be instantly accessed and output, so that even in locations with poor communication environments, the video can be switched smoothly and without delay, either automatically by the computer or manually by the user.

[0119] An example of the flow of the process in which the video storage unit 30 stores video and the process in which the transmission unit 23 transmits the stored video to the terminal 3 will be briefly described below.

[0120] The video storage unit 30 may determine whether to store the video based on a predetermined storage determination condition, and store the video when the storage determination condition is satisfied. The transmitter 23 may determine whether to transmit the video stored in the video storage unit 30 to the terminal 3 based on a predetermined transmission determination condition, and transmit the video to the terminal 3 when the transmission determination condition is satisfied.

[0121] The storage determination condition may include, for example, a condition that the importance of the video is equal to or greater than a predetermined threshold. The transmission determination condition may include, for example, a condition that the communication bandwidth of the observation device 2 capturing the stored video is equal to or greater than a predetermined threshold. The transmission determination condition may include, for example, a condition regarding the presence or absence of a movable monitoring target, such as a machine or a person, in the video to be stored. Specifically, the transmission determination condition may include, for example, a condition that the presence or absence of a movable monitoring target in the video to be stored changes to a state in which the monitoring target is present. The transmission determination condition may also include, for example, a condition regarding the degree of movement of the movable monitoring target in the video to be stored. Specifically, the transmission determination condition may be, for example, a condition that the speed of movement of the movable monitoring target in the video to be stored is greater than a predetermined threshold, a condition that the acceleration of the movable monitoring target in the video to be stored is greater than a predetermined threshold, or a condition that the distance traveled by the movable monitoring target in the video to be stored is greater than a predetermined threshold. Each of the thresholds (predetermined thresholds) may be set, for example, by a remote user making an input to the input unit of the terminal 3 to specify the predetermined threshold.

[0122] [Video Distribution Processing] The video distribution processing executed by the observation device 2 of this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart of the video distribution processing executed by the observation device 2.

[0123] The information detection unit 40 of the observation device 2 detects information related to the video captured by the observation device 2 (video-related information) (step S21).

[0124] The information detection unit 40 may, for example, acquire sensing results from a sensor and detect the video-related information based on the sensing results. The sensor may be a sensor included in the observation device 2, or may be a sensor installed at a location separate from the observation device 2. The sensor installed at a location separate from the observation device 2 may be a sensor installed at a work site, manufacturing site, or other site, or may be a sensor attached to an object to be monitored at the site.

[0125] The information detection unit 40 may detect, based on the sensing results of the sensors, information about some abnormality (abnormality information) at a site such as a work site or a manufacturing site as the video-related information.

[0126] Furthermore, the information detection unit 40 may detect information about the movement of a monitored object (object movement information) as the video-related information based on the sensing results of the sensors. Examples of monitored objects include work machines (such as shovels, cranes, and transport vehicles), people, structures, and obstacles present at a work site. Examples of monitored objects include work machines (such as hoists and transport vehicles), people, and structures present at a manufacturing site. The object movement information may be, for example, information about the speed of the monitored object, information about the acceleration of the monitored object, or information about the distance traveled by the monitored object.

[0127] Furthermore, the detection in step S21 may be detection based on a trained model for information detection using artificial intelligence (AI). That is, the information detection unit 40 may detect information related to the video captured by the observation device 2 (video-related information) using the trained model for information detection. Specifically, for example, the information detection unit 40 may input a determination target portion (e.g., an image included in the video) of the video captured by the observation device 2 to be determined into the trained model for information detection, obtain a determination result for an object included in the determination target portion from the trained model for information detection, and detect the video-related information based on the obtained determination result.

[0128] For example, the learning unit 44 of the observation device 2 may perform supervised learning on the feature quantities of information displayed in the video when the information is detected. This makes it possible to accumulate training data on workers, particularly skilled workers, and future use of this data may enable work optimization and hazard prediction in work practices. Such training data may be stored in the memory unit of the observation device 2 as additional training data for automatic detection. The learning unit 44 may also learn about the feature quantities using a deep learning technique, which automatically defines and learns using a multi-layered neural network. Thereafter, the learning unit 44 may create a trained model for information detection based on the learning results, and the memory unit of the observation device 2 may store the trained model.

[0129] In the above specific example, the learning unit 44 of the observation device 2 generates a trained model by performing supervised learning, but the generation of the trained model may be performed by a learning device other than the observation device 2. The observation device 2 may store the trained model generated by the learning device in a storage unit. This learning device includes a computer. The learning device acquires a training dataset including teacher data and correct labels, inputs the teacher data to a trained model, acquires processing results from the trained model, and trains the trained model based on the error between the correct labels and the processing results according to a learning algorithm.

[0130] For example, if the trained model for information detection has been trained based on the error between the judgment result of the trained model for a training image including a target object and a correct label indicating the type of object included in the training image, the information detection unit 40 can input the target portion to the trained model for information detection, obtain a judgment result from the trained model for information detection, and detect information about the target object as the video-related information based on the obtained judgment result. The information detection unit 40 may sequentially input a large number of time-series images included in a video captured by the observation device 2 to the trained model for information detection and obtain a judgment result about the object included in each image from the trained model for information detection, thereby detecting, for example, information about the movement of an object in the video (object movement information) as the video-related information. The object movement information may be, for example, information about the speed of the object in the video, information about the acceleration of the object in the video, or information about the distance traveled by the object in the video.

[0131] Next, the importance setting unit 41 of the observation device 2 sets the importance of the video based on the video-related information detected in step S21 (step S22). Specifically, the importance setting unit 41 may increase the importance of the video to be captured when, for example, the information detection unit 40 detects some kind of abnormality at the site based on the sensing results of a sensor installed at the site and / or detects that the acceleration of the movement of an object in the video is high, and may decrease the importance of the video to be captured when these conditions are resolved.

[0132] Examples of the sensor include a LiDAR (Light Detection and Ranging), an acoustic sensor, a voice sensor, an acceleration sensor, a gas sensor, a humidity sensor, a light sensor, a vibration sensor, a proximity sensor, a pressure sensor, a magnetic sensor, an infrared sensor, a metal detector, etc. However, the sensor is not limited to these specific examples.

[0133] If the sensor is a LiDAR, the information detection unit 40 may acquire sensing results from the LiDAR and detect the distance to a monitoring target, such as an object, as the video-related information based on the sensing results, and the importance setting unit 41 may set the importance of the video according to the detected distance. In this case, the importance setting unit 41 may set a higher importance level the higher the urgency corresponding to the distance (e.g., the closer the distance to the monitoring target). This allows the computer 1 to preferentially output video of an area with a high urgency to the terminal 3 based on the importance level provided by the observation device 2. The importance level may be set in advance into multiple levels (e.g., five levels) associated with the distance.

[0134] If the sensor is an acoustic sensor, the information detection unit 40 may acquire sensing results from the acoustic sensor and detect the magnitude of sound pressure as the video-related information based on the sensing results, and the importance setting unit 41 may set the importance of the video according to the magnitude of the detected sound pressure. In this case, the importance setting unit 41 may set a higher importance the higher the urgency correlated with the sound pressure (e.g., the higher the sound pressure). This allows the computer 1 to preferentially output videos of areas with high urgency to the terminal 3 based on the importance provided by the observation device 2. The importance rank may be set in advance into multiple levels (e.g., five levels) associated with the magnitude of the sound pressure.

[0135] If the sensor is an audio sensor, the information detection unit 40 may acquire sensing results from the audio sensor and detect audio characteristics, such as audio volume and audio frequency, as the video-related information based on the sensing results, and the importance setting unit 41 may set the importance of the video according to the detected audio characteristics. In this case, the importance setting unit 41 may set a higher importance level the higher the urgency correlated with the audio characteristics (e.g., the louder the audio pressure). This allows the computer 1 to preferentially output videos of areas with high urgency to the terminal 3 based on the importance level provided by the observation device 2. The importance level may be set in advance into multiple levels (e.g., five levels) associated with the audio characteristics.

[0136] If the sensor is an acceleration sensor, the information detection unit 40 may acquire sensing results from the acceleration sensor and detect the acceleration of an object as the video-related information based on the sensing results, and the importance setting unit 41 may set the importance of the video according to the detected acceleration of the object. In this case, the importance setting unit 41 may set a higher importance the higher the degree of abnormality correlated with the acceleration (e.g., the greater the acceleration). This allows the computer 1 to preferentially output videos of areas with a high degree of abnormality to the terminal 3 based on the importance provided by the observation device 2. The importance rank may be set in advance into multiple levels (e.g., five levels) associated with the magnitude of the acceleration. The abnormality correlated with acceleration may be, for example, an abnormality caused by vibration of an object or another abnormality.

[0137] If the sensor is a gas sensor, the information detection unit 40 may acquire sensing results from the gas sensor and detect a gas concentration as the video-related information based on the sensing results, and the importance setting unit 41 may set the importance of the video according to the detected gas concentration. In this case, the importance setting unit 41 may set a higher importance the greater the degree of deterioration of the site environment correlated with the gas concentration (e.g., the higher the gas concentration). This allows the computer 1 to preferentially output to the terminal 3 videos of areas where the site environment is highly deteriorated, based on the importance provided by the observation device 2. The importance rank may be set in advance into multiple levels (e.g., five levels) associated with the magnitude of the gas concentration.

[0138] If the sensor is a humidity sensor, the information detection unit 40 may acquire sensing results from the humidity sensor and detect humidity as the video-related information based on the sensing results, and the importance setting unit 41 may set the importance of the video according to the detected humidity. In this case, the importance setting unit 41 may set a higher importance of the video the greater the degree of decline in operational efficiency of the device correlated with the humidity (e.g., the higher the humidity). This allows the computer 1 to preferentially output to the terminal 3 videos of areas where the degree of decline in operational efficiency of the device is high, based on the importance provided by the observation device 2. The importance rank may be set in advance into multiple levels (e.g., five levels) associated with the level of humidity.

[0139] If the sensor is an optical sensor, the information detection unit 40 may acquire sensing results from the optical sensor and detect the amount of light based on the sensing results as the video-related information, and the importance setting unit 41 may set the importance of the video according to the detected amount of light. In this case, the importance setting unit 41 may set a higher importance of the video as the degree of reduction in visibility correlated with the amount of light increases (e.g., the lower the amount of light). This allows the computer 1 to preferentially output to the terminal 3 an image of an area where the degree of reduction in visibility is high and work efficiency is likely to decrease, based on the importance provided by the observation device 2. The importance rank may be set in advance into multiple levels (e.g., five levels) associated with the magnitude of the amount of light.

[0140] If the sensor is a vibration sensor, the information detection unit 40 may acquire sensing results from the vibration sensor and detect vibrations as the video-related information based on the sensing results, and the importance setting unit 41 may set the importance of the video according to the detected vibration. In this case, the importance setting unit 41 may set a higher importance for the video, the higher the degree of abnormality correlated with the vibration characteristics and in an object such as a structure or equipment at the site. This allows the computer 1 to preferentially output to the terminal 3 videos of areas with a high degree of abnormality in an object such as a structure or equipment based on the importance provided by the observation device 2, allowing the user to discover abnormalities early. The importance ranking may be set in advance into multiple levels (e.g., five levels) associated with the vibration characteristics.

[0141] If the sensor is a proximity sensor, the information detection unit 40 may acquire sensing results from the proximity sensor and detect the distance to a monitoring target, such as an object, as the video-related information based on the sensing results, and the importance setting unit 41 may set the importance of the video according to the detected distance. In this case, the importance setting unit 41 may set a higher importance level the higher the urgency corresponding to the distance (e.g., the closer the distance to the monitoring target). This allows the computer 1 to preferentially output video of an area with a high urgency to the terminal 3 based on the importance level provided by the observation device 2. The importance level may be set in advance into multiple levels (e.g., five levels) associated with the distance.

[0142] If the sensor is a pressure sensor, the information detection unit 40 may acquire sensing results from the pressure sensor and detect pressure as the video-related information based on the sensing results, and the importance setting unit 41 may set the importance of the video according to the detected pressure. In this case, the importance setting unit 41 may set a higher importance the higher the degree of abnormality in the load correlated with the magnitude of the pressure (e.g., the greater the pressure). This allows the computer 1 to preferentially output to the terminal 3 videos of areas where a high load is placed on a machine such as a work machine, based on the importance provided by the observation device 2. The importance rank may be set in advance into multiple levels (e.g., five levels) associated with the magnitude of the pressure.

[0143] If the sensor is a magnetic sensor, the information detection unit 40 may acquire sensing results from the magnetic sensor and detect magnetic field changes as the video-related information based on the sensing results, and the importance setting unit 41 may set the importance of the video according to the detected magnetic field changes. In this case, the importance setting unit 41 may set a higher importance the higher the degree of abnormality of the metal object correlated with the magnetic field changes. This allows the computer 1 to preferentially output to the terminal 3 videos of areas where abnormal metal objects are present at the site based on the importance provided by the observation device 2. The importance rank may be set in advance into multiple levels (e.g., five levels) associated with the magnetic field changes.

[0144] If the sensor is an infrared sensor, the information detection unit 40 may acquire sensing results from the infrared sensor and detect the amount of heat radiation based on the sensing results as the image-related information, and the importance setting unit 41 may set the importance of the image according to the detected amount of heat radiation. In this case, the importance setting unit 41 may set a higher importance level the higher the degree of risk of an overheated area correlated with the amount of heat radiation. This allows the computer 1 to preferentially output images of areas where overheated areas exist at the site to the terminal 3 based on the importance level provided by the observation device 2. The importance level may be set in advance into multiple levels (e.g., five levels) associated with the amount of heat radiation.

[0145] If the sensor is a metal detector, the information detection unit 40 may acquire sensing results from the metal detector and detect metal characteristics, such as the type and size of the metal object, as the video-related information based on the sensing results, and the importance setting unit 41 may set the importance of the video according to the detected metal characteristics. In this case, the importance setting unit 41 may set a higher importance as the degree of danger correlated with the characteristics of the metal object increases. This allows the computer 1 to preferentially output videos of areas where dangerous objects are present at the site to the terminal 3 based on the importance provided by the observation device 2. The importance ranks may be set in advance into multiple levels (e.g., five levels) associated with the characteristics of the metal object.

[0146] Note that, when the detection in step S21 is based on a trained model for information detection, the information detection unit 40 is configured to input the video captured by the observation device 2 (e.g., an image included in the video captured by the observation device 2) into the trained model for information detection and acquire the determination result (e.g., the type of object in the video) from the trained model for information detection, but is not limited to this specific example. For example, the information detection unit 40 may be configured to input the sensing result of the sensor into the trained model for information detection and acquire the determination result from the trained model for information detection. In this case, the determination result may be a determination result regarding the presence or absence of an abnormality (the degree of abnormality) that is correlated with the sensing result of the sensor. Specific examples of the sensor include those described below. The sensing result of a sound-related sensor, such as an acoustic sensor or a voice sensor, is correlated with the presence or absence of an abnormality regarding sound. The sensing result of an acceleration sensor is correlated with the presence or absence of an abnormality regarding acceleration or speed. The sensing result of a gas sensor is correlated with the presence or absence of an abnormality regarding gas concentration. The sensing result of a humidity sensor is correlated with the presence or absence of an abnormality regarding humidity. The sensing results of an optical sensor correlate with the presence or absence of abnormalities in the amount of light. The sensing results of a vibration sensor correlate with the presence or absence of abnormalities in vibration. The sensing results of a proximity sensor correlate with the presence or absence of abnormalities in the distance to the monitored object. The sensing results of a pressure sensor correlate with the presence or absence of abnormalities in pressure. The sensing results of a magnetic sensor correlate with the presence or absence of abnormalities in magnetic field changes. The sensing results of an infrared sensor correlate with the presence or absence of abnormalities in the distance to the monitored object. The sensing results of a metal detector correlate with the presence or absence of abnormalities in the characteristics of metal (degree of danger).

[0147] The information detection unit 40 may also be configured to input the video captured by the observation device 2 and / or the sensing results of the sensor into a trained model for information detection and acquire the judgment results (e.g., a judgment result of the type of object in the video, a judgment result of the presence or absence of an abnormality) from the trained model for information detection. The information detection unit 40 may also be configured to input the video captured by the observation device 2 and / or the sensing results of the sensor into a trained model for information detection and acquire an importance level as a judgment result from the trained model for information detection. This importance level may be correlated with the presence or absence of an abnormality, for example, and the presence of such an abnormality may indicate a high importance level. In this case, the learning device may acquire a training dataset including teacher data (e.g., sensing results) and a correct answer label (e.g., the presence or absence of an abnormality), input the teacher data into a trained model, acquire a processing result from the trained model, and train the trained model based on the error between the correct answer label and the processing result according to a learning algorithm.

[0148] Next, the importance providing unit 50 of the observation device 2 provides the set importance to the video aggregating computer 1 (step S23). Specifically, the importance providing unit 50 provides the set importance to the video aggregating computer 1 by transmitting the set importance to the computer 1 via the network so that the video aggregating computer 1 can allocate a communication bandwidth for the observation device 2 that captures the video to be transmitted to the terminal 3.

[0149] Next, the instruction receiving unit 51 of the observation device 2 receives an instruction regarding the image quality of the video (the image quality instruction) from the video aggregation computer 1 (step S24). Specifically, the instruction receiving unit 51 may receive an instruction from the video aggregation computer 1 to increase the image quality of the video from the observation device 2 to which a wide bandwidth communication band is assigned (video for which the communication band is set high) and to decrease the image quality of the video from the observation device 2 to which a narrow bandwidth communication band is assigned (video for which the communication band is set low). The instruction receiving unit 51 may receive an instruction from the computer 1 to set the image quality of the video from the observation device 2 to which a wide bandwidth communication band is assigned to high image quality and set the image quality of the video from the observation device 2 to which a narrow bandwidth communication band is assigned to low image quality. The instruction receiving unit 51 may receive information regarding the allocation of the communication band from the computer 1 as the image quality instruction.

[0150] Next, the image quality setting unit 42 of the observation device 2 sets the image quality of the captured image based on the received instruction (step S25). Specifically, the image quality setting unit 42 increases or decreases the image quality of the captured image based on the received instruction. Image quality includes resolution and / or frame rate, and the instruction unit 21 of the computer 1 may issue an instruction to the observation device 2 to increase or decrease the image resolution and / or frame rate. Specific aspects of image quality, such as the number of resolution levels and the number of frame rate levels, are not particularly important.

[0151] The acceleration detection unit 43 of the observation device 2 detects the acceleration of the observation device 2. Examples of the acceleration detection unit 43 include a gyro sensor and an acceleration sensor. In step S25, if the acceleration of the observation device 2 itself detected by the acceleration detection unit 43 of the observation device 2 is large, for example, the image quality setting unit 42 may set the image quality of the captured image to a low level without following instructions from the image aggregation computer 1. Specifically, if the acceleration of the observation device 2 is equal to or greater than a predetermined threshold, the image quality setting unit 42 may lower the image quality to a predetermined value, or may lower the image quality by a predetermined degree. Furthermore, the image quality setting unit 42 may set the image quality to a lower level as the acceleration of the observation device 2 increases. The image quality setting unit 42 may control the image quality based on changes in parameters such as image brightness and contrast.

[0152] Next, the video providing unit 52 of the observation device 2 provides the video of the set image quality to the video aggregation computer 1 (step S26). Specifically, the video providing unit 52 provides the video of the image quality set in the above-mentioned step S25 to the video channel 4 of the video aggregation computer 1. At this time, the video providing unit 52 provides the video of the set image quality to the port 5 set according to the communication bandwidth. If the observation device 2 generated a signal to set parameters such as brightness and contrast low in the above-mentioned step S25, the video providing unit 52 also provides this signal to the video aggregation computer 1.

[0153] This completes the video distribution process.

[0154] According to the observation device 2 of this embodiment, for example, if a sensor or the like installed on-site detects an abnormality or a high acceleration of movement in the video, the importance of video of the monitored location, facility, or other object related to the abnormality and / or fast-moving video can be increased. As a result, the observation device 2 is instructed by the computer 1 to set high-quality video for high-importance video and low-quality video for low-importance video, allowing the computer 1 to control the amount of data transmitted from multiple observation devices 2 scattered around the site. Therefore, even in an environment with limited data transmission capacity for video, even with minimal infrastructure, problems such as video delays and video quality degradation can be suppressed, and the video can be smoothly switched according to the importance of the video, either automatically by the video aggregation computer 1 or based on the importance manually input by the user.

[0155] If the observation device 2 is an unmanned aerial vehicle such as a drone, the movement of the unmanned aerial vehicle (such as uniform motion) is remotely controlled, and the subject of the unmanned aerial vehicle is, for example, a work machine and its surroundings at a work site, the unmanned aerial vehicle can capture bird's-eye view footage of the work machine and its surroundings. That is, the unmanned aerial vehicle can capture footage of the work machine itself, the area in front of the work machine, the area behind the work machine, the area to the right of the work machine, and the area to the left of the work machine. In this case, the image quality setting unit 42 may detect the movement of the work machine included in the bird's-eye view footage and set the image quality based on the movement of the work machine (e.g., based on the speed and / or acceleration of the work machine). The image quality setting unit 42 may also detect the movement of a moving object included in the bird's-eye view footage, specifically, a moving object present in front of, behind, to the right, or to the left of the work machine, and set the image quality based on the movement of the moving object (e.g., based on the speed and / or acceleration of the moving object). The image quality setting unit 42 may, for example, detect the movement of at least one of the work machine and the mobile object from the overhead-view video, and control the image quality based on optical flow difference information based on the acceleration and / or speed of that movement. The image quality setting unit 42 may, for example, set a higher importance level the greater the speed and / or acceleration of the work machine. Similarly, the image quality setting unit 42 may, for example, set a higher importance level the greater the speed and / or acceleration of the mobile object. This improves safety during remote operation. The image quality setting unit 42 may also control the image quality based on changes in parameters such as brightness and contrast in the overhead-view video.

[0156] If the observation device 2 is, for example, an imaging device such as a camera fixedly installed at a predetermined location on the work site, and the subject of the camera is, for example, a work machine and its surroundings at the work site, the imaging device can capture an area appropriate to the installation location. For example, it is preferable for the imaging device to capture video of at least one of the areas behind the work machine, the area to the right of the work machine, and the area to the left of the work machine. This is because users typically tend to focus on the area in front of the work machine, while they tend not to focus on the areas behind, to the right, and to the left of the work machine. The image quality setting unit 42 may detect the movement of the work machine included in the video and set the image quality based on the movement of the work machine (e.g., based on the speed and / or acceleration of the work machine). Furthermore, the image quality setting unit 42 may detect the movement of a moving object included in the video, specifically, a moving object present, for example, in the area behind, to the right, or to the left of the work machine, and set the image quality based on the movement of the moving object (e.g., based on the speed and / or acceleration of the moving object). The image quality setting unit 42 may, for example, detect the movement of at least one of the work machine and the mobile object from the overhead-view video, and control the image quality based on optical flow difference information based on the acceleration and / or speed of that movement. The image quality setting unit 42 may, for example, set a higher importance level the greater the speed and / or acceleration of the work machine. Similarly, the image quality setting unit 42 may, for example, set a higher importance level the greater the speed and / or acceleration of the mobile object. This improves safety during remote operation. The image quality setting unit 42 may also control the image quality based on changes in parameters such as brightness and contrast in the overhead-view video.

[0157] Furthermore, with the observation device 2 of this embodiment, the amount of data transmission can be controlled by lowering the image quality of video captured when the observation device 2 itself is moving unstably. Therefore, in an environment where the amount of data transmission available for video is limited, even with the minimum necessary infrastructure, it is possible to prevent problems such as video delays and degradation of video quality, and to smoothly switch between videos according to their importance, either automatically by the video aggregation computer 1 or based on the importance manually input by the user.

[0158] Furthermore, according to the observation device 2 of this embodiment, artificial intelligence (AI) learns from past images about information that influences the setting of importance, allowing the observation device to set the importance of the images it captures, so that the importance can be automatically increased or decreased depending on the content of the captured image, and the images to be displayed can be switched as appropriate.

[0159] The above-described means and functions are realized by a computer 1 including a CPU, memory, various terminals, etc., reading and executing a predetermined program. The program may be provided, for example, in the form of a cloud service provided from one or more terminals via a network, specifically, for example, in the form of SaaS (Software as a Service). The program may also be provided, for example, in the form of a computer-readable recording medium. In this case, the computer reads the program from the recording medium, transfers it to an internal or external recording device, records it, and executes it. The program may also be pre-recorded on a non-transitory recording device (non-transitory recording medium), such as a magnetic disk, optical disk, or magneto-optical disk, and may be configured to be provided to the terminal from the recording device via a communication line.

[0160] Specifically, the program of this embodiment is a program readable by a computer 1, and causes the computer 1 to execute the following steps: estimating a network available bandwidth for a remote user's terminal 3; allocating a communication bandwidth to each of multiple observation devices 2 based on the estimated network available bandwidth or the resources of the terminal 3; and receiving images from multiple observation devices 2 using the communication bandwidth allocated to the multiple observation devices 2, and transmitting the received images to the terminal 3.

[0161] Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments. Furthermore, the effects described in the embodiments of the present invention are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments of the present invention.

[0162] When a remotely controlled object, such as a construction machine, is remotely operated, users involved in the remote operation, such as remote operators and supervisors, need to prepare a network communication environment to ensure smooth remote operation of the remotely controlled object. However, when a mobile object, such as a vehicle, communicates with other devices, for example, via a mobile phone network, the available network bandwidth is prone to fluctuations, which can easily cause problems such as packet loss and video delays due to insufficient communication bandwidth. Specifically, when an observation device is mounted on a mobile object or when a user's terminal is mounted on a mobile object, the communication environment is prone to fluctuations. Furthermore, in areas such as mountainous regions and the sea, the communication environment, such as network communication speed and communication capacity, may not always be satisfactory. Furthermore, when multiple observation devices, such as multiple cameras, are deployed on-site and images from the multiple observation devices are transmitted to user terminals via a network, problems such as video delays and reduced video quality are likely to occur.

[0163] The video aggregation computer 1, observation device 2, video aggregation method, and program of this embodiment enable a remote user's terminal to display video according to the environment when video from multiple observation devices is displayed, even in an environment where there is a limit to the amount of data transmission that can be used for video.

[0164] Summary of the Embodiment A video aggregation computer 1 having a first feature is a computer for displaying videos from a plurality of distributed observation devices 2 on a terminal 3 of a remote user. This video aggregation computer 1 includes an estimation unit 10 that estimates a network available bandwidth for the terminal 3, an allocation unit 11 that allocates a communication bandwidth to each of the plurality of observation devices 2 according to the estimated network available bandwidth or the resources of the terminal 3, and a transmission unit 23 that receives videos from the plurality of observation devices 2 using the communication bandwidth allocated to the plurality of observation devices 2 and transmits the received videos to the terminal 3. This allows the terminal 3 to display multiple videos with the amount of data received by the computer 1 from the plurality of observation devices 2 using the communication bandwidth allocated to the plurality of observation devices 2. Specifically, for example, the transmission unit 23 may receive videos from each observation device 2 with a data amount corresponding to the communication bandwidth allocated to each of the plurality of observation devices 2 and transmit the received videos to the terminal 3, and the terminal 3 may display the videos with the data amount transmitted from the transmission unit 23.

[0165] A computer 1 having the first feature can, when displaying images from multiple observation devices 2 on a remote user's terminal 3, cause the terminal 3 to display images appropriate to the environment, even in an environment where there is a limit to the amount of data transmission that can be used for the images.

[0166] The computer 1 receives video images using the communication bandwidth allocated to the plurality of observation devices 2 and transmits the received video images to the terminal 3. This allows the computer 1 to control the amount of data transmission from the numerous observation devices 2 scattered around the site. Therefore, in an environment where the amount of data transmission available for video images is limited, even with the bare minimum infrastructure, the computer 1 can display video images on the terminal 3 in accordance with the available network bandwidth or the resources of the terminal 3 while suppressing problems such as video delays and degradation of video quality.

[0167] In the computer 1 having the second feature, at least one of the plurality of observation devices 2 may be provided on a moving body.

[0168] Even if the observation device 2 is mounted on a mobile body and the available network bandwidth is prone to fluctuations, the computer 1 having the second feature can prevent problems such as packet loss and video delays caused by insufficient communication bandwidth.

[0169] It is preferable that the computer 1 having the third feature further includes an importance receiving unit 20 that receives the importance of the video, and the allocation unit 11 allocates a communication bandwidth to each of the multiple observation devices 2 according to the available network bandwidth and the importance, or according to the resources of the terminal 3 and the importance.

[0170] According to the computer 1 having the third feature, video images are received from the plurality of observation devices 2 using a communication bandwidth allocated to the plurality of observation devices 2 according to the available network bandwidth and the importance of the video, or using a communication bandwidth allocated to the plurality of observation devices 2 according to the resources of terminal 3 and the importance, and the received video images are transmitted to terminal 3. This allows the computer 1 to control the amount of data transmitted from the numerous observation devices 2 scattered around the site, while taking into consideration not only the available network bandwidth and the resources of terminal 3, but also the importance of the video. Therefore, in an environment where the amount of data transmission available for video is limited, even with the minimum necessary infrastructure, the computer 1 can display video images on terminal 3 according to the importance of the video, while suppressing problems such as video delays and degradation of video quality.

[0171] In the computer 1 having the fourth feature, it is preferable that the importance level receiving unit receives an input of the importance level from a user.

[0172] According to the computer 1 having the fourth feature, the user of the computer 1 or the user of the terminal 3 can set the importance of an image when switching to another image. Therefore, the user can input to increase or decrease the importance of an image showing a monitored object such as a machine operated by the user or a site monitored by the user, and the computer 1 can switch the image displayed on the terminal 3 in accordance with the user's input.

[0173] In a computer 1 having the fifth feature, the allocation unit 11 may determine whether the terminal 3 can process the amount of data corresponding to the network available bandwidth estimated by the estimation unit based on the performance of the resources of the terminal 3, and allocate a communication bandwidth to each of the multiple observation devices 2 based on the determination result.

[0174] Specifically, for example, in a computer 1 having the fifth feature, the allocation unit 11 determines whether the terminal 3 can process the amount of data corresponding to the network available bandwidth estimated by the estimation unit based on the resource performance of the terminal 3, and if it is determined that the terminal 3 can process the data, it may allocate a communication bandwidth to each of the multiple observation devices 2 according to the network available bandwidth, and if it is determined that the terminal 3 cannot process the data, it may allocate a communication bandwidth to each of the multiple observation devices 2 according to the resource performance of the terminal 3.

[0175] Furthermore, in a computer 1 having the fifth feature, the allocation unit 11 may determine whether the terminal 3 can process the amount of data corresponding to the network available bandwidth estimated by the estimation unit based on the resource performance of the terminal 3, and if it is determined that the terminal 3 can process the data, allocate a communication bandwidth to each of the multiple observation devices 2 according to the network available bandwidth and the importance, and if it is determined that the terminal 3 cannot process the data, allocate a communication bandwidth to each of the multiple observation devices 2 according to the resource performance of the terminal 3 and the importance.

[0176] In the computer 1 having the sixth feature, the terminal 3 may display multiple images on one screen.

[0177] According to the computer 1 having the sixth feature, multiple images transmitted from multiple observation devices 2 are displayed on a single screen on the terminal 3. Therefore, even in an environment where there is a limit to the amount of data transmission available for images, and even with the minimum necessary infrastructure, the user can monitor in detail on a single screen the locations, machines, and other monitored objects captured by the multiple images.

[0178] In a computer 1 having the seventh feature, the transmitting unit may transmit video data to the terminal so that the video captured by the observation device assigned the highest communication bandwidth on the terminal is displayed larger than other videos.

[0179] According to the computer 1 having the seventh feature, the image from the observation device to which the widest communication bandwidth is assigned is displayed large on the terminal 3. Therefore, even in an environment where there is a limit to the amount of data transmission that can be used for the image, even with the minimum necessary infrastructure, it becomes easy to monitor the location where the image that the user is most interested in, the monitored object such as a machine, etc.

[0180] In the computer 1 having the eighth feature, the allocation unit may increase the communication bandwidth allocated to an observation device located near the observation device to which the highest communication bandwidth has been allocated.

[0181] According to the computer 1 having the eighth feature, the terminal 3 can display video including other monitored objects such as other machines or other places that are close to a monitored object such as a machine or place that is attracting a lot of attention. This allows the user to simultaneously monitor not only the monitored object such as a machine or place that is attracting attention, but also the environment around this monitored object.

[0182] The computer 1 having the ninth feature preferably further includes a video storage unit for storing video.

[0183] According to the computer 1 having the ninth feature, for example, it is possible to store images captured by observation devices with high communication bandwidth, images of high importance, etc., and to switch images without delay even in a field with a poor communication environment by keeping the output of the images in a standby state in the computer 1. Furthermore, by storing images in the computer 1 (server), it is possible to search for images of the captured subject.

[0184] In the computer 1 having the tenth feature, the transmission unit may transmit the stored video to the terminal in accordance with the communication bandwidth.

[0185] According to the computer 1 having the tenth feature, for example, when the communication bandwidth allocated to the observation device capturing the stored video increases, the video that is on standby for transmission can be immediately accessed and transmitted to the terminal. Therefore, even in a site with a poor communication environment, the computer can automatically switch between videos without delay.

[0186] The computer 1 having the eleventh feature may further include an instruction unit 21 that instructs the observation device 2 to increase or decrease the image quality of the image by increasing or decreasing the resolution of the image.

[0187] According to the computer 1 having the eleventh feature, the observation device 2 can change the image quality of the video by changing the resolution of the video based on instructions received from the instruction unit 21 of the computer 1. Specifically, for example, the observation device 2 can deliver video with important details clearly by increasing the resolution of video with high importance, and can enable smooth viewing even in a slow network environment or on a device with limited processing power by decreasing the resolution of video with low importance.

[0188] The computer 1 having the twelfth feature may further include an instruction unit that increases or decreases the frame rate of the video to thereby increase or decrease the image quality of the video.

[0189] According to the computer 1 having the twelfth feature, the observation device 2 can change the image quality of the video by changing the frame rate of the video based on instructions received from the instruction unit 21 of the computer 1. Specifically, for example, the observation device 2 can distribute video with a lot of fast movement by increasing the frame rate of video with high importance, and reduce the frame rate of video with low importance to reduce the data size of the video, thereby enabling smooth viewing even in a slow network environment or an unstable network environment caused by the movement of a mobile object, etc. By adjusting the frame rate, the smoothness of the video (the smoothness of the movement contained in the video) can be adjusted.

[0190] The observation device 2 having the thirteenth characteristic comprises an instruction receiving unit 51 that receives instructions (image quality instructions) regarding the image quality of the image from the computer 1, an image quality setting unit 42 that sets the image quality of the image to be captured based on the received instructions, and an image providing unit 52 that provides the image of the set image quality to the image aggregation computer 1.

[0191] The observation device 2 having the thirteenth feature can set the image quality of the video based on the image quality instruction received from the computer 1 and provide the video with the set image quality to the computer 1. Specifically, for example, the observation device 2 may receive information regarding the communication bandwidth allocation from the computer 1 as the image quality instruction. In this case, based on the communication bandwidth allocation, the observation device 2 may set the image quality of the video from the observation device 2 allocated with a wide communication bandwidth (e.g., video of high importance) to high quality and the image quality of the video from the observation device 2 allocated with a narrow communication bandwidth (e.g., video of low importance) to low quality. This allows for control of the amount of data transmitted from multiple observation devices 2 scattered throughout the site. Therefore, even in an environment with limited data transmission capacity for video, even with minimal infrastructure, problems such as video delays and video quality degradation are suppressed. The computer 1 can then smoothly switch between videos depending on the communication bandwidth (e.g., the importance of the video).

[0192] The embodiments having the above first to thirteenth features are in the categories of computers, observation devices, methods, or programs, but similar actions and effects according to the category can also be achieved in other categories such as systems, which will be described later.

[0193] As described above, the video aggregation computer 1, observation device 2, video aggregation method, and program of this embodiment can display video from multiple observation devices on a remote user's terminal in a manner appropriate to the environment, even in an environment where there is a limit to the amount of data transmission that can be used for video.

[0194] The present invention further includes the following video aggregation system.

[0195] The video aggregation system comprises a plurality of observation devices 2 that are distributed and arranged, and a video aggregation computer 1, wherein the video aggregation computer 1 comprises an estimation unit 10 that estimates the network available bandwidth for a terminal 3, an allocation unit 11 that allocates a communication bandwidth to each of the plurality of observation devices 2 in accordance with the network available bandwidth estimated by the estimation unit 10 or the resources of the terminal 3, and a transmission unit 23 that receives video from the plurality of observation devices 2 using the communication bandwidth allocated to the plurality of observation devices 2 and transmits the received video to the terminal 3, and the observation device 2 comprises an instruction receiving unit 51 that receives instructions regarding the image quality of the video from the video aggregation computer 1, an image quality setting unit 42 that sets the image quality of the video to be captured based on the received instructions, and a video providing unit 52 that provides video of the set image quality to the video aggregation computer 1.

[0196] In this video aggregation system, when images from multiple observation devices are displayed on a remote user's terminal, the terminal can display images appropriate to the environment, even in an environment where there is a limit to the amount of data transmission that can be used for the images.

[0197] The video aggregation system may further include the terminal 3. In this case, the terminal 3 may display a plurality of videos with a data volume received by the video aggregation computer 1 from the plurality of observation devices 2 in the communication bandwidth allocated to the plurality of observation devices 2.

[0198] The video aggregation system may include the terminal 3 as a component, but the terminal 3 is not an essential component of the video aggregation system.

[0199] The present invention is not limited to the above-described embodiment, and includes the following modifications, for example.

[0200] [Variation 1] In the video switching systems according to the embodiments shown in Figures 1, 4, and 5, the site where the observation device 2 is located is a work site where a work machine 60 such as a shovel, crane, or bulldozer performs work, but this is not limited to the above-described embodiment. The site where the observation device 2 is located may also be, for example, a manufacturing site. In this case, a production line for manufacturing products from raw materials is located at the manufacturing site. In this production line, multiple processes are performed to manufacture products from raw materials. Multiple observation devices 2 are located at the manufacturing site to capture images of the status of each of the multiple processes.

[0201] Specifically, for example, the production line may include a first process, a second process, and a third process, and the multiple observation devices 2 may include a first observation device 2 for photographing the situation of the first process, a second observation device 2 for photographing the situation of the second process, and a third observation device 2 for photographing the situation of the third process.

[0202] In variant example 1, the configurations of the computer 1, the multiple observation devices 2, and the terminal 3 are the same as the configurations of the video aggregation computer 1, the multiple observation devices 2, and the terminal 3 in the embodiment described with reference to Figures 1 to 7, so detailed explanations of these will be omitted.

[0203] In this first modification, similarly to the above embodiment, the video aggregation computer 1 is configured to display video from a plurality of observation devices 2 that are distributed at a manufacturing site on a user's terminal 3. This computer 1 includes an estimation unit 10 that estimates a network available bandwidth for the terminal 3, an allocation unit 11 that allocates a communication bandwidth to each of the plurality of observation devices 2 in accordance with the estimated network available bandwidth or the resources of the terminal 3, and a transmission unit 23 that receives video from the plurality of observation devices 2 in the communication bandwidth allocated to the plurality of observation devices 2 and transmits the received video to the terminal 3.

[0204] The terminal 3 is a terminal for a remote user. When multiple observation devices 2 are installed at a manufacturing site, the terminal 3 may be installed at a location away from the manufacturing line and may be equipped with a display device that displays an image of the manufacturing site including the manufacturing line.

[0205] In this modified example 1, the computer 1 performs the arithmetic processing shown in Fig. 3 and the arithmetic processing shown in Fig. 6. Also, in this modified example 1, the observation device 2 may perform the arithmetic processing shown in Fig. 7. Also, in this modified example 1, the terminal 3 displays, for example, as shown in Fig. 1, Fig. 4 and / or Fig. 5 based on instructions from the computer 1.

[0206] Furthermore, the site where the observation device 2 is installed may be, for example, a site where monitoring and maintenance of infrastructure facilities such as a plant or power plant is required.

[0207] [Modification 2] In a video switching system according to Modification 2, the computer 1 performs processing to switch at least one of the image quality and size of the video output at the terminal 3 in accordance with at least one of a specific action and a specific video.

[0208] The site where the observation device 2 is placed is a work site, and a work machine 60 such as a shovel, a crane, or a bulldozer is placed at this work site.

[0209] The work machine 60 shown in Fig. 1 is, for example, a shovel. This work machine 60 includes a self-propelled lower traveling body 61, an upper rotating body 62 rotatably supported on the lower traveling body 61, and a working device 63 supported on the upper rotating body 62. In the specific example shown in Fig. 1, the lower traveling body 61 includes a crawler-type traveling device, but the traveling device may also include tires. The working device 63 includes, for example, a boom rotatably supported on the upper rotating body 62, an arm rotatably supported at the tip of the boom, and a bucket rotatably supported at the tip of the arm.

[0210] The work machine 60 is equipped with an attitude detector 64 for detecting the attitude of the work machine 60. The attitude detector 64 may include a boom attitude detector for detecting the attitude of the boom, an arm attitude detector for detecting the attitude of the arm, and a bucket attitude detector for detecting the attitude of the bucket. The attitude detector 64 may further include a rotating unit attitude detector for detecting the attitude of the upper rotating unit 62 relative to the undercarriage 61. The attitude detector 64 may further include an attitude detector that detects the degree of inclination of the work machine 60 relative to a horizontal plane.

[0211] The video switching system according to Modification 2 includes a remote control device. The remote control device includes a terminal 3 including a display device as shown in Figures 1, 4, and 5, and a remote controller 65 for remotely controlling the work machine 60. The remote controller 65 includes a plurality of remote control levers for moving the work implement 63 of the work machine 60 and rotating the upper rotating body 62. The remote controller 65 also includes at least one of an operation lever and an operation pedal (not shown) for driving the undercarriage 61 of the work machine 60. A user can remotely control the work machine 60 located at a work site by operating the remote controller 65 of the remote control device while viewing the screen of the display device of the terminal 3.

[0212] The computer 1 according to Modification 2 may perform processing to automatically switch at least one of the image quality and size of the video output on the terminal 3 in accordance with a specific operation of the work machine 60. The specific operation is stored in advance in the computer 1. The user inputs information to the input section of the terminal 3 shown in FIG. 2 to specify the specific operation, and the computer 1 sets the operation corresponding to the input as the specific operation.

[0213] The specific motion may be, for example, at least one of a motion of the working implement 63 and a motion of the upper rotating body 62. Specifically, the specific motion may be a traveling motion of the work machine 60 in a predetermined direction, a traveling motion of the work machine 60 going uphill, or a traveling motion of the work machine 60 going downhill. The specific motion may also be a boom motion, an arm motion, a bucket motion, or a swing motion of the upper rotating body 62. The specific motion may also be a motion in which the traveling speed of the work machine 60 is equal to or greater than a predetermined threshold, a motion in which the operating speed (rotation speed) of the boom, arm, or bucket of the working implement 63 is equal to or greater than a predetermined threshold, or a motion in which the swing speed of the upper rotating body 62 is equal to or greater than a predetermined threshold.

[0214] The computer 1 or the controller of the work machine 60 can determine whether the specific action has been performed based on the detection results input from a motion detection device for determining the specific action. The motion detection device may be, for example, the attitude detector 64, the observation device 2, the remote controller 65, or any other device. If the motion detection device is the observation device 2, the computer 1 or the controller of the work machine 60 may determine whether the specific action has been performed by image recognition of video acquired from the observation device 2. If the motion detection device is the attitude detector 64, the computer 1 or the controller of the work machine 60 can determine whether the specific action has been performed based on the detection results from the attitude detector 64. If the motion detection device is the remote controller 65, the computer 1 or the controller of the work machine 60 can determine whether the specific action has been performed based on the amount of lever operation applied to the remote controller 65.

[0215] When the specific action is performed, the computer 1 may perform processing to automatically switch at least one of the image quality and size of the video output on the terminal 3. Specifically, when the specific action is performed, the computer 1 may instruct the terminal 3 to increase the size of the video related to the specific action on the terminal 3 compared to before the specific action was performed, and the terminal 3 may increase the size of the video related to the specific action on the display device of the terminal 3 in accordance with the instruction compared to before the specific action was performed.

[0216] Furthermore, when the specific action is performed, the computer 1 instructs the observation device 2 that captures the video so that the image quality of the video related to the specific action is higher than before the specific action was performed, and the observation device 2 sets the image quality of the video to be higher in accordance with the instruction than before the specific action was performed, provides the video of the set image quality to the computer 1, and the computer 1 may output the video to the terminal 3.

[0217] Furthermore, the computer 1 may perform processing to switch at least one of the image quality and size of the image output on the terminal 3 in accordance with a specific image. The computer 1 stores the specific image in advance. The specific image may be, for example, an image whose contrast is equal to or lower than a predetermined threshold, or an image whose resolution is equal to or lower than a predetermined threshold. A user inputs the specific threshold into the input unit of the terminal 3 shown in FIG. 2 , and the computer 1 sets a value corresponding to the input as the specific threshold. The computer 1 or a controller of the terminal 3 can acquire the contrast of the image displayed on the terminal 3.

[0218] When the specific video is displayed on terminal 3, computer 1 may perform processing to automatically switch at least one of the image quality and size of the video output on terminal 3. Specifically, when the specific video is displayed on terminal 3, computer 1 may instruct terminal 3 to increase the size of the specific video on terminal 3 compared to before the specific video was displayed, and terminal 3 may increase the size of the specific video on the display device of terminal 3 in accordance with the instruction compared to before the specific video was displayed.

[0219] Furthermore, when the specific image is displayed on the terminal 3, the computer 1 may instruct the terminal 3 to increase the contrast of the specific image compared to before the specific image was displayed, and the terminal 3 may increase the contrast of the image in accordance with the instruction compared to before the specific image was displayed.

[0220] [Modification 3] The video switching system according to Modification 3 has the following preset function: That is, the computer 1 is configured to be able to save video display settings associated with specific identification conditions.

[0221] The image display setting items may include, for example, a setting item regarding the number of images to be displayed on terminal 3, a setting item regarding the size of the images to be displayed on terminal 3, and a setting item regarding the display position of the images on terminal 3.

[0222] The specific identification condition may be, for example, a condition related to an identifier (identification information) for identifying a user, or a condition related to an identifier (identification information) for identifying a target object such as a work machine, manufacturing equipment, or facility located at a work site. The specific identification condition may also be, for example, a condition related to the operating speed of a remotely controlled target such as the work machine 60, or a condition related to the characteristics of the image. The specific identification condition may also be a condition related to the content of an operation performed at a work site. The content of the operation may include, for example, a work machine operation such as an excavation operation or a reversing operation at a work site, or a vehicle operation such as a transport operation or a reversing operation at a manufacturing site. The specific identification condition may also be a condition related to weather, a condition related to a time period, or a condition related to the quality of the communication environment.

[0223] The user inputs to the input section of the terminal 3 shown in Figure 2 to specify the specific identification condition and the video display setting associated therewith, and the computer 1 associates and saves the specific identification condition with the video display setting based on the input. When the specific identification condition is met, the computer 1 causes the terminal 3 to output video using the video display setting associated with the identification condition. Therefore, the computer 1 can cause the terminal 3 to display video in a display mode that matches the specific identification condition, such as the user's preferences or the type of work machine 60.

[0224] While the above embodiment has been described primarily in terms of the case where the observation device 2 is installed at a work site, environments in which the amount of data transmission is limited may also be, for example, enclosed spaces such as underground spaces, tunnels, and factories, regions in Japan or overseas where communication infrastructure is not adequately developed, space, or regions with congested communications. Furthermore, environments in which the amount of data transmission is limited may also be, for example, environments in which at least one of the observation device 2, computer 1, and terminal 3 communicate while moving.

[0225] The communication means between the computer 1 and the observation device 2 and between the computer 1 and the terminal 3 are not limited to the above-mentioned internet line, a network such as a mobile phone network, or a network constructed by software virtualization. The communication means may be, for example, satellite communication, LPWA (Low Power Wide Area), or other communication means. Satellite communication is a communication method for data communication between the ground and an artificial satellite. LPWA is suitable, for example, when the observation device 2 and the computer 1 are relatively close to each other, or when the computer 1 and the terminal 3 are relatively close to each other, and is expected to have a cost-reducing effect.

[0226] REFERENCE SIGNS LIST 1 Video aggregation computer 2 Observation device 3 Terminal 4 Video channel 5 Port 10 Estimation unit 11 Allocation unit 12 Determination unit 20 Importance reception unit 21 Instruction unit 22 Reception unit 23 Transmission unit 30 Video storage unit 40 Information detection unit 41 Importance setting unit 42 Image quality setting unit 43 Acceleration detection unit 44 Learning unit 50 Importance provision unit 51 Instruction reception unit 52 Video provision unit

Claims

1. A video aggregation computer for displaying images from multiple distributed observation devices on a terminal, comprising: an estimation unit that estimates the network available bandwidth for the terminal; an allocation unit that allocates communication bandwidth to each of the multiple observation devices in accordance with the network available bandwidth estimated by the estimation unit or the resources of the terminal; and a transmission unit that receives images from the multiple observation devices using the communication bandwidth allocated to the multiple observation devices and transmits the received images to the terminal.

2. The image aggregation computer according to claim 1, wherein at least one of said plurality of observation devices is provided on a moving body.

3. A video aggregation computer as described in claim 1 or 2, further comprising an importance receiving unit that receives the importance of the video, and wherein the allocation unit allocates communication bandwidth to each of the multiple observation devices according to the available network bandwidth and the importance, or according to the resources of the terminal and the importance.

4. The video aggregation computer according to claim 3, wherein the importance level receiving unit receives an input of the importance level from a user.

5. A video aggregation computer as described in claim 1 or 2, wherein the allocation unit determines whether the terminal can process the amount of data corresponding to the network available bandwidth estimated by the estimation unit based on the performance of the terminal's resources, and allocates communication bandwidth to each of the multiple observation devices based on the determination result.

6. The video aggregation computer according to any one of claims 1 to 5, wherein the terminal displays multiple videos on one screen.

7. A video aggregation computer as described in any one of claims 1 to 6, wherein the transmitting unit transmits video data to the terminal so that the video captured by the observation device assigned the highest communication bandwidth on the terminal is displayed larger than other videos.

8. A video aggregation computer according to any one of claims 1 to 7, wherein the allocation unit increases the communication bandwidth allocated to an observation device located in the vicinity of an observation device to which the highest communication bandwidth has been allocated.

9. The video aggregation computer according to any one of claims 1 to 8, further comprising a video storage unit for storing video.

10. The video aggregation computer according to claim 9, wherein the transmitting unit transmits the stored video to the terminal in accordance with the communication bandwidth.

11. An image aggregation computer according to any one of claims 1 to 10, further comprising an instruction unit that instructs the observation device to increase or decrease the image quality of the image by increasing or decreasing the resolution of the image.

12. An image aggregation computer according to any one of claims 1 to 10, further comprising an instruction unit that instructs the observation device to increase or decrease the frame rate of the image, thereby increasing or decreasing the image quality of the image.

13. An observation device communicably connected to a video aggregation computer described in any one of claims 1 to 12, comprising: an instruction receiving unit that receives instructions regarding the image quality of the video from the video aggregation computer; an image quality setting unit that sets the image quality of the video to be captured based on the received instructions; and an image providing unit that provides the video of the set image quality to the video aggregation computer.

14. A video aggregation method for displaying video from multiple distributed observation devices on a terminal, comprising: a step in which a computer estimates a network available bandwidth for the terminal; a step in which the computer allocates a communication bandwidth to each of the multiple observation devices according to the estimated network available bandwidth or the resources of the terminal; a step in which the computer receives video from the multiple observation devices using the communication bandwidth allocated to the multiple observation devices and transmits the received video to the terminal; and a step in which the terminal displays video using the amount of data received by the computer from the multiple observation devices using the communication bandwidth allocated to the multiple observation devices.

15. A computer-readable program that causes a computer for displaying images from multiple distributed observation devices on a terminal to execute the following steps: estimating the network available bandwidth for the terminal; allocating a communication bandwidth to each of the multiple observation devices in accordance with the estimated network available bandwidth or the resources of the terminal; and receiving images from the multiple observation devices using the communication bandwidth allocated to the multiple observation devices and transmitting the received images to the terminal.

Images