Video transmission system, video transmission method, and video receiving device

Abstract

A video transmission system (2) is configured to comprise: a video data acquisition unit (11) that acquires first video data indicating a first video captured by a camera (1); a first inference unit (12) that acquires intermediate data, which is data different from the first video data, from a first learning model (13) by imparting to the first learning model (13) the first video data acquired by the video data acquisition unit (11); a data sending unit (14) that sends the intermediate data acquired by the first inference unit (12); a data reception unit (15) that receives the intermediate data sent from the data sending unit (14); and a second inference unit (16) that imparts to a second learning model (17) the intermediate data received by the data reception unit (15) to acquire, from the second learning model (17), second video data indicating a predicted video of a second video for which, compared to the first video, the time-of-imaging by the camera (1) is farther ahead in time by at least the transmission time of the intermediate data from the data sending unit (14) to the data reception unit (15).

Description

[Technical field]

[0001] The present disclosure relates to a video transmission system, a video transmission method, and a video receiving device. [Background technology]

[0002] A worker in a remote location may issue work instructions to a site where a certain task is being performed. For example, the worker in the remote location may issue work instructions while checking the site situation by viewing an image of the site displayed on a monitor. The work instructions are instructions to operate machines such as robots, cars, and cameras that are present at the site. 2. Description of the Related Art There is a video transmission system that transmits video data showing on-site video to a remote location.

[0003] Incidentally, Patent Document 1 discloses a technique for shortening the transmission time of video data from when the video data is transmitted from a transmitting unit until when the video data is received by a receiving unit. The transmitting unit compresses the video data and transmits the compressed video data to the receiving unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-29746 A Summary of the Invention [Problem to be solved by the invention]

[0005] In a video transmission system, it is impossible to reduce the transmission time of video data from the local site to a remote site to zero. If the transmission time of video data is not zero, delays will occur in the confirmation of the local situation by workers in remote locations. If there is a delay in confirming the local situation, the work instructions given by the workers will also be delayed, which creates the problem that the workers may give inappropriate work instructions. Even with the technology disclosed in Patent Document 1, it is impossible to reduce the transmission time of video data to zero. Therefore, even if the technology could be applied to a video transmission system, the above problem cannot be solved.

[0006] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a video transmission system and a video transmission method that can assist workers in remote locations in issuing appropriate work instructions to those on-site. [Means for solving the problem]

[0007] A video transmission system according to the present disclosure includes a video data acquisition unit that acquires first video data indicating a first video captured by a camera, a first inference unit that provides the first video data acquired by the video data acquisition unit to a first learning model and acquires intermediate data, which is data different from the first video data, from the first learning model, a data transmission unit that transmits the intermediate data acquired by the first inference unit, and a data receiving unit that receives the intermediate data transmitted from the data transmission unit. a transmission time determination unit that determines a transmission time of the intermediate data from a transmission time of the intermediate data by the data transmission unit and a reception time of the intermediate data by the data reception unit; The intermediate data received by the data receiving unit is given to a second learning model, and the second learning model determines that the camera's shooting time is longer than the first image. Also and a second inference unit that acquires second image data indicating a predicted image of the second image that is ahead of the transmission time of the inter-image data or more. Effect of the Invention

[0008] According to the present disclosure, workers in remote locations can be assisted in issuing appropriate work instructions to the local area. [Brief description of the drawings]

[0009] [Figure 1] 1 is a configuration diagram showing a video transmission system 2 according to a first embodiment. [Diagram 2] 2 is a hardware configuration diagram showing hardware of a video transmitting device 3 included in a video transmission system 2 according to the first embodiment. FIG. [Diagram 3] 2 is a hardware configuration diagram showing hardware of a video receiving device 5 included in the video transmission system 2 according to the first embodiment. FIG. [Figure 4] FIG. 2 is an explanatory diagram showing an example of a learning model 30. [Diagram 5] FIG. 13 is a hardware configuration diagram of a computer in the case where the video transmitting device 3 or the video receiving device 5 is realized by software, firmware, or the like. [Figure 6] 10 is a flowchart showing a processing procedure of the video transmitting device 3, which is part of a video transmission method. [Figure 7] 10 is a flowchart showing a processing procedure of the video receiving device 5, which is part of a video transmission method. [Figure 8] FIG. 1 is an explanatory diagram showing a plurality of images captured by a camera 1 at different times. [Figure 9] FIG. 11 is a configuration diagram showing a video transmission system 2 according to a second embodiment. [Figure 10] FIG. 11 is a hardware configuration diagram showing hardware of a video receiving device 5 included in a video transmission system 2 according to a second embodiment. [Figure 11] FIG. 11 is a configuration diagram showing a video transmission system 2 according to a third embodiment. [Figure 12] FIG. 11 is a hardware configuration diagram showing hardware of a video receiving device 5 included in a video transmission system 2 according to a third embodiment. [Figure 13] FIG. 11 is a configuration diagram showing a video transmission system 2 including a video receiving device 5 according to a fourth embodiment. [Figure 14] 14 is a hardware configuration diagram showing hardware of a video transmitting device 3 included in the video transmission system 2 shown in FIG. [Figure 15] 14 is a hardware configuration diagram showing hardware of a video receiving device 5 included in the video transmission system 2 shown in FIG. [Figure 16] FIG. 11 is a configuration diagram showing another video transmission system 2 according to the first to fourth embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] In order to describe the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0011] Embodiment 1 FIG. 1 is a configuration diagram showing a video transmission system 2 according to the first embodiment. FIG. 2 is a hardware configuration diagram showing hardware of the video transmitting device 3 included in the video transmission system 2 according to the first embodiment. FIG. 3 is a hardware configuration diagram showing hardware of the video receiving device 5 included in the video transmission system 2 according to the first embodiment. 1, a camera 1 captures an image of a subject. The subject is an object to be captured by the camera 1, and corresponds to natural scenery, flowers, insects, animals, people, buildings, roads, automobiles, trains, airplanes, and the like. The camera 1 outputs to the video transmission system 2 first video data representing a first video in which a subject is captured.

[0012] The video transmission system 2 includes a video transmitting device 3, a transmission path 4, and a video receiving device 5. The video transmitting device 3 is the video transmitting side, and the video receiving device 5 is the video receiving side. The video transmission device 3 includes a video data acquisition unit 11, a first inference unit 12, and a data transmission unit . The transmission line 4 is a wired transmission line or a wireless transmission line. One end of the transmission line 4 is connected to the video transmitting device 3 , and the other end of the transmission line 4 is connected to the video receiving device 5 . 1 includes a transmission path 4. However, this is merely an example, and the transmission path 4 may be provided outside the video transmission system 2, and the video transmission system 2 may include a video transmitting device 3 and a video receiving device 5. The video receiving device 5 includes a data receiving unit 15 and a second inference unit 16 .

[0013] The video data acquisition unit 11 is realized by, for example, a video data acquisition circuit 21 shown in FIG. The video data acquisition unit 11 acquires the first video data output from the camera 1 . The video data acquisition unit 11 outputs the first video data to the first inference unit 12.

[0014] The first inference unit 12 is realized by, for example, a first inference circuit 22 shown in FIG. The first inference unit 12 includes a first learning model 13 . The first inference unit 12 provides the first video data acquired by the video data acquisition unit 11 to the first learning model 13, and acquires intermediate data, which is data different from the first video data, from the first learning model 13. The intermediate data is data in the middle stage until the first video data is converted into second video data, which will be described later, in the learning model 30 shown in FIG. 1, it is assumed that the amount of intermediate data is smaller than the amount of first video data. However, this is merely an example, and the amount of intermediate data is not limited to being smaller than the amount of first video data. The first inference unit 12 outputs the intermediate data to the data transmission unit 14. 1, the first inference unit 12 includes a first learning model 13. However, this is merely an example, and the first learning model 13 may be provided outside the first inference unit 12.

[0015] Each of the first learning model 13 and a second learning model 17 described below is a part of a learning model 30 shown in FIG. FIG. 4 is an explanatory diagram showing an example of the learning model 30. As shown in FIG. The learning model 30 is realized by, for example, a neural network. The learning model 30 includes an input layer 31, M intermediate layers 32-1 to 32-M, N intermediate layers 33-1 to 33-N, and an output layer 34. Each of M and N is an integer of 2 or more. The learning model 30 shown in Fig. 4 includes M intermediate layers 32-1 to 32-M. However, this is merely an example, and the learning model 30 may include only the intermediate layer 32-1 of the M intermediate layers 32-1 to 32-M. Moreover, the learning model 30 shown in Fig. 4 includes N intermediate layers 33-1 to 33-N. However, this is merely an example, and the learning model 30 may include only the intermediate layer 33-1 of the N intermediate layers 33-1 to 33-N.

[0016] During learning, the first video data acquired by the video data acquisition unit 11 is provided to the input layer 31 of the learning model 30. In addition, a predicted video of a second video, the shooting time of which by the camera 1 is earlier than that of the first video by at least the transmission time of the intermediate data, is provided to the learning model 30 as teacher data. The transmission time of the intermediate data is the time it takes for the intermediate data to reach the data receiving unit 15 from the data transmitting unit 14. In the video transmission system 2 shown in FIG. 1, the transmission time of the intermediate data is fixed, and it is assumed that the transmission time of the intermediate data is a given value in the video transmission system 2. If the processing time in each of the video data acquisition unit 11, the first inference unit 12, the data transmission unit 14, the data receiving unit 15 and the second inference unit 16 is short enough to be negligible, video data showing a predicted image of the second image, the shooting time of the camera 1 being earlier than the first image by the transmission time of the intermediate data, is provided to the learning model 30 as teacher data. On the other hand, if the respective processing times are not so short that they can be ignored, video data showing a predicted image of the second image, whose shooting time by camera 1 is ahead of the first image by the total time of each processing time and the transmission time of the intermediate data, is provided to the learning model 30 as training data. The learning model 30 is trained so that, when first image data is provided to the input layer 31, second image data showing a predicted image of a second image whose capture time is earlier than the first image by the transmission time of the intermediate data is output from the output layer 34, provided that the processing time for each is short enough to be ignored. The learning model 30 is trained so that, when first image data is provided to the input layer 31, second image data showing a predicted image of a second image whose capture time is earlier than the first image by the total time of each processing time and the transmission time of the intermediate data is output from the output layer 34, provided that the respective processing times are not so short as to be negligible.

[0017] The input layer 31 of the learning model 30 has, for example, the same number of input terminals as the number of pixels constituting the first image. The first image data is data indicating the pixel values ​​of each pixel, and each pixel value is provided to each input terminal in the input layer 31. The output layer 34 of the learning model 30 has, for example, the same number of output terminals as the number of pixels constituting the predicted image of the second image. The second image data is data indicating the pixel values ​​of each pixel, and each pixel value is output from each output terminal in the output layer 34. The first learning model 13 includes an input layer 31 and M intermediate layers 32-1 to 32-M, which are included in the learned learning model 30. The second learning model 17 includes an intermediate layer 32-M, N intermediate layers 33-1 to 33-N, and an output layer , which are included in the learned learning model 30. The technique for generating the first learning model 13 so as to include the input layer 31 and M intermediate layers 32-1 to 32-M, and the technique for generating the second learning model 17 so as to include the intermediate layer 32-M, N intermediate layers 33-1 to 33-N, and the output layer 34 are themselves publicly known techniques.

[0018] The data transmission unit 14 is realized by, for example, a data transmission circuit 23 shown in FIG. The data transmitting unit 14 transmits the intermediate data acquired by the first inference unit 12 to the data receiving unit 15 via the transmission path 4 .

[0019] The data receiving unit 15 is realized by, for example, a data receiving circuit 24 shown in FIG. The data receiving unit 15 receives the intermediate data transmitted from the data transmitting unit 14 . The data receiving unit 15 outputs the intermediate data to the second inference unit 16 .

[0020] The second inference unit 16 is realized by, for example, a second inference circuit 25 shown in FIG. The second inference unit 16 includes a second learning model 17 . The second inference unit 16 provides the intermediate data received by the data receiving unit 15 to a second learning model 17 and obtains from the second learning model 17 second image data indicating a predicted image of a second image whose shooting time by camera 1 is earlier than the first image by more than the transmission time of the intermediate data. 1, for ease of explanation, it is assumed that the processing times of the video data acquisition unit 11, the first inference unit 12, the data transmission unit 14, the data reception unit 15, and the second inference unit 16 can be ignored. In this case, the second inference unit 16 provides the intermediate data received by the data reception unit 15 to the second learning model 17, and acquires from the second learning model 17 second video data indicating a predicted video of a second video whose shooting time by the camera 1 is ahead of the first video by the transmission time of the intermediate data. When the respective processing times cannot be ignored, the second inference unit 16 provides the intermediate data received by the data receiving unit 15 to a second learning model 17, and obtains from the second learning model 17 second image data indicating a predicted image of a second image in which the shooting time of the camera 1 is ahead of the first image by the total time of each processing time and the transmission time of the intermediate data. The second inference unit 16 outputs the second video data to, for example, the display device 6 or a video processing device (not shown). 1, the second inference unit 16 includes the second learning model 17. However, this is merely an example, and the second learning model 17 may be provided outside the second inference unit 16.

[0021] The display device 6 displays the predicted image of the second image indicated by the second image data output from the second inference section 16 on a monitor. The image processing device (not shown) analyzes the subject and the like captured in the predicted image of the second image according to the second image data output from the second inference unit 16 .

[0022] In Fig. 1, it is assumed that each of the components of the video transmission device 3, that is, the video data acquisition unit 11, the first inference unit 12, and the data transmission unit 14, is realized by dedicated hardware as shown in Fig. 2. That is, it is assumed that the video transmission device 3 is realized by a video data acquisition circuit 21, a first inference circuit 22, and a data transmission circuit 23. 1, it is assumed that the data receiving unit 15 and the second inference unit 16, which are components of the video receiving device 5, are each realized by dedicated hardware as shown in Fig. 3. That is, it is assumed that the video receiving device 5 is realized by a data receiving circuit 24 and a second inference circuit 25. Each of the video data acquisition circuit 21, the first inference circuit 22, the data transmission circuit 23, the data receiving circuit 24 and the second inference circuit 25 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these.

[0023] The components of the video transmission device 3 are not limited to those realized by dedicated hardware, and the video transmission device 3 may be realized by software, firmware, or a combination of software and firmware. Furthermore, the components of the video receiving device 5 are not limited to those realized by dedicated hardware, and the video receiving device 5 may be realized by software, firmware, or a combination of software and firmware. The software or firmware is stored as a program in the memory of a computer. The computer means hardware that executes the program, and includes, for example, a CPU (Central Processing Unit), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor).

[0024] FIG. 5 is a hardware configuration diagram of a computer in the case where the video transmitting device 3 or the video receiving device 5 is realized by software, firmware, or the like. When the video transmission device 3 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures in the video data acquisition unit 11, the first inference unit 12, and the data transmission unit 14 is stored in the memory 41. Then, a processor 42 of the computer executes the program stored in the memory 41. When the video receiving device 5 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures in the data receiving unit 15 and the second inference unit 16 is stored in the memory 41. Then, a processor 42 of the computer executes the program stored in the memory 41.

[0025] Fig. 2 shows an example in which each of the components of the video transmission device 3 is realized by dedicated hardware, and Fig. 5 shows an example in which the video transmission device 3 is realized by software, firmware, etc. However, this is merely one example, and some of the components in the video transmission device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc. Fig. 3 shows an example in which each of the components of the video receiving device 5 is realized by dedicated hardware, and Fig. 5 shows an example in which the video receiving device 5 is realized by software, firmware, etc. However, this is merely one example, and some of the components in the video receiving device 5 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0026] Next, the operation of the video transmission system 2 shown in FIG. 1 will be described. FIG. 6 is a flowchart showing a processing procedure of the video transmitting device 3, which is part of a video transmission method. FIG. 7 is a flowchart showing a processing procedure of the video receiving device 5, which is part of a video transmission method.

[0027] During learning, in addition to the first video data acquired by the video data acquisition unit 11, the learning model 30 shown in Figure 4 is provided with video data indicating a predicted video of a second video, the shooting time of which by the camera 1 is earlier than that of the first video by the transmission time of the intermediate data, as teacher data. The learning model 30 is trained to obtain a future image for a certain image as a predicted image when the certain image is given, using an image sequence for the first image represented by each of the plurality of first image data and an image sequence for a predicted image represented by each of the plurality of teacher data. That is, the learning model 30 is trained to output, when the first image data is given to the input layer 31, image data indicating a predicted image of a second image whose shooting time by the camera 1 is earlier than that of the first image by the transmission time of the intermediate data from the output layer 34. FIG. 8 is an explanatory diagram showing a plurality of images captured by the camera 1 at different times. 8, T0 to T6 are the shooting times of the video images. The shooting time T0 is the oldest shooting time among the shooting times T0 to T6, and the shooting time T6 is the newest shooting time among the shooting times T0 to T6. The time difference ΔT between each of the photographing times T0 to T6 is expressed by the following formula (1). ΔT =T1-T0=T2-T1=T3-T2=T4-T3=T5-T4=T6-T5 (1) For example, if the transmission time of the intermediate data is equal to the time 3×ΔT (shooting time T3−shooting time T0), when first image data showing a first image at shooting time T0 is provided to the input layer 31, second image data showing an image at shooting time T3 (=T0+3×ΔT) is output from the output layer 32 as a predicted image of the second image. Also, when first image data showing a first image at shooting time T1 is provided to the input layer 31, second image data showing an image at shooting time T4 (=T1+3×ΔT) is output from the output layer 34 as a predicted image of the second image.

[0028] The trained learning model 30 includes an input layer 31, M intermediate layers 32-1 to 32-M, N intermediate layers 33-1 to 33-N, and an output layer . The first learning model 13 and the second learning model 17 are generated by dividing the trained learning model 30 so that each of the first learning model 13 and the second learning model 17 includes a common intermediate layer 32-M. That is, a first learning model 13 is generated to include an input layer 31 and M intermediate layers 32-1 to 32-M, and a second learning model 17 is generated to include an intermediate layer 32-M, N intermediate layers 33-1 to 33-N, and an output layer 34.

[0029] The camera 1 outputs first video data representing a first video to a video data acquisition unit 11 of the video transmission system 2. The video data acquisition unit 11 acquires the first video data output from the camera 1 (step ST1 in FIG. 6). The video data acquisition unit 11 outputs the first video data to the first inference unit 12.

[0030] The first inference unit 12 acquires the first video data from the video data acquisition unit 11 . The first inference unit 12 provides the first video data to the first learning model 13, and acquires intermediate data, which is data different from the first video data, from the first learning model 13 (step ST2 in FIG. 6). That is, the first inference unit 12 provides the first video data to the input layer 31, and acquires intermediate data from the intermediate layer 32-M. The first inference unit 12 outputs the intermediate data to the data transmission unit 14. A program that executes a general compression process on video data has a syntax that executes a branch, and therefore the processing time of the compression process may vary. The first inference unit 12 obtains intermediate data by providing the first video data to the first learning model 13, and is not a program that executes a syntax that executes a branch. Therefore, the processing time of the first inference unit 12 does not vary as much as the general compression process. As described above, in the video transmission system 2 shown in FIG. 1, for the sake of simplicity, the processing time of the first inference unit 12 is ignored.

[0031] The data transmission unit 14 acquires the intermediate data from the first inference unit 12 . The data transmitting unit 14 transmits the intermediate data to the data receiving unit 15 via the transmission path 4 (step ST3 in FIG. 6).

[0032] The data receiving unit 15 receives the intermediate data transmitted from the data transmitting unit 14 (step ST11 in FIG. 7). The data receiving unit 15 outputs the intermediate data to the second inference unit 16 .

[0033] The second inference unit 16 acquires the intermediate data from the data receiving unit 15 . The second inference unit 16 provides the intermediate data to the second learning model 17, and acquires the second video data from the second learning model 17 (step ST12 in FIG. 7). That is, the second inference unit 16 provides the intermediate data to the intermediate layer 32-M, and obtains the second image data from the output layer . The transmission time of the intermediate data is assumed to be, for example, equal to the time 3×ΔT of the shooting time T3−the shooting time T0. In this case, for example, when the first image data showing the first image at the shooting time T0 is provided to the input layer 31 of the first learning model 13, the second image data showing the image at the shooting time T3 (=T0+3×ΔT) is output from the output layer 34 of the second learning model 17 as the predicted image of the second image. For example, when first image data showing a first image captured at a shooting time T2 is provided to the input layer 31 of the first learning model 13, second image data showing an image captured at a shooting time T5 (= T2 + 3 × ΔT) is output from the output layer 34 of the second learning model 17 as a predicted image of the second image. Therefore, the image represented by the second image data output from the output layer 34 of the second learning model 17 is a predicted image of the second image captured by camera 1 at a time earlier than the first image, i.e., a predicted image of the image captured in real time by camera 1. The second inference unit 16 outputs the second video data to, for example, the display device 6 or a video processing device (not shown).

[0034] The display device 6 displays the predicted image indicated by the second image data output from the second inference section 16 on a monitor. Workers in remote locations can check the local situation by viewing the predicted image displayed on a monitor.

[0035] In the above-described first embodiment, the video transmission system 2 is configured to include a video data acquisition unit 11 that acquires first video data showing a first video, a first inference unit 12 that provides the first video data acquired by the video data acquisition unit 11 to a first learning model 13 and acquires intermediate data, which is data different from the first video data, from the first learning model 13, a data transmission unit 14 that transmits the intermediate data acquired by the first inference unit 12, a data reception unit 15 that receives the intermediate data transmitted from the data transmission unit 14, and a second inference unit 16 that provides the intermediate data received by the data reception unit 15 to a second learning model 17 and acquires from the second learning model 17 second video data showing a predicted video of a second video in which the shooting time of the camera 1 is ahead of the first video by at least the transmission time of the intermediate data from the data transmission unit 14 to the data reception unit 15. Therefore, the video transmission system 2 can support a worker or the like in a remote location to give an appropriate work instruction to the site.

[0036] In the video transmission system 2 shown in Fig. 1, the trained learning model 30 is realized by a trained neural network. However, the trained learning model 30 is not limited to being realized by a trained neural network, and may be realized by trained deep learning, for example. Therefore, each of the first learning model 13 and the second learning model 17 may be realized by, for example, a part of trained deep learning.

[0037] Embodiment 2 In the second embodiment, a video transmission system 2 including a transmission time specifying unit 18 that specifies the transmission time of intermediate data from a data transmitting unit 14 to a data receiving unit 15 will be described.

[0038] In the video transmission system 2 shown in FIG. 1, the transmission time of the intermediate data is fixed and is a known value. In the video transmission system 2 shown in FIG. 9, it is assumed that the transmission time of the intermediate data is variable and is not a predetermined value.

[0039] FIG. 9 is a configuration diagram showing a video transmission system 2 according to the second embodiment. FIG. 10 is a hardware configuration diagram showing hardware of a video receiving device 5 included in a video transmission system 2 according to the second embodiment. 9 and 10, the same reference numerals as those in FIGS. 1 and 3 denote the same or corresponding parts, and therefore the description thereof will be omitted. The hardware of the video transmitting device 3 included in the video transmission system 2 shown in Fig. 9 is similar to the hardware of the video transmitting device 3 included in the video transmission system 2 shown in Fig. 1. Therefore, the hardware configuration diagram of the video transmitting device 3 included in the video transmission system 2 shown in Fig. 9 is that of Fig. 2.

[0040] The transmission time specifying unit 18 is realized by, for example, a transmission time specifying circuit 26 shown in FIG. The transmission time specifying unit 18 specifies the transmission time Time of the intermediate data from the transmission time of the intermediate data by the data transmitting unit 14 and the reception time of the intermediate data by the data receiving unit 15. For example, the video transmitting device 3 and the video receiving device 5 are time-synchronized, and a timestamp indicating the time when the first video data was acquired by the video data acquiring unit 11 is added to the intermediate data transmitted from the data transmitting unit 14. The transmission time determining unit 18 calculates, as the transmission time Time, the difference between the time Tr when the intermediate data was received by the data receiving unit 15 and the time Ts indicated by the timestamp. The transmission time specifying unit 18 outputs the transmission time Time to the second inference unit 19 .

[0041] The second inference unit 19 is realized by, for example, a second inference circuit 27 shown in FIG. The second inference unit 19 includes a second learning model 17 . The second inference unit 19 acquires second video data from the second learning model 17, the second video data indicating a predicted video of a second video in which the shooting time of the camera 1 is earlier than the first video by more than the transmission time Time identified by the transmission time identification unit 18. 9, for ease of explanation, it is assumed that the processing times of the video data acquisition unit 11, the first inference unit 12, the data transmission unit 14, the data reception unit 15, and the second inference unit 19 can be ignored. In this case, the second inference unit 19 provides the intermediate data received by the data reception unit 15 to the second learning model 17, and acquires, from the second learning model 17, second video data indicating a predicted video of a second video in which the shooting time of the camera 1 is ahead of the first video by the transmission time Time specified by the transmission time specification unit 18. When the respective processing times cannot be ignored, the second inference unit 19 provides the intermediate data received by the data receiving unit 15 to the second learning model 17, and obtains from the second learning model 17 second image data indicating a predicted image of the second image in which the shooting time of the camera 1 is ahead of the first image by the sum of the respective processing times and the transmission time Time of the intermediate data. The second inference unit 19 outputs the second video data to, for example, the display device 6 or a video processing device (not shown).

[0042] In Fig. 9, it is assumed that the data receiving unit 15, the transmission time specifying unit 18, and the second inference unit 19, which are components of the video receiving device 5, are each realized by dedicated hardware as shown in Fig. 10. That is, it is assumed that the video receiving device 5 is realized by a data receiving circuit 24, a transmission time specifying circuit 26, and a second inference circuit 27. Each of the data receiving circuit 24, the transmission time determining circuit 26 and the second inference circuit 27 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0043] The components of the video receiving device 5 are not limited to those realized by dedicated hardware, and the video receiving device 5 may be realized by software, firmware, or a combination of software and firmware. When the video receiving device 5 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures in the data receiving unit 15, the transmission time identifying unit 18, and the second inference unit 19 is stored in a memory 41 shown in Fig. 5. Then, a processor 42 shown in Fig. 5 executes the program stored in the memory 41.

[0044] Fig. 10 shows an example in which each of the components of the video receiving device 5 is realized by dedicated hardware, and Fig. 5 shows an example in which the video receiving device 5 is realized by software, firmware, etc. However, this is merely one example, and some of the components in the video receiving device 5 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0045] Next, the operation of the video transmission system 2 shown in FIG. 9 will be described. In the video transmission system 2 shown in FIG. 9, in addition to the first video data acquired by the video data acquisition unit 11 during learning, the video data acquisition unit 11 also acquires, as teacher data, a video data set that indicates that the shooting time of the camera 1 is longer than that of the first video and that is the maximum expected transmission time Time max When the first video data is provided to the input layer 31, the learning model 30 outputs from the output layer 34 the maximum transmission time Time max The second image data showing an image whose shooting time is advanced by only the first image data is output.

[0046] The video transmission device 3 shown in FIG. 9 operates in the same manner as the video transmission device 3 shown in FIG. However, the data transmitting unit 14 of the video transmitting device 3 shown in Figure 9 adds a timestamp indicating the time Ts at which the first video data was acquired by the video data acquiring unit 11 to the intermediate data, and transmits the time-stamped intermediate data to the video receiving device 5 via the transmission path 4.

[0047] In the video receiving device 5 shown in Figure 9, the second inference unit 19 performs the following pre-processing before starting the process of acquiring second video data from the second learning model 17 and outputting the second video data to the display device 6, etc. The pre-processing by the video receiving device 5 will now be described in detail. The data receiving unit 15 receives the intermediate data with the time stamp transmitted from the data transmitting unit 14 . The data receiving unit 15 outputs the intermediate data to the second inference unit 19 .

[0048] The second inference unit 19 acquires the intermediate data from the data receiving unit 15 . The second inference unit 19 provides the intermediate data to the second learning model 17, and infers from the second learning model 17 that the shooting time of the camera 1 is closer to the maximum transmission time Time max Then, second image data showing an image that advances by the same amount is acquired. The second inference unit 19 stores the acquired second video data in an internal memory.

[0049] For the sake of convenience, the maximum transmission time is Time max is assumed to be equal to the shooting time T9-shooting time T0 (time 9×ΔT). In this case, the second inference unit 19, for example, provides intermediate data relating to the first image captured at the shooting time T0 to the second learning model 17, and obtains second image data indicating the image captured at the shooting time T9 (=T0+9×ΔT) from the second learning model 17, and stores the second image data in the internal memory. For example, when intermediate data relating to a first video image captured at a shooting time T1 is provided to the second learning model 17, the second inference unit 19 infers from the second learning model 17 that the first video image captured at the shooting time T 10 Second image data showing an image at (=T1+9×ΔT) is acquired, and the second image data is stored in the internal memory. For example, when intermediate data relating to a first video image captured at a shooting time T2 is provided to the second learning model 17, the second inference unit 19 infers from the second learning model 17 that the first video image captured at the shooting time T 11Second image data showing an image at (=T2+9×ΔT) is acquired, and the second image data is stored in the internal memory. For example, when intermediate data relating to the first video at the shooting time T8 is provided to the second learning model 17, the second inference unit 19 infers from the second learning model 17 that the first video at the shooting time T 17 Second image data showing an image of (=T8+9×ΔT) is acquired, and the second image data is stored in the internal memory. As a result, nine pieces of second video data are stored in the internal memory of the second inference unit 19, and the pre-processing by the video receiving device 5 is completed. 10 Footage of, taken at time T 11 Image of, ..., taken at time T 17 Then, the second video data representing each of the above videos is stored, and the pre-processing by the video receiving device 5 ends.

[0050] Next, the video receiving device 5 after the preprocessing is completed will be described. The data receiving unit 15 receives the intermediate data with the time stamp transmitted from the data transmitting unit 14 . The data receiving unit 15 outputs the intermediate data with the time stamp to the second inference unit 19 . Furthermore, the data receiving unit 15 outputs to the transmission time specifying unit 18 both the timestamp added to the intermediate data and time information indicating the reception time Tr of the intermediate data.

[0051] The transmission time specifying unit 18 obtains from the data receiving unit 15 both the time stamp and the time information indicating the reception time Tr. The transmission time specifying unit 18 calculates, as the transmission time Time of the intermediate data, the difference between the reception time Tr of the intermediate data by the data receiving unit 15 and the time Ts indicated by the time stamp, as shown in the following formula (2). Time=Tr-Ts (2) The transmission time specifying unit 18 outputs time information indicating the transmission time Time to the second inference unit 19.

[0052] The second inference unit 19 acquires the intermediate data with the time stamp from the data receiving unit 15. Moreover, the second inference unit 19 acquires time information indicating the transmission time Time from the transmission time determination unit 18. The second inference unit 19 adds the transmission time Time to the time Ts indicated by the time stamp added to the intermediate data. The second inference unit 19 acquires, from among the nine pieces of second video data stored in the internal memory, second video data showing an image whose shooting time by camera 1 is earlier than that of the first video by the transmission time Time as a predicted image of the second video. For example, if the transmission time Time is equal to the time ΔT of the shooting time T1-the shooting time T0, and the time Ts indicated by the time stamp is the shooting time T9, the second inference unit 19 selects the second image data corresponding to the shooting time T1 from the nine pieces of second image data stored in the internal memory as a predicted image of the second image. 10 The second image data showing the image at (=T9+ΔT) is obtained. For example, if the transmission time Time is equal to the time 2×ΔT of the shooting time T2−the shooting time T0, and the time Ts indicated by the time stamp is the shooting time T9, the second inference unit 19 selects the second image data corresponding to the shooting time T1 from the nine pieces of second image data stored in the internal memory as a predicted image of the second image. 11 The second image data showing the image at (=T9+2×ΔT) is obtained. For example, if the transmission time Time is equal to the time 3×ΔT (the shooting time T3−the shooting time T0) and the time Ts indicated by the time stamp is the shooting time T9, the second inference unit 19 selects the second image data corresponding to the shooting time T1 from the nine pieces of second image data stored in the internal memory as a predicted image of the second image. 12 The second image data showing the image at (=T9+3×ΔT) is obtained. The second inference unit 19 outputs the acquired second video data to, for example, the display device 6 or a video processing device (not shown).

[0053] Moreover, the second inference unit 19 updates the second video data stored in the internal memory. That is, the second inference unit 19 provides the intermediate data output from the data receiving unit 15 to the second learning model 17, and determines from the second learning model 17 that the shooting time of the camera 1 is longer than the maximum transmission time Time max Then, second image data showing an image that advances by the same amount is acquired. For example, the maximum transmission time max is equal to the time 9×ΔT of the photographing time T9−the photographing time T0, and if the time Ts indicated by the time stamp is the photographing time T9, the second inference unit 19 calculates from the second learning model 17 that the photographing time T 18 The second image data showing the image at (=T9+9×ΔT) is obtained. The second inference unit 19 detects the photographing time T 18 The second image data showing the image of (=T9+9×ΔT) is stored in the internal memory. Moreover, the second inference unit 19 discards the second video data representing the video image captured at the oldest shooting time T9 from among the second video data stored in the internal memory.

[0054] In the above-described second embodiment, the video transmission system 2 is configured to include a transmission time specifying unit 18 that specifies the transmission time of the intermediate data from the transmission time of the intermediate data by the data transmitting unit 14 and the reception time of the intermediate data by the data receiving unit 15, and the second inference unit 19 acquires, from the second learning model 17, second video data indicating a predicted video of a second video in which the shooting time of the camera 1 is ahead of the first video by at least the transmission time specified by the transmission time specifying unit 18. Therefore, the video transmission system 2 can support a worker in a remote location to issue appropriate work instructions to the site even if the transmission time of the intermediate data varies.

[0055] Embodiment 3 In the third embodiment, a video transmission system 2 will be described in which a second inference unit 51 provides intermediate data received by a data receiving unit 15 to any one of second learning models 17-g (g=1, . . . , G) among a plurality of second learning models 17-1 to 17-G, and acquires second video data from any one of the second learning models 17-g. G is an integer equal to or greater than 2.

[0056] FIG. 11 is a configuration diagram showing a video transmission system 2 according to the third embodiment. 11, the same reference numerals as those in FIGS. 1 and 9 denote the same or corresponding parts, and therefore their explanation will be omitted. FIG. 12 is a hardware configuration diagram showing hardware of a video receiving device 5 included in a video transmission system 2 according to the third embodiment. 12, the same reference numerals as those in FIG. 3 and FIG. 10 denote the same or corresponding parts, and therefore the description thereof will be omitted. The hardware of the video transmitting device 3 included in the video transmission system 2 shown in Fig. 11 is similar to the hardware of the video transmitting device 3 included in the video transmission system 2 shown in Fig. 1 or the hardware of the video transmitting device 3 included in the video transmission system 2 shown in Fig. 9. Therefore, the hardware configuration diagram of the video transmitting device 3 included in the video transmission system 2 shown in Fig. 11 is that of Fig. 2.

[0057] The second inference unit 51 is realized by, for example, a second inference circuit 61 shown in FIG. The second inference unit 51 includes second learning models 17-1 to 17-G. The second inference unit 51 provides the intermediate data received by the data receiving unit 15 to any one of the second learning models 17-g (g = 1,..., G) among the second learning models 17-1 to 17-G, and acquires second video data from any one of the second learning models 17-g. The second inference unit 51 outputs the acquired second video data to, for example, the display device 6 or a video processing device (not shown). 11 shows an example in which the second inference unit 51 is applied to the video transmission system 2 shown in Fig. 1. However, this is merely an example, and the second inference unit 51 may be applied to the video transmission system 2 shown in Fig. 9.

[0058] The second learning model 17-1 is the same learning model as the second learning model 17 shown in FIG. The second learning model 17-g (g=2, . . . , G) includes an intermediate layer 32-M, N intermediate layers 33-1 to 33-N, and an output layer 34, similar to the second learning model 17-1. However, the second learning model 17-g (g = 2,...,G) differs from the second learning model 17-1 in that it is further given intermediate data received by the data receiving unit 15 and teacher data and is re-learned.

[0059] The shooting time T indicated by the training data given to the second learning model 17-g (g = 2, . . . , G) j The subject in the image and the shooting time T indicated by the training data given to the learning model 30 j The subject shown in the image of j is the same subject. j = 1,...,J, where J is an integer equal to or greater than 2. However, the video represented by the training data provided to the second learning model 17-g is, for example, a processed video of the video represented by the training data provided to the learning model 30. For example, the image represented by the teacher data provided to the second learning model 17-2 has been processed so that the image represented by the teacher data provided to the learning model 30 is an image taken during the daytime hours (hereinafter referred to as "daytime image"). For example, the image represented by the teacher data provided to the second learning model 17-3 is processed so that the image represented by the teacher data provided to the learning model 30 is an image taken on a sunny day (hereinafter referred to as "sunny day image"). For example, the image represented by the training data provided to the second learning model 17-4 has been processed so that the image represented by the training data provided to the learning model 30 is an image shot in the summer (hereinafter referred to as "summer image"). Daytime images are generally clearer than images taken during nighttime hours, sunny day images are generally clearer than images taken on cloudy or rainy days, and summer images are generally clearer than images taken in seasons other than summer. If the image captured by camera 1 is taken during daytime hours, on a sunny day, and in the summer season, the image captured by camera 1 will generally be clearer than each of the processed daytime image, the processed sunny day image, and the processed summer image.

[0060] 11, it is assumed that the data receiving unit 15 and the second inference unit 51, which are components of the video receiving device 5, are each realized by dedicated hardware as shown in Fig. 12. That is, it is assumed that the video receiving device 5 is realized by the data receiving circuit 24 and the second inference circuit 61. Each of the data receiving circuit 24 and the second inference circuit 61 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0061] The components of the video receiving device 5 are not limited to those realized by dedicated hardware, and the video receiving device 5 may be realized by software, firmware, or a combination of software and firmware. When the video receiving device 5 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures in the data receiving unit 15 and the second inference unit 51 is stored in a memory 41 shown in Fig. 5. Then, a processor 42 shown in Fig. 5 executes the program stored in the memory 41.

[0062] Fig. 12 shows an example in which each of the components of the video receiving device 5 is realized by dedicated hardware, and Fig. 5 shows an example in which the video receiving device 5 is realized by software, firmware, etc. However, this is merely one example, and some of the components in the video receiving device 5 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0063] Next, the operation of the video transmission system 2 shown in FIG. 11 will be described. 11, for convenience of explanation, an example in which G=4 will be described. However, G is not limited to 4, and may be G=2, G=3, or G≧5. 11, it is assumed that the second learning models 17-1 to 17-4 have a priority order. For ease of explanation, it is assumed here that the second learning model 17-1 has the highest priority order, the second learning model 17-2 has the second highest priority order, the second learning model 17-3 has the third highest priority order, and the second learning model 17-4 has the lowest priority order.

[0064] The video transmission device 3 shown in FIG. 11 operates in the same manner as the video transmission device 3 shown in FIG. The data receiving unit 15 of the video receiving device 5 receives the intermediate data transmitted from the data transmitting unit . The data receiving unit 15 outputs the intermediate data to the second inference unit 51.

[0065] The second inference unit 51 acquires the intermediate data from the data receiving unit 15 . If the time Tr at which the intermediate data was received by the data receiving unit 15 is within the daytime, and the first video was taken on a sunny day in the summer, the second inference unit 51 provides the intermediate data to the second learning model 17-1. Then, the second inference unit 51 obtains second video data indicating a predicted video of the second video from the second learning model 17-1. The information indicating whether the first video is a video taken on a fine day may be provided from outside the video transmission system 2, or may be added to the intermediate data. The information indicating whether the first video is a video taken in the summer season may be provided from outside the video transmission system 2, or may be added to the intermediate data. In addition, the information indicating whether the video is a video taken in the summer season may be obtained from a calendar held by the video receiving device 5.

[0066] If the reception time Tr of the intermediate data by the data receiving unit 15 is included in the nighttime hours, the second inference unit 51 provides the intermediate data to the second learning model 17-2 and obtains second video data indicating a daytime image from the second learning model 17-2 as a predicted image of the second image. If the reception time Tr of the intermediate data by the data receiving unit 15 is included in the daytime and the first video is a video taken on a cloudy day or a rainy day, the second inference unit 51 provides the intermediate data to the second learning model 17-3. Then, the second inference unit 51 obtains the second video data showing a sunny day video as a predicted video of the second video from the second learning model 17-3. If the reception time Tr of the intermediate data by the data receiving unit 15 is included in the daytime, and the first video is an image captured on a fine day in a season other than summer, the second inference unit 51 provides the intermediate data to the second learning model 17-4. Then, the second inference unit 51 obtains the second video data showing a summer video as a predicted video of the second video from the second learning model 17-4. The second inference unit 51 outputs the acquired second video data to, for example, the display device 6 or a video processing device (not shown).

[0067] In the above-described third embodiment, the video transmission system 2 shown in FIG. 11 is configured such that there are second learning models 17-1 to 17-G, and the second video data output from the second learning models 17-1 to 17-G indicate predicted videos of different second videos, and the second inference unit 51 provides the intermediate data received by the data receiving unit 15 to any one of the second learning models 17-g among the second learning models 17-1 to 17-G to obtain the second video data from any one of the second learning models 17-g. Therefore, the video transmission system 2 shown in FIG. 11 can support a worker in a remote location to give an appropriate work instruction to the site, as with the video transmission system 2 shown in FIG. 1, and can obtain clear video, etc. even if the shooting environment of the camera 1 changes.

[0068] In the video transmission system 2 shown in Fig. 11, the second inference unit 51 provides the intermediate data received by the data receiving unit 15 to one of the second learning models 17-1 to 17-G based on the reception time Tr of the intermediate data, etc. However, this is merely an example, and the second inference unit 51 may obtain a control signal from outside the video transmission system 2 indicating one of the second learning models 17-1 to 17-G to which the intermediate data is to be provided, and provide the intermediate data to the second learning model indicated by the control signal.

[0069] Embodiment 4 In the fourth embodiment, a video receiving device 5 in which an inference unit 73 includes a learning model 30 will be described.

[0070] FIG. 13 is a configuration diagram showing a video transmission system 2 including a video receiving device 5 according to the fourth embodiment. FIG. 14 is a hardware configuration diagram showing the hardware of the video transmitting device 3 included in the video transmission system 2 shown in FIG. FIG. 15 is a hardware configuration diagram showing the hardware of the video receiving device 5 included in the video transmission system 2 shown in FIG. 13, 14 and 15, the same reference numerals as those in FIGS. 1, 2 and 3 indicate the same or corresponding parts, and therefore the description thereof will be omitted.

[0071] The video transmission device 3 includes a video data acquisition unit 11 and a data transmission unit 71. The video receiving device 5 includes a data receiving unit 72 and an inference unit 73. The data transmission unit 71 is realized by, for example, a data transmission circuit 81 shown in FIG. The data transmitting unit 71 transmits the first video data acquired by the video data acquiring unit 11 to the data receiving unit 72 via the transmission path 4 .

[0072] The data receiving unit 72 is realized by, for example, a data receiving circuit 82 shown in FIG. The data receiving unit 72 receives the first video data transmitted from the data transmitting unit 71 . The data receiving unit 72 outputs the first video data to the inference unit 73 .

[0073] The inference unit 73 is realized by, for example, an inference circuit 83 shown in FIG. The inference unit 73 includes a learning model 30 shown in FIG. The inference unit 73 provides the first video data received by the data receiving unit 72 to the learning model 30, and acquires, from the learning model 30, second video data indicating a predicted video of a second video in which the shooting time of the camera 1 is ahead of the first video by at least the transmission time of the first video data from the data transmitting unit 71 to the data receiving unit 72 of the video transmitting device 3. In the video transmission system 2 shown in Fig. 13, the transmission time of the first video data is fixed, and the transmission time of the first video data is assumed to be a given value in the video transmission system 2. 13, for ease of explanation, it is assumed that the processing times of the video data acquisition unit 11, the data transmission unit 71, the data reception unit 72, and the inference unit 73 can be ignored. In this case, the inference unit 73 provides the first video data received by the data reception unit 72 to the learning model 30, and acquires, from the learning model 30, second video data indicating a predicted video of a second video whose shooting time by the camera 1 is ahead of the first video by the transmission time of the first video data. When the respective processing times cannot be ignored, the inference unit 73 provides the first video data received by the data receiving unit 72 to the learning model 30, and obtains from the learning model 30 second video data indicating a predicted video of a second video in which the shooting time of the camera 1 is ahead of the first video by the sum of the respective processing times and the transmission time of the first video data. The inference unit 73 outputs the second video data to, for example, the display device 6 or a video processing device (not shown). 13, the inference unit 73 includes the learning model 30. However, this is merely an example, and the learning model 30 may be provided outside the inference unit 73.

[0074] In the video transmission system 2 shown in Fig. 13, the data transmitting unit 71, the data receiving unit 72, and the inference unit 73 are applied to the video transmission system 2 shown in Fig. 1. However, this is merely an example, and the data transmitting unit 71, the data receiving unit 72, and the inference unit 73 may be applied to the video transmission system 2 shown in Fig. 9 or the video transmission system 2 shown in Fig. 11.

[0075] 13, it is assumed that the video data acquisition unit 11 and the data transmission unit 71, which are components of the video transmission device 3, are each realized by dedicated hardware as shown in Fig. 14. That is, it is assumed that the video transmission device 3 is realized by a video data acquisition circuit 21 and a data transmission circuit 81. 13, it is assumed that the data receiving unit 72 and the inference unit 73, which are components of the video receiving device 5, are each realized by dedicated hardware as shown in Fig. 15. That is, it is assumed that the video receiving device 5 is realized by a data receiving circuit 82 and an inference circuit 83. Each of the image data acquisition circuit 21, the data transmission circuit 81, the data receiving circuit 82 and the inference circuit 83 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0076] The components of the video transmission device 3 are not limited to those realized by dedicated hardware, and the video transmission device 3 may be realized by software, firmware, or a combination of software and firmware. Furthermore, the components of the video receiving device 5 are not limited to those realized by dedicated hardware, and the video receiving device 5 may be realized by software, firmware, or a combination of software and firmware. When the video transmission device 3 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures in the video data acquisition unit 11 and the data transmission unit 71 is stored in the memory 41 shown in Fig. 5. Then, the processor 42 shown in Fig. 5 executes the program stored in the memory 41. When the video receiving device 5 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures in the data receiving unit 72 and the inference unit 73 is stored in the memory 41 as shown in Fig. 5. Then, the processor 42 shown in Fig. 5 executes the program stored in the memory 41.

[0077] Fig. 14 shows an example in which each of the components of the video transmission device 3 is realized by dedicated hardware, and Fig. 5 shows an example in which the video transmission device 3 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the video transmission device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc. 15 shows an example in which each of the components of the video receiving device 5 is realized by dedicated hardware, while Fig. 5 shows an example in which the video receiving device 5 is realized by software, firmware, etc. However, this is merely one example, and some of the components in the video receiving device 5 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0078] Next, the operation of the video transmission system 2 shown in FIG. 13 will be described. The camera 1 outputs first video data representing a first video to a video data acquisition unit 11 of the video transmission system 2. The video data acquisition unit 11 acquires the first video data output from the camera 1 . The video data acquisition unit 11 outputs the first video data to the data transmission unit 71. The data transmission unit 71 acquires the first video data from the video data acquisition unit 11 . The data transmitting unit 71 transmits the first video data to the data receiving unit 72 via the transmission path 4 .

[0079] The data receiving unit 72 receives the first video data transmitted from the data transmitting unit 71 . The data receiving unit 72 outputs the first video data to the inference unit 73 .

[0080] The inference unit 73 acquires the first video data from the data receiving unit 72 . The inference unit 73 provides the first video data to the learning model 30 and obtains from the learning model 30 second video data indicating a predicted video of a second video in which the shooting time of the camera 1 is ahead of the first video by the transmission time of the first video data. That is, the inference unit 73 provides the first image data to the input layer 31 and obtains the second image data from the output layer . It is assumed that the transmission time of the first video data in the transmission path 4 is, for example, equal to the time 3×ΔT (shooting time T3−shooting time T0). In this case, for example, when the first video data showing the first video at the shooting time T0 is provided to the input layer 31 of the learning model 30, the second video data showing the video at the shooting time T3 (=T0+3×ΔT) is output from the output layer 34 of the learning model 30 as a predicted video of the second video. For example, when first image data showing a first image captured at a shooting time T2 is provided to the input layer 31 of the learning model 30, second image data showing an image captured at a shooting time T5 (= T2 + 3 × ΔT) is output from the output layer 34 of the learning model 30 as a predicted image of the second image. The inference unit 73 outputs the second video data to, for example, the display device 6 or a video processing device (not shown).

[0081] In the above-described fourth embodiment, the video receiving device 5 is configured to include a data receiving unit 72 that receives first video data indicating a first video transmitted from the video transmitting device 3, and an inference unit 73 that provides the first video data received by the data receiving unit 72 to the learning model 30 and acquires, from the learning model 30, second video data indicating a predicted video of a second video whose shooting time of the camera 1 is ahead of the first video by at least the transmission time of the first video data. Therefore, the video receiving device 5 can support a worker or the like in a remote location to give appropriate work instructions to the site.

[0082] In the first to fourth embodiments, the video transmitting device 3 transmits the intermediate data or the first video data to the video receiving device 5. In addition to the video transmitting device 3 transmitting the intermediate data or the first video data to the video receiving device 5, the video receiving device 5 may transmit an operation signal for a machine 93 present on-site to the video transmitting device 3 as shown in Fig. 16 . FIG. 16 is a configuration diagram showing another video transmission system 2 according to the first to fourth embodiments. The operation signal transmission unit 91 acquires an operation signal from a remote control 90 for a worker or the like in a remote location to remotely operate the machine 93. If the machine 93 is, for example, a camera, the operation signal may be, for example, a signal including a command to change the orientation of the camera. If the machine 93 is, for example, a robot, the operation signal may be, for example, a signal including a command to move the hand of the robot. The operation signal transmitting unit 91 transmits the acquired operation signal to the operation signal receiving unit 92 via the transmission path 4 . The operation signal receiving unit 92 receives the operation signal transmitted from the operation signal transmitting unit 91 and outputs the operation signal to the machine 93 . The machine 93 is a machine present in the field. The machine 93 may be, for example, a robot, a car, or a camera. The machine 93 operates in accordance with the operation signal output from the operation signal receiving unit 92 . In the video transmission system 2 shown in Fig. 16, an operation signal transmitter 91 and an operation signal receiver 92 are applied to the video transmission system 2 shown in Fig. 9. However, this is merely an example, and the operation signal transmitter 91 and the operation signal receiver 92 may be applied to the video transmission system 2 shown in Fig. 1, Fig. 11, or Fig. 13.

[0083] In addition, the present disclosure allows free combination of the respective embodiments, modification of any of the components of each embodiment, or omission of any of the components of each embodiment. [Industrial Applicability]

[0084] The present disclosure is suitable for a video transmission system, a video transmission method, and a video receiving device. [Explanation of symbols]

[0085] REFERENCE SIGNS LIST 1 camera, 2 video transmission system, 3 video transmission device, 4 transmission path, 5 video reception device, 6 display device, 11 video data acquisition unit, 12 first inference unit, 13 first learning model, 14 data transmission unit, 15 data reception unit, 16 second inference unit, 17 second learning model, 17-1 to 17-G second learning model, 18 transmission time determination unit, 19 second inference unit, 21 video data acquisition circuit, 22 first inference circuit, 23 data transmission circuit, 24 data reception circuit, 25 second inference circuit, 26 transmission time determination circuit, 27 second inference circuit, 30 learning model, 31 input layer, 32-1 to 32-M intermediate layer, 33-1 to 33-N intermediate layer, 34 output layer, 41 memory, 42 processor, 51 second inference unit, 61 second inference circuit, 71 data transmission unit, 72 Data receiving unit, 73 inference unit, 81 data transmission circuit, 82 data receiving circuit, 83 inference circuit, 90 remote control, 91 operation signal transmission unit, 92 operation signal receiving unit, 93 machine.

Claims

1. a video data acquisition unit that acquires first video data representing a first video captured by the camera; a first inference unit that provides the first video data acquired by the video data acquisition unit to a first learning model and acquires intermediate data, which is data different from the first video data, from the first learning model; a data transmission unit that transmits the intermediate data acquired by the first inference unit; a data receiving unit that receives the intermediate data transmitted from the data transmitting unit; a transmission time determination unit that determines a transmission time of the intermediate data from a transmission time of the intermediate data by the data transmission unit and a reception time of the intermediate data by the data reception unit; a second inference unit that provides the intermediate data received by the data receiving unit to a second learning model and acquires, from the second learning model, second video data indicating a predicted video of a second video whose shooting time by the camera is earlier than that of the first video by at least a transmission time of the intermediate data; A video transmission system comprising:

2. There are a plurality of the second learning models; The second image data output from the plurality of second learning models indicate predicted images of second images different from each other, The second inference unit includes: The video transmission system described in claim 1, characterized in that the intermediate data received by the data receiving unit is provided to any one of the multiple second learning models, and second video data is obtained from any one of the second learning models.

3. The images represented by the training data given to the second learning model include at least one of images taken during daytime hours, images taken on a sunny day, and images taken in the summer season.

3. The video transmission system according to claim 1 or 2.

4. The image represented by the training data given to the second learning model is processed to be clearer.

3. The video transmission system according to claim 1 or 2.

5. a video data acquisition unit that acquires first video data representing a first video captured by the camera; an inference unit that provides the first video data acquired by the video data acquisition unit to a first learning model, acquires intermediate data from the first learning model, which is data different from the first video data, provides the acquired intermediate data to a second learning model, identifies a transmission time of the intermediate data from a transmission time that is a time when the intermediate data is transmitted from the first learning model to the second learning model and a reception time that is a time when the intermediate data is received by the second learning model, and acquires from the second learning model second video data indicating a predicted video of a second video in which the shooting time of the camera is earlier than that of the first video by at least the transmission time of the intermediate data; A video transmission system comprising:

6. The video data acquisition unit acquires first video data representing a first video captured by the camera; a first inference unit provides the first video data acquired by the video data acquisition unit to a first learning model, and acquires intermediate data, which is data different from the first video data, from the first learning model; a data transmission unit that transmits the intermediate data acquired by the first inference unit; a data receiving unit receives the intermediate data transmitted from the data transmitting unit; a transmission time determination unit determines a transmission time of the intermediate data from a transmission time of the intermediate data by the data transmission unit and a reception time of the intermediate data by the data reception unit; A second inference unit provides the intermediate data received by the data receiving unit to a second learning model, and acquires, from the second learning model, second video data indicating a predicted video of a second video whose shooting time by the camera is earlier than that of the first video by at least the transmission time of the intermediate data. Video transmission method.

7. a data receiving unit that receives first video data indicating a first video captured by a camera and transmitted from the video transmitting device; a transmission time determination unit that determines a transmission time of the first video data based on a transmission time of the first video data transmitted from the video transmitting device and a reception time of the first video data by the data receiving unit; an inference unit that provides the first video data received by the data receiving unit to a learning model and acquires, from the learning model, second video data indicating a predicted video of a second video whose shooting time by the camera is earlier than that of the first video by at least a transmission time of the first video data; A video receiving device comprising:

Images