Video transmitting device, video transmission system, video transmitting method, and computer program

Abstract

To provide a video transmitting device capable of suppressing the distortion of a video captured by a camera mounted on a moving body and wirelessly transmitting the video to an external device.SOLUTION: A video transmitting device transmits a video in real time to a video receiving device via a transmission path with limited communication bandwidth. The video transmitting device includes a video acquisition unit that acquires a main direction video, which is a video in a main direction of a moving body, a video change estimation unit that estimates changes in the main direction video, a video compression unit that generates a compressed video by compressing the main direction video, a data amount control unit that controls the data amount of the compressed video based on an estimation result by the video change estimation unit, and a video transmitting unit that transmits the compressed video to the video receiving device.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a video transmitting device, a video transmission system, a video transmitting method, and a computer program. [Background technology]

[0002] In order to remotely operate heavy machinery such as hydraulic excavators, images captured by a camera mounted on the upper rotating body of the heavy machinery are transmitted to a decoder by wireless communication using a mobile phone network or the like, and are displayed on a monitor. Here, the upper rotating body is a part that is placed on a lower traveling body and on which attachment parts such as an arm and a packet are mounted. An operator remotely operates the heavy machinery while watching the image displayed on the monitor. When the upper rotating body of the heavy machinery is rotated left and right by remote operation, the image changes greatly. This increases the amount of image data transmitted by wireless communication, and an image with a data amount exceeding the available bandwidth of wireless communication is sent to the transmission path. It is known that this causes image delays (for example, see Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Akira Nozue, Fujita Corporation, and 2 others, “3. Report on ultra-long distance remote control demonstration experiment using mobile phone communication”, [online], Japan Construction Machinery Construction Association, [Retrieved June 6, 2023], Internet<URL:https: / / jcmanet.or.jp / bunken / symposium / 2012 / 2012r03.pdf> Summary of the Invention [Problem to be solved by the invention]

[0004] In a moving body such as heavy machinery with a camera attached to a rotating part, or a moving body such as a mobile robot without a rotating part, in order to perform remote monitoring or remote operation, it is necessary to wirelessly transmit images captured by a camera mounted on the moving body. For this reason, when the moving body turns, delays in the image occur, and image distortion such as missing images or block noise occurs. Such image distortion can also occur when transmitting images captured by a camera mounted on a moving body other than heavy machinery or a mobile robot wirelessly.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide a video transmission device, a video transmission system, a video transmission method, and a computer program that can suppress distortion of images captured by a camera mounted on a moving object and transmit the images wirelessly to an external device. [Means for solving the problem]

[0006] A video transmission device according to one embodiment of the present disclosure is a video transmission device that transmits video in real time to a video receiving device via a transmission path with limited communication bandwidth, and includes a video acquisition unit that acquires a main direction video, which is a video of the main direction of a moving body, a video change estimation unit that estimates changes in the main direction video, a video compression unit that generates compressed video by compressing the main direction video, a data amount control unit that controls the data amount of the compressed video based on the estimation result by the video change estimation unit, and a video transmission unit that transmits the compressed video to the video receiving device.

[0007] The present invention can be realized not only as a video transmission device having such a characteristic processing unit, but also as a video transmission method having such characteristic processing steps, or as a computer program for causing a computer to execute such steps, or as a semiconductor integrated circuit that realizes part or all of the video transmission device, or as a video transmission system including the video transmission device. Effect of the Invention

[0008] According to the present disclosure, it is possible to suppress distortion of images captured by a camera mounted on a moving object and transmit the images wirelessly to an external device. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a video transmission system according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a configuration of a video transmission device according to an embodiment of the present disclosure. [Diagram 3] FIG. 3 is a diagram showing an example of a change over time in the amount of image change Δv. [Figure 4] FIG. 4 is a diagram showing an example of correspondence relationship information between the image change amount Δv and control information of the data amount of compressed image by the data amount control unit. [Diagram 5] FIG. 5 is a diagram showing an example of movement of a moving object. [Figure 6] FIG. 6 is a diagram showing an example of a change in angular velocity when the moving body makes the turn shown in FIG. [Figure 7] FIG. 7 is a diagram for explaining an example of control of the data amount of each image constituting a compressed video. [Figure 8] FIG. 8 is a block diagram illustrating an example of a configuration of a video receiving device according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a flowchart illustrating an example of a video compression process performed by the video transmission device according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a flowchart showing an example of the details of the control information determination process (step S3 in FIG. 9). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] [Overview of the embodiment of the present disclosure] First, an overview of the embodiments of the present disclosure will be listed and described. (1) A video transmission device according to one embodiment of the present disclosure is a video transmission device that transmits video in real time to a video receiving device via a transmission path having a limited communication bandwidth, and includes a video acquisition unit that acquires a main direction video, which is a video of the main direction of a moving body, a video change estimation unit that estimates a change in the main direction video, a video compression unit that generates compressed video by compressing the main direction video, a data amount control unit that controls the data amount of the compressed video based on the estimation result by the video change estimation unit, and a video transmission unit that transmits the compressed video to the video receiving device.

[0011] According to this configuration, for example, when it is predicted that the change in the main direction video is large, the data amount of the compressed video can be controlled by increasing the compression rate of the video. This makes it possible to transmit the compressed video to the video receiving device so as not to exceed the available bandwidth of the transmission path. Therefore, it is possible to suppress image distortion when the video receiving device expands and displays the compressed video. Note that the estimation of the change in the main direction video includes not only the estimation of the current change in the main direction video, but also the prediction of the future change in the main direction video. Furthermore, the change in the main direction video refers to the change in the main direction video accompanying a change in the traveling direction, such as the straight or rotating direction, of a moving body such as heavy machinery with a rotating part, or a moving body such as a mobile robot without a rotating part.

[0012] (2) In the above (1), the data amount control unit may control the data amount of the compressed video so that a transmission delay time of the compressed video by the video transmission unit is equal to or less than a predetermined time threshold.

[0013] According to this configuration, it is possible to prevent delays in the display of video when the video receiving device expands and displays compressed video.

[0014] (3) In the above (1) or (2), the video transmission device may further include an operation information acquisition unit that acquires operation information regarding the operation of the moving body, and the video change estimation unit may estimate a change in the main direction video prior to the change in the main direction video based on the operation information.

[0015] With this configuration, it is possible to predict the change in the main direction video before the change begins, and therefore it is possible to control the amount of compressed video data from the start of the change in the main direction video, thereby increasing the possibility that the compressed video can be transmitted to the video receiving device without exceeding the available bandwidth of the transmission path.

[0016] (4) In any of (1) to (3) above, the data amount control unit may execute at least one of the following based on the prediction result by the image change prediction unit: control of the compression rate of the main direction video; instructing to thin out images constituting the main direction video; instructing to copy images constituting the main direction video; control of the resolution of the main direction video; instructing to transmit only I-frame images of the compressed video; and transmitting a refresh signal to the video compression unit.

[0017] According to this configuration, the amount of compressed video data can be reduced, and the compressed video can be transmitted to the video receiving device without exceeding the available bandwidth of the transmission path.

[0018] (5) In any of (1) to (4) above, the video transmission device may further include an attention area extraction unit that extracts an attention area from the main direction image, and the data amount control unit may control the compression rate of at least one of the attention area and a non-attention area excluding the attention area based on the estimation result by the image change estimation unit.

[0019] According to this configuration, the data amount of the compressed video can be controlled by controlling the compression rate of the region of interest or the region of non-interest.

[0020] (6) In any of (1) to (5) above, the video transmission device may further include a measurement information acquisition unit that acquires measurement information regarding the movement of the moving body, and the video change estimation unit may estimate a change in the main direction video based on the measurement information.

[0021] With this configuration, the amount of compressed video data can be controlled by increasing the compression rate of the main direction video as the movement of the moving object increases. Also, changes in the main direction video can be predicted from the movement of the moving object.

[0022] (7) In any one of (1) to (6) above, the image change estimation unit may estimate the change in the main direction image based on a motion vector between images that constitute the main direction image.

[0023] With this configuration, the amount of compressed video data can be controlled by increasing the compression rate of the main direction video as the image motion increases. Also, changes in the main direction video can be predicted from the image motion.

[0024] (8) A video transmission system according to another embodiment of the present disclosure includes a video transmitting device that transmits video in real time via a transmission path having a limited communication bandwidth, and a video receiving device that receives the video from the video transmitting device, wherein the video transmitting device includes a video acquisition unit that acquires a main direction video, which is a video of a main direction of a moving body, a video change estimation unit that estimates a change in the main direction video, a video compression unit that generates compressed video by compressing the main direction video, a data amount control unit that controls the data amount of the compressed video based on the estimation result by the video change estimation unit, and a video transmitting unit that transmits the compressed video to the video receiving device.

[0025] According to this configuration, for example, when it is expected that the change in the main direction image will be large, the data amount of the compressed image can be controlled by increasing the compression rate of the image. This makes it possible to transmit the compressed image to the image receiving device so as not to exceed the available bandwidth of the transmission path. Therefore, it is possible to prevent the image from being distorted when the compressed image is expanded and displayed by the image receiving device.

[0026] (9) A video transmission method according to another embodiment of the present disclosure is a video transmission method by a video transmitting device that transmits video in real time to a video receiving device via a transmission path having a limited communication bandwidth, and includes the steps of acquiring a main direction video, which is a video of a main direction of a moving body, estimating a change in the main direction video, generating compressed video by compressing the main direction video, controlling the data amount of the compressed video based on the estimated result of the change in the main direction video, and transmitting the compressed video to the video receiving device.

[0027] This configuration includes the characteristic processing steps of the above-mentioned video transmission device, and therefore can achieve the same functions and effects as the above-mentioned video transmission device.

[0028] (10) A computer program according to another embodiment of the present disclosure is a computer program for causing a computer to function as a video transmitting device that transmits video in real time to a video receiving device via a transmission path having a limited communication bandwidth, and causes the computer to function as a video acquisition unit that acquires a main direction video, which is a video of the main direction of a moving object, a video change estimation unit that estimates changes in the main direction video, a video compression unit that generates compressed video by compressing the main direction video, a data amount control unit that controls the data amount of the compressed video based on a result of estimation by the video change estimation unit, and a video transmission unit that transmits the compressed video to the video receiving device.

[0029] According to this configuration, the computer can function as the above-mentioned video transmission device, and therefore, the same effects and advantages as those of the above-mentioned video transmission device can be achieved.

[0030] [Details of the embodiment of the present disclosure] Hereinafter, the embodiments of the present disclosure will be described with reference to the drawings. Note that each of the embodiments described below shows a specific example of the present disclosure. The numerical values, shapes, materials, components, the arrangement and connection forms of the components, steps, and the order of steps shown in the following embodiments are examples and do not limit the present disclosure. In addition, among the components in the following embodiments, components that are not described in the independent claims are components that can be added arbitrarily. In addition, each figure is a schematic diagram and is not necessarily illustrated strictly.

[0031] In addition, the same components are denoted by the same reference numerals, and since their functions and names are also the same, their explanations will be omitted as appropriate.

[0032] [Overall configuration of video transmission system 1] FIG. 1 is a diagram illustrating an example of a configuration of a video transmission system according to an embodiment of the present disclosure. With reference to FIG. 1, a video transmission system 1 includes a video transmitting device 20 and a video receiving device .

[0033] The video transmission device 20 generates compressed video by compressing in real time the video captured by the camera 21 mounted on the moving object 2. The moving object 2 is, for example, an autonomous vehicle such as an autonomous driving car. However, the moving object 2 is not limited to an autonomous vehicle, and may be an autonomously traveling robot, or a remotely controlled vehicle or robot. The moving object 2 may also be heavy machinery such as a hydraulic excavator equipped with a rotating part. In this case, the camera 21 may be attached to the rotating part of the heavy machinery.

[0034] The camera 21 is installed so that the optical axis of the camera 21 is parallel to the main direction of the moving body 2, and captures the main direction of the moving body 2. The main direction of the moving body 2 is, for example, the traveling direction of the moving body 2. However, the main direction of the moving body 2 is not limited to this. For example, a direction perpendicular to the traveling direction of the moving body 2 may be the main direction of the moving body 2.

[0035] The video transmitting device 20 is connected to the video receiving device 40 via a transmission path 30. The transmission path 30 has a limited communication band. The transmission path 30 may be a wired transmission path or a wireless transmission path. The video transmitting device 20 transmits compressed video to the video receiving device 40 via the transmission path 30 in real time.

[0036] The video transmission device 20 is connected to an automatic driving ECU 22 and an angular velocity sensor (gyro sensor) 23 mounted on the moving object 2. The automatic driving ECU 22 estimates the surrounding conditions of the moving object 2 based on the measurement results of various sensors mounted on the moving object 2, such as the camera 21 and LiDAR (Light Detection And Ranging). For example, the automatic driving ECU 22 estimates the surrounding conditions, such as the shape of the road, the position information of other moving objects, and the position information of pedestrians, based on the measurement results of a three-dimensional space measured by the LiDAR. In addition, the automatic driving ECU 22 estimates the surrounding conditions, such as traffic light light color information and road sign information, from the image of the main direction of the moving object 2 captured by the camera 21 (hereinafter referred to as the "main direction image"). The automatic driving ECU 22 generates operation information, which is instruction information for the moving object 2, such as the traveling direction and traveling speed of the moving object 2, based on the estimated surrounding conditions. The automatic driving ECU 22 controls the operation systems or drive systems of the moving body 2, such as the power steering, accelerator, and brakes, based on the generated operation information, thereby causing the moving body 2 to automatically drive.

[0037] The video transmission device 20 acquires operation information related to the operation of the moving object 2 from the automatic driving ECU 22, and controls the data amount of the compressed video based on the operation information. The method of controlling the data amount will be described later.

[0038] Furthermore, the video transmission device 20 acquires measurement information of the angular velocity of the moving object 2 from the angular velocity sensor 23, and controls the data amount of the compressed video based on the angular velocity measurement information. The method of controlling the data amount will be described later.

[0039] The video receiving device 40 receives compressed video from the video transmitting device 20 via the transmission path 30. The video receiving device 40 restores the video by expanding the received compressed video, and displays the restored video on a display 40A connected to the video receiving device 40.

[0040] [Configuration of video transmission device 20] FIG. 2 is a block diagram showing an example of a configuration of the video transmission device 20 according to an embodiment of the present disclosure. The video transmission device 20 includes a video acquisition unit 201, a region of interest extraction unit 202, an operation information acquisition unit 203, a measurement information acquisition unit 204, a video change estimation unit 205, a video compression unit 206, a data amount control unit 207, a region of interest information output unit 208, a multiplexing unit 209, and a video transmission unit 210.

[0041] (Video acquisition unit 201) The video acquisition unit 201 acquires, from the camera 21, main direction video of the moving object 2 captured by the camera 21. The main direction video is a moving image made up of time-series images.

[0042] (Attention area extraction unit 202) The attention area extraction unit 202 extracts an attention area from the main direction image acquired by the image acquisition unit 201. The attention area is, for example, an area including a predetermined object. The attention area includes a fine attention area and a coarse attention area. For example, the fine attention area is an area including an object having a higher importance than the coarse attention area. As an example, the fine attention area is an area including a distant vehicle or pedestrian, and the coarse attention area is an area including a nearby large vehicle or truck. However, the number of types of attention areas is not limited to two, and may be one, or may be three or more. For example, a nearby large vehicle or truck can be recognized even if compressed at a high compression rate, so it is classified as a coarse attention area. On the other hand, a distant vehicle or pedestrian is classified as a fine attention area because it is difficult to recognize unless compressed at a low compression rate, or because it is highly important.

[0043] Each image constituting the main direction video can be divided into a plurality of block images (e.g., 8×8 block images), for example. The attention area extraction unit 202 inputs each image into a learning model for determining an attention area. The learning model determines whether each block image constituting the input image corresponds to a sub-attention area and a coarse attention area, and outputs the block number of the block image corresponding to the sub-attention area and the block number of the block image corresponding to the coarse attention area. The attention area extraction unit 202 acquires the block numbers of the sub-attention area and the coarse attention area output by the learning model as the extraction results of the attention area.

[0044] Here, the learning model is, for example, a convolution neural network (CNN), a recurrent neural network (RNN), an autoencoder, etc. Using block images including various objects contained in the region of interest as training data, learning of the objects progresses by a machine learning method such as deep learning, and the parameters of the learning model are determined.

[0045] (Operation information acquisition unit 203) The operation information acquisition unit 203 acquires operation information related to the operation of the moving object 2. For example, the operation information acquisition unit 203 acquires operation information from the automatic driving ECU 22. As described above, the operation information indicates the traveling direction and traveling speed of the moving object 2. The operation information may include one-off instruction information for the moving object 2, such as turning 30° to the right. The operation information may also include instruction information related to a series of traveling of the moving object 2, such as turning 30° to the right, decelerating to a speed of 30 km / h, and then continuing to travel for 15 seconds.

[0046] When the moving body 2 does not perform autonomous driving or when a dangerous situation (e.g., an accident or a breakdown) occurs in the moving body 2 and it is difficult to continue autonomous driving, the moving body 2 is remotely controlled. In such a case, the moving body 2 receives operation information from an operation device used by a remote operator, and travels based on the received operation information. Also, the operation information acquisition unit 203 acquires the operation information from the operation device.

[0047] (Measurement information acquisition unit 204) The measurement information acquiring unit 204 acquires measurement information of the angular velocity of the moving body 2 from the angular velocity sensor 23. For example, the measurement information acquiring unit 204 acquires an angular velocity about a rotation axis in the vertical direction from the angular velocity sensor 23. This allows obtaining the angular velocity when the moving body 2 rotates rightward or leftward.

[0048] (Image change prediction unit 205) Image change estimation section 205 estimates a change in the main direction video of moving object 2 based on at least one of the operation information acquired by operation information acquisition section 203 and the angular velocity measurement information acquired by measurement information acquisition section 204. Note that estimation of a change in the main direction video includes estimation of the current change status of the main direction video as well as prediction of the future change status of the main direction video. Furthermore, a change in the main direction video refers to a change in the main direction video accompanying a change in the traveling direction, such as the straight forward direction or rotational direction, of moving object 2.

[0049] Specifically, based on the operation information, the image change estimation unit 205 predicts the future movement of the moving object 2. The movement of the moving object 2 includes, for example, the moving direction, speed, turning direction, angular velocity during turning, and the like of the moving object 2.

[0050] For example, when the operation information includes an operation instruction to rotate 90° to the right, the automatic driving ECU 22 transmits an instruction to the operation system or drive system of the moving body 2 to control the moving direction of the moving body 2. At this time, there is a time lag between the transmission of the operation instruction and the start of the change in the moving direction of the moving body 2. The image change estimation unit 205 predicts the future movement of the moving body 2 taking the time lag into consideration. For example, the image change estimation unit 205 predicts a series of movements of the moving body 2, in which the moving body 2 starts turning to the right 500 msec after the operation instruction is transmitted, turns so that the average angular velocity for 3 seconds is 30° / sec, and then returns the steering wheel to the front position, and the angular velocity becomes 0° / sec. The image change estimation unit 205 may predict the movement of the moving body 2 from the operation information by referring to table information or a formula that defines in advance the correspondence between the operation information and the movement of the moving body 2.

[0051] Furthermore, the image change estimation unit 205 may predict the movement of the moving object 2 by inputting the operation information into a learning model and acquiring the movement of the moving object 2 from the learning model. Here, the learning model is, for example, CNN, RNN, AutoEncoder, etc. For example, using the operation information and the movement teacher data of the moving object 2 as training data, learning of the movement of the moving object 2 in response to the operation information is progressed by a machine learning method such as deep learning, and the parameters of the learning model are determined.

[0052] Based on the predicted movement of the moving object 2, the image change estimation unit 205 predicts an image change amount Δv indicating the degree of change in the main direction image prior to the change in the main direction image. For example, the image change amount Δv is expressed on a 10-point scale, with the larger the value, the greater the degree of change in the main direction image per unit time. The image change estimation unit 205 determines the image change amount Δv from the predicted movement of the moving object 2 by referring to table information or a formula indicating the relationship between the movement of the moving object 2 and the image change amount Δv.

[0053] FIG. 3 is a diagram showing an example of a temporal change in the image change amount Δv. FIG. 3 shows an image change amount Δv when the moving object 2 performs a predicted movement of the moving object 2 in response to operation information including an operation instruction to rotate 90° to the right, for example. The time t is set to 0 sec when the transmission instruction is transmitted. 500 msec after the operation instruction is transmitted, the moving object 2 starts turning. Therefore, the image change amount Δv at 0 sec and 0.5 sec is 0. Thereafter, the angular velocity gradually increases, reaches a peak, and then gradually decreases, until the angular velocity becomes 0° / sec. Therefore, the image change amount Δv increases from 1.0 sec to 2.5 sec, and decreases from 2.5 sec to 4.0 sec. Also, the image change amount Δv at 4.5 sec and 5.0 sec is 0.

[0054] The image change estimation unit 205 may input the movement of the moving object 2 into a learning model and obtain the image change amount Δv from the learning model to predict the image change amount Δv. Here, the learning model is, for example, CNN, RNN, AutoEncoder, etc. For example, using the movement of the moving object 2 and the image change amount Δv as teacher data, learning of the image change amount Δv for the movement of the moving object 2 is progressed by a machine learning method such as deep learning, and the parameters of the learning model are determined.

[0055] The image change estimation unit 205 may estimate the image change amount Δv based on the measurement information of the angular velocity acquired by the measurement information acquisition unit 204. For example, the image change estimation unit 205 estimates the movement of the moving object 2 by replacing the angular velocity of the moving object 2 with the angular velocity indicated by the measurement information acquired by the measurement information acquisition unit 204, among the movements of the moving object 2 predicted in advance based on the operation information. The image change estimation unit 205 estimates the image change amount Δv indicating the degree of change in the main direction image based on the estimated movement of the moving object 2. That is, the image change estimation unit 205 estimates the image change amount Δv from the estimated movement of the moving object 2 by referring to the table information or formula indicating the relationship between the movement of the moving object 2 and the image change amount Δv described above. The image change estimation unit 205 may also estimate the image change amount Δv by inputting the estimated movement of the moving object 2 into the above-mentioned learning model and acquiring the image change amount Δv from the learning model.

[0056] The image change estimation unit 205 may estimate the image change amount Δv from at least one of the operation information and the measurement information by referring to table information or a formula indicating the relationship between at least one of the operation information and the measurement information and the image change amount Δv. The image change estimation unit 205 may also estimate the image change amount Δv by inputting at least one of the operation information and the measurement information into a learning model and acquiring the image change amount Δv from the learning model. Here, the learning model is, for example, CNN, RNN, AutoEncoder, etc. For example, by using at least one of the operation information and the measurement information and the image change amount Δv as teacher data, learning of the image change amount Δv for at least one of the operation information and the measurement information is progressed by a machine learning method such as deep learning, and the parameters of the learning model are determined. The image change estimation unit 205 outputs the predicted or estimated image change amount Δv to the data amount control unit 207 .

[0057] (Video Compression Unit 206) The video compression unit 206 compresses the main direction video acquired by the video acquisition unit 201 to generate compressed video. The compression method is, for example, H.264 / MPEG-4 AVC, H.265 / MPEG-H HEVC, or the like.

[0058] During compression, the video compression unit 206 compresses the sub-attention area, the coarse attention area, and areas other than the sub-attention area and the coarse attention area (hereinafter referred to as "non-attention areas") at different compression rates. In other words, the video compression unit 206 compresses the main direction video so that the compression rates decrease in the order of the non-attention area, the coarse attention area, and the sub-attention area. As a result, the data amounts of compressed areas of the same size increase in the order of the non-attention area, the coarse attention area, and the sub-attention area. It is assumed that the compression rate of each area is preset.

[0059] (Data amount control unit 207) The data amount control unit 207 controls the data amount of the compressed video generated by the video compression unit 206 based on the video change amount Δv acquired from the video change estimation unit 205. For example, the data amount control unit 207 controls the compression rate of the main direction video based on the video change amount Δv. That is, the data amount control unit 207 controls the compression rate of the main direction video by changing the compression rate of any of the sub-attention area, the coarse attention area, and the non-attention area. For example, the data amount control unit 207 increases the compression rate of the sub-attention area when the video change amount Δv is a predetermined value.

[0060] Furthermore, the data amount control unit 207 may instruct the video compression unit 206 to spatially thin out images constituting the main direction video based on the video change amount Δv. For example, the data amount control unit 207 instructs the video compression unit 206 to thin out images when the video change amount Δv is a predetermined value.

[0061] Furthermore, the data amount control unit 207 may instruct the video compression unit 206 to copy the images constituting the main direction video based on the video change amount Δv. For example, the data amount control unit 207 instructs the video compression unit 206 to copy the images when the video change amount Δv is a predetermined value. Based on the instruction, the video compression unit 206 copies the images of the odd frames to the images of the even frames. This results in the images changing at two-frame intervals. This makes it possible to eliminate changes in the video from odd frames to even frames, and reduce the data amount of the compressed video.

[0062] Furthermore, data amount control unit 207 may control the resolution of the main direction video based on the video change amount Δv. For example, when the video change amount Δv is a predetermined value, data amount control unit 207 instructs video compression unit 206 to lower the resolution of the main direction video. As a result, video compression unit 206 reduces the data amount of the compressed video by, for example, converting 4K main direction video into 2K main direction video and then compressing it.

[0063] Furthermore, the data amount control unit 207 may instruct the video compression unit 206 to transmit only I frame images of the compressed video based on the video change amount Δv. For example, the data amount control unit 207 instructs the video compression unit 206 to transmit only I frame images when the video change amount Δv is a predetermined value.

[0064] Furthermore, the data amount control unit 207 may transmit a refresh signal to the video reception device 40 via the video transmission unit 210 based on the video change amount Δv, and may output the refresh signal to the video compression unit 206. For example, when the video change amount Δv is a predetermined value, the data amount control unit 207 transmits and outputs the refresh signal. This allows the video reception device 40 to discard images up to the next I frame image and play back videos from the next I frame. Furthermore, the video compression unit 206 can discard images up to the next I frame image and compress videos from the next I frame. The video reception device 40 can clear out unprocessed videos stored in its own buffer, and thus video playback delays are eliminated. Accordingly, a decrease in the transmission speed of the transmission path 30 can be prevented.

[0065] Furthermore, the data amount control unit 207 may instruct the attention region extraction unit 202 to change the extraction sensitivity of the sub attention region and the coarse attention region based on the image change amount Δv. For example, the data amount control unit 207 increases the sensitivity of the sub attention region when the image change amount Δv is a predetermined value. This allows a region that has been classified as a sub attention region to be classified as a coarse attention region.

[0066] FIG. 4 is a diagram showing an example of correspondence relationship information between the image change amount Δv and control information for the data amount of compressed image by the data amount control unit 207. In FIG.

[0067] When the image change amount Δv is "1", the data amount control unit 207 instructs the video compression unit 206 to increase the compression rate of the non-target region. For example, the data amount control unit 207 transmits an instruction to the video compression unit 206 to increase the value of the quantization parameter QP of the non-target region from its default value by 5. In accordance with the instruction, the video compression unit 206 increases the value of the quantization parameter QP of the non-target region by 5 from a predetermined default value, and then compresses the main direction video.

[0068] When the image change amount Δv is "2", the data amount control unit 207 instructs the video compression unit 206 to further increase the compression rate of the non-target region. For example, the data amount control unit 207 transmits an instruction to the video compression unit 206 to increase the value of the quantization parameter QP of the non-target region by 10 from the default value. In accordance with the instruction, the video compression unit 206 increases the value of the quantization parameter QP of the non-target region by 10 from the predetermined default value, and then compresses the main direction video.

[0069] When the image change amount Δv is "3", the data amount control unit 207 gives the image compression unit 206 the same instruction as when the image change amount Δv is "2". In addition, the data amount control unit 207 gives an instruction to increase the compression rate of the coarse attention region. For example, the data amount control unit 207 sends an instruction to the image compression unit 206 to increase the value of the quantization parameter QP of the coarse attention region by 5 from the default value. In accordance with the instruction, the video compression unit 206 further increases the value of the quantization parameter QP of the coarse attention region by 5 from the predetermined default value, and then compresses the main direction image.

[0070] When the image change amount Δv is "4", the data amount control unit 207 gives the image compression unit 206 the same instruction as when the image change amount Δv is "2". In addition, the data amount control unit 207 gives an instruction to further increase the compression rate of the coarse attention region. For example, the data amount control unit 207 sends an instruction to the image compression unit 206 to increase the value of the quantization parameter QP of the coarse attention region by 10 from the default value. In accordance with the instruction, the video compression unit 206 further increases the value of the quantization parameter QP of the coarse attention region by 10 from the predetermined default value, and then compresses the main direction image.

[0071] When the image change amount Δv is "5", the data amount control unit 207 issues the same instruction to the video compression unit 206 as when the image change amount Δv is "4". In addition, the data amount control unit 207 sends an instruction to the video compression unit 206 to reduce the number of frames of the main direction video by half. In accordance with the instruction, the video compression unit 206 further thins out images so that the number of frames of the main direction video is reduced to half, and then compresses the main direction video. For example, the video compression unit 206 thins out images of the 60 fps (frames per second) main direction video to generate a 30 fps video.

[0072] When the image change amount Δv is "6", the data amount control unit 207 issues the same instruction to the video compression unit 206 as when the image change amount Δv is "4". In addition, the data amount control unit 207 sends an instruction to the video compression unit 206 to reduce the number of frames of the main direction video to 1 / 4. In accordance with the instruction, the video compression unit 206 further thins out images so that the number of frames of the main direction video is reduced to 1 / 4, and then compresses the main direction video. For example, the video compression unit 206 thins out images of the 60 fps main direction video to generate a 15 fps video.

[0073] When the image change amount Δv is "7", the data amount control unit 207 issues the same instruction to the video compression unit 206 as when the image change amount Δv is "6". In addition, the data amount control unit 207 sends an instruction to the video compression unit 206 to reduce the image resolution of the main direction video to 1 / 2. In accordance with the instruction, the video compression unit 206 further reduces the image resolution of the main direction video to 1 / 2, and then compresses the main direction video. For example, the video compression unit 206 reduces the resolution of 8K main direction video to 4K video.

[0074] When the image change amount Δv is "8", the data amount control unit 207 issues the same instruction to the video compression unit 206 as when the image change amount Δv is "6". In addition, the data amount control unit 207 sends an instruction to the video compression unit 206 to reduce the image resolution of the main direction video to 1 / 4. In accordance with the instruction, the video compression unit 206 further reduces the image resolution of the main direction video to 1 / 4 and compresses the main direction video. For example, the video compression unit 206 reduces the resolution of 8K main direction video to 2K video.

[0075] When the image change amount Δv is "9", the data amount control unit 207 gives the image compression unit 206 the same instruction as when the image change amount Δv is "8". In addition, the data amount control unit 207 gives an instruction to increase the compression rate of the sub-attention region. For example, the data amount control unit 207 sends an instruction to the image compression unit 206 to increase the value of the quantization parameter QP of the sub-attention region by 5 from the default value. In accordance with the instruction, the video compression unit 206 further increases the value of the quantization parameter QP of the sub-attention region by 5 from the predetermined default value, and then compresses the main direction image.

[0076] When the image change amount Δv is "10", the data amount control unit 207 gives the image compression unit 206 the same instruction as when the image change amount Δv is "8". In addition, the data amount control unit 207 gives an instruction to further increase the compression rate of the sub-attention region. For example, the data amount control unit 207 sends an instruction to the image compression unit 206 to increase the value of the quantization parameter QP of the sub-attention region by 10 from the default value. In accordance with the instruction, the video compression unit 206 further increases the value of the quantization parameter QP of the sub-attention region by 10 from the predetermined default value, and then compresses the main direction image.

[0077] The above-mentioned control information may be determined in advance so that the transmission delay time of the compressed video by the video transmission unit 210 is equal to or less than a predetermined time threshold. This allows the data amount control unit 207 to control the data amount of the compressed video so that the transmission delay time of the compressed video is equal to or less than the predetermined time threshold.

[0078] (Attention area information output unit 208) The region of attention information output unit 208 creates region of attention information indicating the position in the image of the region of attention extracted by the region of attention extraction unit 202, and outputs the region of attention information to the multiplexing unit 209. For example, the region of attention information includes the block number of the block image corresponding to the fine region of attention and the block number of the block image corresponding to the coarse region of attention.

[0079] (Multiplex section 209) The multiplexing unit 209 generates multiplexed data by multiplexing the compressed video generated by the video compression unit 206 and the attention area information acquired from the attention area information output unit 208 , and outputs the multiplexed data to the video transmission unit 210 .

[0080] (Video transmission unit 210) The video transmitting unit 210 transmits the multiplexed data obtained from the multiplexing unit 209 to the video receiving device 40 via the transmission path 30 .

[0081] [An example of the operation of the video transmission device 20] 5 is a diagram showing an example of the movement of the moving object 2. For example, the moving object 2 turns 90° to the right from a state facing the main direction 3A to a state facing the main direction 3B. During this time, the main direction changes from a direction toward (a) to a direction toward (e).

[0082] FIG. 6 is a diagram showing an example of change in angular velocity when the moving body 2 turns as shown in FIG. 5. The horizontal axis of FIG. 6 indicates time, and the vertical axis indicates the angular velocity of the moving body 2. The angular velocity increases from 0 over time, reaches a maximum at the time when the main direction of the moving body 2 faces (c) (hereinafter referred to as "time (c)". The same applies to (a), (b), (d), and (e)), and then decreases over time to reach 0 at time (e). Such angular velocity of the moving body 2 is estimated by the image change estimation unit 205. As the angular velocity increases, the image change amount Δv also increases. Therefore, when compressed using the same compression method, the data amount of the compressed image also increases in proportion to the angular velocity. In other words, the higher the angular velocity, the higher the possibility that compressed image with a data amount exceeding the available bandwidth of the transmission path 30 (the upper limit of the communication bandwidth of the transmission path 30 available for communication) is transmitted to the transmission path 30.

[0083] Therefore, by controlling the amount of data of the compressed video in accordance with the above-mentioned image change amount Δv, the amount of data of the compressed video can be kept below a certain value, thereby preventing compressed video with a data amount exceeding the available bandwidth from being transmitted to the transmission path 30.

[0084] Fig. 7 is a diagram for explaining an example of controlling the data amount of each image constituting the compressed video. The horizontal axis of the bar graph shown in Fig. 7 indicates time, and the vertical axis indicates the data amount of each image constituting the compressed video. Each bar corresponds to the data amount of one image constituting the compressed video. Time points (a) to (e) shown in Fig. 7 correspond to time points (a) to (e) shown in Fig. 5 and Fig. 6. Fig. 7 shows the data amount of each image when it is assumed that a main direction video (i.e., a still image) composed of the same image is compressed from time points (a) to (e).

[0085] For example, suppose that at time (b), images (frames) constituting the main direction video are thinned out. As a result, the time interval between images constituting the compressed video becomes wider. Furthermore, suppose that at time (c), the compression rate of the attention area is increased. As a result, the amount of data of each image constituting the compressed video becomes smaller. Furthermore, suppose that at time (d), the compression rate of the attention area is decreased. As a result, the amount of data of each image constituting the compressed video becomes larger. Furthermore, suppose that at time (e), the images constituting the main direction video are not thinned out. As a result, the time interval between images constituting the compressed video becomes narrower.

[0086] In reality, the main direction image is not a still image but a changing image. Therefore, at time point (b), the amount of data of the compressed image increases more than that shown in FIG. 7. However, because images are thinned out, the time average of the multiple compressed images at time point (b) can be kept below the available bandwidth. Moreover, at time point (c), the amount of data of the compressed image increases even more than that shown in FIG. 7. However, because the compression rate of the region of interest is increased, the time average of the multiple compressed images at time point (c) can be kept below the available bandwidth.

[0087] [Configuration of video receiving device 40] FIG. 8 is a block diagram showing an example of a configuration of a video receiving device 40 according to an embodiment of the present disclosure. The video receiving device 40 includes a video receiving unit 401 , a video decompression unit 402 , an attention area information extraction unit 403 , a video cropping unit 404 , and a display control unit 405 .

[0088] The video receiving unit 401 receives multiplexed data from the video transmitting device 20 via the transmission path 30, and outputs the multiplexed data to a video decompressing unit 402 and an attention area information extracting unit 403. The multiplexed data is compressed video with attention area information added.

[0089] When a refresh signal is transmitted from the video transmission device 20 , the video reception unit 401 receives the refresh signal and outputs it to the video decompression unit 402 .

[0090] The attention area information extraction unit 403 extracts attention area information from the multiplexed data, and outputs it to the video decompression unit 402 and the video cropping unit 404 .

[0091] The video decompression unit 402 extracts the compressed video from the multiplexed data and decompresses it. When decompressing the compressed video, the video decompression unit 402 refers to attention area information and decompresses the compressed video using a decompression method corresponding to each of the fine attention area, the coarse attention area, and the non-attention area. The video decompression unit 402 outputs the decompressed video to the video cropping unit 404 and the display control unit 405.

[0092] When the video decompression unit 402 receives a refresh signal, it discards images up to the next I frame of the compressed video, and decompresses the compressed video from the next I frame.

[0093] The image cropping unit 404 crops out the expanded image based on the attention area information. For example, the image cropping unit 404 crops out an image included in the sub attention area from the expanded image. The image cropping unit 404 outputs the cropped image to the display control unit 405.

[0094] Display control section 405 displays on display 40A the video clipped by video clipping section 404. Display control section 405 may also display the expanded video directly on display 40A.

[0095] [About video compression processing] FIG. 9 is a flowchart showing an example of a video compression process by the video transmission device 20 according to an embodiment of the present disclosure. The video transmission device 20 acquires a main direction video of the moving object 2 from the camera 21 (step S1).

[0096] The video transmission device 20 extracts a detailed attention region and a rough attention region from the acquired main direction video (step S2). The video transmitting device 20 determines control information for controlling the amount of compressed video data (step S3). FIG. 10 is a flowchart showing an example of the details of the control information determination process (step S3 in FIG. 9). The video transmission device 20 acquires operation information of the moving object 2 (step S11).

[0097] The video transmission device 20 predicts the amount of video change Δv of the main direction video based on the acquired operation information (step S12).

[0098] Based on the predicted image change amount Δv, the image transmission device 20 refers to the relationship information shown in FIG. 4 and generates control information corresponding to the image change amount Δv (step S13).

[0099] The video transmission device 20 acquires measurement information of the angular velocity of the moving object 2 measured by the angular velocity sensor 23 (step S14).

[0100] The video transmission device 20 estimates the video change amount Δv based on the operation information of the moving object 2 and the measurement information of the angular velocity (step S15).

[0101] Based on the estimated image change amount Δv, the image transmission device 20 refers to the relationship information shown in FIG. 4 and generates control information corresponding to the image change amount Δv (step S16).

[0102] Note that the processes from step S14 to step S16 are optional processes, and therefore, these processes do not have to be executed.

[0103] 9 again, based on the determined control information, video transmission device 20 controls the compression rate of the attention area or non-attention area included in the main direction video, controls the thinning interval of the images constituting the main direction video, controls the resolution of the images, etc. Then, video transmission device 20 compresses the main direction video to generate compressed video (step S4).

[0104] The video transmitting device 20 generates attention area information indicating the position in the image of the attention area extracted in step S2 (step S5).

[0105] The video transmitting device 20 multiplexes the compressed video generated in step S4 by adding the attention area information generated in step S5 to the compressed video (step S6).

[0106] The video transmitting device 20 transmits the compressed video to which the attention area information has been added in step S6 to the video receiving device 40 via the transmission path 30 (step S7). The video transmitting device 20 repeatedly executes the video compression process shown in FIG.

[0107] As described above, when it is estimated that the change in the main direction video is large, the video transmitting device 20 can control the data amount of the compressed video by increasing the video compression rate. This makes it possible to transmit the compressed video to the video receiving device 40 so as not to exceed the available bandwidth of the transmission path 30. This makes it possible to prevent the video from being distorted when the video receiving device 40 expands and displays the compressed video.

[0108] Furthermore, the data amount control unit 207 can control the data amount of the compressed video so that the transmission delay time of the compressed video by the video transmission unit 210 is equal to or less than a predetermined time threshold value. This makes it possible to prevent delays in the display of the video when the video reception device 40 expands and displays the compressed video.

[0109] Furthermore, the image change estimation unit 205 estimates the image change amount Δv based on the operation information. Therefore, it is possible to estimate the change in the main direction video before the change in the main direction video starts. This makes it possible to control the amount of compressed video data from the start of the change in the main direction video. This further increases the possibility that the compressed video can be transmitted to the video receiving device 40 without exceeding the available bandwidth of the transmission path 30.

[0110] <Variation 1> In the above embodiment, in order to reduce the data amount of the compressed image, the data amount control unit 207 performs control to increase the compression rate of the sub-attention region or the coarse attention region. Alternatively, the data amount control unit 207 may control the data amount of the compressed image by transmitting an instruction to the attention region extraction unit 202 for controlling the detection sensitivity of the sub-attention region or the coarse attention region based on the image change amount Δv.

[0111] For example, the attention area extraction unit 202 inputs each image constituting the main direction video to a learning model and obtains a confidence level indicating the likelihood of the image being a region of interest from the learning model. The attention area extraction unit 202 classifies block images constituting each image into a fine attention area, a coarse attention area, or a non-attention area by thresholding the confidence level. For example, the attention area extraction unit 202 classifies block images with a confidence level of 80 or more into fine attention areas, block images with a confidence level of less than 80 and 60 or more into coarse attention areas, and block images with a confidence level of less than 60 into non-attention areas.

[0112] In the first modification, for example, when the image change amount Δv is a predetermined value, the data amount control unit 207 instructs the attention area extraction unit 202 to change the detection sensitivity of the attention area. In accordance with the instruction, the attention area extraction unit 202 changes the certainty threshold for classifying into a sub attention area to a higher value. As a result, a part of the block image that was classified into a sub attention area before the change of the certainty threshold is classified into a coarse attention area, and the data amount of the compressed image can be reduced.

[0113] Furthermore, the attention region extraction unit 202 may change the certainty threshold for classifying the regions into coarse attention regions to a higher value, so that some of the block images that were classified into coarse attention regions before the change in the certainty threshold are classified into non-attention regions, thereby reducing the amount of data of the compressed video.

[0114] For example, the attention region extraction unit 202 classifies block images with a certainty factor of 90 or more into sub-attention regions, block images with a certainty factor of less than 90 and 70 or more into coarse attention regions, and block images with a certainty factor of less than 70 into non-attention regions. As a result, block images with a certainty factor of less than 90 and 80 or more are reclassified from sub-attention regions to coarse attention regions. Also, block images with a certainty factor of less than 70 and 60 or more are reclassified from coarse attention regions to non-attention regions. This makes it possible to reduce the amount of data of the compressed video.

[0115] <Variation 2> The image change estimation unit 205 may calculate a motion vector indicating the amount of image movement between frames (e.g., between adjacent frames) and estimate the image change amount Δv from the magnitude of the motion vector. For example, the image change estimation unit 205 calculates the magnitude of the motion vector for each macroblock included in the image and calculates the average magnitude of the motion vector. The image change estimation unit 205 determines the image change amount Δv by thresholding the average magnitude of the motion vector. Note that the threshold is determined in advance so that the value of the image change amount Δv increases as the average magnitude of the motion vector increases.

[0116] <Variation 3> In the above embodiment, the image change estimation unit 205 predicts the image change amount Δv of the main direction image from the operation information. The image change estimation unit 205 may predict the future image change amount Δv from the change amount of the motion vector magnitude. Here, the motion vector is calculated in the same manner as in the second modification.

[0117] For example, the image change estimation unit 205 calculates the difference between the magnitude of the motion vector at the current time and the magnitude of the motion vector a predetermined time ago as the amount of change in the magnitude of the motion vector. The image change estimation unit 205 refers to table information indicating the correspondence between the amount of change in the magnitude of the motion vector and the amount of image change Δv, and predicts the amount of image change Δv corresponding to the calculated amount of change as the amount of image change Δv a predetermined time after the current time. Note that the table information is set so that the value of the amount of image change Δv increases as the amount of change in the magnitude of the motion vector increases. This makes it possible to predict in advance that the amount of image change Δv will increase a little later when it is detected that the magnitude of the motion vector has started to change.

[0118] <Variation 4> In the above embodiment, the image change estimation unit 205 predicts the image change amount Δv of the main direction image from the operation information. The image change estimation unit 205 may predict the image change amount Δv of the main direction image based on the measurement information of the angular velocity acquired by the measurement information acquisition unit 204.

[0119] For example, the image change estimation unit 205 calculates the difference between the angular velocity at the current time and the angular velocity a predetermined time ago as the amount of change in the angular velocity. The image change estimation unit 205 refers to table information indicating the correspondence between the amount of change in the angular velocity and the amount of image change Δv, and predicts the amount of image change Δv corresponding to the calculated amount of change as the amount of image change Δv a predetermined time after the current time. Note that the table information is set so that the value of the amount of image change Δv increases as the amount of change in the angular velocity increases. This makes it possible to predict in advance that the amount of image change Δv will increase a little later when it is detected that the angular velocity has started to change.

[0120] <Variation 5> Although the measurement information acquiring unit 204 acquires measurement information of the angular velocity of the moving object 2 from the angular velocity sensor 23, the measurement information acquired by the measurement information acquiring unit 204 is not limited to measurement information of angular velocity. For example, the measurement information acquiring unit 204 may acquire measurement information of the speed of the moving object 2 from a speedometer mounted on the moving object 2. In addition, the measurement information acquiring unit 204 may acquire measurement information of the acceleration of the moving object 2 from an acceleration sensor mounted on the moving object 2.

[0121] The image change estimation section 205 may estimate the change in the main direction image based on measurement information of the speed or acceleration of the moving object 2 .

[0122] [Note] Each process (each function) of the above-mentioned embodiment is realized by a processing circuit including one or more processors. The processing circuit may be configured by an integrated circuit or the like in which one or more memories, various analog circuits, and various digital circuits are combined in addition to the one or more processors. The one or more memories store programs (instructions) that cause the one or more processors to execute each of the above processes. The one or more processors may execute each of the above processes according to the programs read from the one or more memories, or may execute each of the above processes according to a logic circuit designed in advance to execute each of the above processes. The processor may be various processors suitable for computer control, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit). The physically separated processors may cooperate with each other to execute each of the above processes. For example, the processors mounted on each of a plurality of physically separated computers may execute the above processes in cooperation with each other via a network such as a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, etc. The program may be installed in the memory from an external server device or the like via the network, or may be distributed in a state stored in a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a semiconductor memory, and may be installed in the memory from the recording medium.

[0123] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims. [Explanation of symbols]

[0124] 1. Video transmission system 2. Mobile 3A Main direction 3B Main direction 20 Video transmission device 21 Camera 22 Autonomous Driving ECU 23 Angular rate sensor 30 Transmission Line 40 Video receiving device 40A Display 201 Video Acquisition Unit 202 Attention area extraction part 203 Operation information acquisition unit 204 Measurement information acquisition unit 205 Image Change Prediction Unit 206 Video Compression Section 207 Data volume control section 208 Area of ​​interest information output unit 209 Multiplex section 210 Video transmission unit 401 Video receiving unit 402 Video expansion unit 403 Region of interest information extraction unit 404 Video Extraction Section 405 Display control unit

Claims

1. A video transmitting device that transmits video to a video receiving device in real time via a transmission path with a limited communication bandwidth, an image acquisition unit that acquires a main direction image that is an image of a main direction of a moving object; an image change estimation unit that estimates a change in the main direction image; a video compression unit that generates a compressed video by compressing the main direction video; a data amount control unit that controls the data amount of the compressed image based on a result of the estimation by the image change estimation unit; a video transmitting unit configured to transmit the compressed video to the video receiving device.

2. The video transmission device according to claim 1 , wherein the data amount control unit controls the data amount of the compressed video so that a transmission delay time of the compressed video by the video transmission unit is equal to or less than a predetermined time threshold.

3. The video transmission device further includes an operation information acquisition unit that acquires operation information related to an operation of the moving object, 3. The video transmission device according to claim 1, wherein the video change inferring section infers a change in the main direction video prior to the change in the main direction video based on the operation information.

4. 3. The video transmission device of claim 1 or 2, wherein the data amount control unit executes at least one of controlling the compression rate of the main direction video, instructing to thin out images constituting the main direction video, instructing to copy images constituting the main direction video, controlling the resolution of the main direction video, instructing to transmit only I frame images of the compressed video, and transmitting a refresh signal to the video compression unit based on a result of estimation by the video change estimation unit.

5. The video transmission device further includes an attention area extraction unit that extracts an attention area from the main direction video, 3. The video transmission device according to claim 1, wherein the data amount control unit controls a compression rate of at least one of the region of interest and a non-region of interest other than the region of interest, based on a result of the estimation by the image change estimation unit.

6. The video transmission device further includes a measurement information acquisition unit that acquires measurement information related to a movement of the moving object, The video transmission device according to claim 1 , wherein the video change estimation unit estimates the change in the main direction video based on the measurement information.

7. 3. The video transmission device according to claim 1, wherein the video change estimation unit estimates the change in the main direction video based on a motion vector between images that constitute the main direction video.

8. A video transmission device that transmits video in real time via a transmission path having a limited communication bandwidth; a video receiving device for receiving the video from the video transmitting device, The video transmission device includes: an image acquisition unit that acquires a main direction image that is an image of a main direction of a moving object; an image change estimation unit that estimates a change in the main direction image; a video compression unit that generates a compressed video by compressing the main direction video; a data amount control unit that controls the data amount of the compressed image based on a result of the estimation by the image change estimation unit; a video transmitting unit that transmits the compressed video to the video receiving device.

9. A video transmission method by a video transmitting device for transmitting video in real time to a video receiving device via a transmission path having a limited communication bandwidth, comprising: acquiring a main direction image which is an image of a main direction of a moving object; estimating a change in the main view; generating a compressed image by compressing the main direction image; controlling the amount of data of the compressed image based on the predicted change in the main direction image; transmitting the compressed video to the video receiving device.

10. A computer program for causing a computer to function as a video transmitting device that transmits video to a video receiving device in real time via a transmission path with a limited communication bandwidth, comprising: The computer, an image acquisition unit that acquires a main direction image which is an image of a main direction of a moving object; an image change estimation unit for estimating a change in the main direction image; a video compression unit for generating a compressed video by compressing the main direction video; a data amount control unit that controls the data amount of the compressed video based on a result of the estimation by the video change estimation unit; and A computer program product that causes the computer to function as a video transmission unit that transmits the compressed video to the video receiving device.

Images