Video distribution device, system, method, and program

Abstract

The present invention suppresses any decrease in the accuracy of predicting communication throughput. This video delivery device (1) comprises a pixel shift unit (11) for causing the pixel values of pixels in a captured video to change when the data volume of the captured video after encoding decreases, an encoding unit (12) for encoding a captured image, a transmission unit (13) for transmitting the encoded captured image, and a prediction unit (14) for predicting communication throughput using the effective throughput of transmission of the captured image.

Description

Technical Field

[0001] The present disclosure relates to a video distribution apparatus, system, method, and computer-readable medium.

Background Art

[0002] Techniques for predicting communication throughput and realizing optimal video distribution have been proposed. In such techniques, the communication throughput is predicted while distributing video, and the bit rate and frame rate of the video are optimized based on the prediction result. As a result, the video is transmitted stably without interruption.

[0003] In such techniques, the prediction of communication throughput is performed using the effective throughput of video transmission. Therefore, when the bit rate of video transmission decreases when there is margin in the line, there is a problem that the prediction accuracy of communication throughput decreases.

[0004] Note that Patent Document 1 discloses a technique related to a server that performs streaming distribution.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] As described above, there is a problem that the prediction accuracy of communication throughput decreases in a video distribution system.

[0007] In view of the above circumstances, an object of the present disclosure is to provide a video distribution apparatus, system, method, and computer-readable medium that suppress a decrease in the prediction accuracy of communication throughput.

Means for Solving the Problems

[0008] To achieve the above object, the present disclosure provides a pixel shift means for changing pixel values of pixels in the captured video when the data amount after encoding of the captured video becomes small; an encoding means for encoding the captured video; a transmission means for transmitting the encoded captured video; a prediction means for predicting communication throughput using the effective throughput of transmission of the captured video. A video distribution apparatus is provided.

[0009] The present disclosure provides a video distribution apparatus; a receiving apparatus; and the video distribution apparatus when the data amount after encoding of the captured video becomes small, changes pixel values of one or more pixels included in the captured video, encodes the captured video, transmits the encoded captured video to the receiving apparatus, and predicts communication throughput using the effective throughput of transmission of the captured video. A video distribution system is provided.

[0010] The present disclosure provides a method for a computer to when the data amount after encoding of the captured video becomes small, change pixel values of pixels in the captured video, encode the captured video, transmit the encoded captured video and predict communication throughput using the effective throughput of transmission of the captured video. A video distribution method is provided.

[0011] The present disclosure provides a process for a computer to when the data amount after encoding of the captured video becomes small, change pixel values of pixels in the captured video; a process for encoding the captured video; A process of transmitting the encoded captured video and A process of predicting the communication throughput using the actual throughput of the transmission of the captured video, and A non-transitory computer-readable medium storing a program for causing the above to be executed is provided.

Advantages of the Invention

[0012] The video distribution apparatus, system, method, and computer-readable medium according to the present disclosure suppress a decrease in the prediction accuracy of communication throughput.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Modes for Carrying Out the Invention

[0014] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and redundant descriptions are omitted as necessary for clarity of explanation.

[0015] (Embodiment 1) FIG. 1 is a block diagram showing the overall configuration of a video distribution apparatus 1 according to Embodiment 1. The video distribution apparatus 1 is a computer that performs communication throughput prediction (bandwidth prediction) based on the actual throughput of video distribution. The video distribution apparatus 1 is connected to a receiving apparatus (not shown) that receives video via a network (not shown). The network (not shown) may be wired or wireless. The video distribution apparatus 1 transmits, for example, captured video from an in-vehicle camera or a surveillance camera to the receiving apparatus (not shown). The video distribution apparatus 1 includes a pixel shift unit 11, an encoding unit 12, a transmission unit 13, and a prediction unit 14.

[0016] When the data amount of the captured video after encoding becomes small, the pixel shift unit 11 changes the pixel values of the pixels in the captured video. When the captured video is a moving image, the pixel shift unit 11 may change the pixel values of the pixels within a frame (hereinafter also referred to as a captured image). The process of changing the pixel values of the pixels in the captured video is also referred to as pixel shift processing. For example, when the captured video becomes all black at night or the like, the data amount of the captured video after encoding becomes small, so pixel shift processing is performed. The pixel shift unit 11 may, for example, select a number of pixels according to the hit rate from all the pixels included in the captured image, and change the pixel values of the selected pixels according to the shift width.

[0017] The encoding unit 12 encodes the captured video. When pixel shift processing is performed, the encoding unit 12 encodes the video on which pixel shift processing has been performed, and when not, encodes the captured video as it is. The encoding unit 12 may encode the captured video according to a target bit rate based on the prediction result of the prediction unit 14 described later.

[0018] The transmission unit 13 transmits the captured video encoded by the encoding unit 12 to the receiving apparatus (not shown). The prediction unit 14 performs prediction of communication throughput (bandwidth prediction) using the actual throughput of transmission of the captured video. Here, the receiving apparatus (not shown) may feedback the actual throughput to the video distribution apparatus 1.

[0019] In this way, when the data amount after encoding the captured video becomes small, the video distribution device according to this embodiment changes the pixel values of the pixels in the captured video. Therefore, since it is possible to prevent a decrease in the effective throughput of transmission, it is possible to suppress a decrease in the prediction accuracy of the communication throughput.

[0020] Note that the video distribution device 1 includes a processor, a memory, and a storage device as a configuration not shown in the figure. Further, a computer program in which the processing of the information processing method according to this embodiment is implemented is stored in the storage device. Then, the processor causes the memory to read the computer program from the storage device and executes the computer program. Thereby, the processor realizes the functions of the pixel shift unit 11, the encoding unit 12, the transmission unit 13, and the prediction unit 14.

[0021] Alternatively, the pixel shift unit 11, the encoding unit 12, the transmission unit 13, and the prediction unit 14 may each be realized by dedicated hardware. Further, some or all of the components of each device may be realized by general-purpose or dedicated circuitry, a processor, etc. or a combination thereof. These may be configured by a single chip or by a plurality of chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-described circuitry, etc. and a program. Further, as the processor, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (field-programmable gate array), a quantum processor (quantum computer control chip), etc. can be used.

[0022] In addition, when some or all of the components of the video distribution device 1 are realized by a plurality of information processing devices, circuits, etc., the plurality of information processing devices, circuits, etc. may be centrally arranged or may be distributed. For example, the information processing devices, circuits, etc. may be realized in a form in which each is connected via a communication network, such as a client-server system or a cloud computing system. Further, the function of the video distribution device 1 may be provided in the form of SaaS (Software as a Service).

[0023] (Embodiment 2) Embodiment 2 is a specific example of Embodiment 1 described above. The video distribution system according to Embodiment 2 uses a technique called an adaptive network technique to predict the communication throughput while distributing video, and optimizes the bit rate and frame rate of the video based on the prediction result.

[0024] FIG. 2 is a block diagram showing the configuration of a video distribution system 1000 according to the present Embodiment 2. The video distribution system 1000 includes a camera 100, a transmission terminal 200, and a reception device 300. The transmission terminal 200 and the reception device 300 are connected by a network 400. The network 400 includes, for example, a network using a communication line standard such as LTE (Long Term Evolution), WiFi (registered trademark), or a wireless communication network such as a fifth-generation mobile communication system.

[0025] The camera 100 is also referred to as a video capture unit. The camera 100 captures a captured video. The video distribution system 1000 may include a plurality of cameras 100.

[0026] The transmission terminal 200 is a specific example of the video distribution device 1 described above. The transmission terminal 200 includes a pixel shift unit 210, an encoder 220, a transmission unit 230, a communication throughput prediction unit 240, and an adaptive network control unit 250. The transmission terminal 200 may be an in-vehicle terminal mounted on a vehicle.

[0027] The pixel shift unit 210 is a specific example of the pixel shift unit 11 described above. The pixel shift unit 210 performs pixel shift processing when the data amount after encoding the captured video of the camera 100 becomes extremely small.

[0028] Specifically, the pixel shift processing is performed when the captured video becomes a simple or uniform video such as all black. For example, when the camera 100 is covered with a hand or a cloth, the captured video becomes a simple video. Also, in the case of an in-vehicle camera, the captured video at night may become a simple video. In such cases, the data after encoding becomes extremely small, and the transmission bit rate decreases.

[0029] Hereinafter, the pixel shift processing will be specifically described. Note that the conditions for executing the pixel shift processing will be described later. The pixel shift unit 210 first determines whether each pixel included in the captured image is a target of the pixel shift processing. The pixel shift unit 210 selects a number of pixels calculated by the product of the vertical pixel number (height), the horizontal pixel number (width), and the hit rate (%). The hit rate may be set in advance. The pixel shift unit 210 may randomly select (extract) the target of the pixel shift processing.

[0030] Next, the pixel shift unit 210 changes the pixel value of the selected pixel. When the RGB (Red Green Blue) value of the selected pixel is smaller than the threshold value (for example, 128 when the maximum value is 255), the pixel shift unit 210 adds the shift width to the RGB value. As a result, the pixel becomes slightly brighter. Also, when the RGB value of the selected pixel is equal to or greater than the threshold value (for example, 128 when the maximum value is 255), the pixel shift unit 210 subtracts the shift width from the RGB value.

[0031] As a specific example, the case of performing pixel shift processing on a pixel with RGB values (232, 108, 55) will be described. 232 is 128 or more, 108 is less than 128, and 55 is less than 128. Therefore, when the shift width is 1, the pixel shift unit 210 changes RGB(232, 108, 55) to RGB(231, 109, 56).

[0032] As a result of performing pixel shift (for example, hit rate 0.1%, shift width 64), the captured image becomes an image with noise partially added. The hit rate and the shift width may be appropriately set according to the use case. When the hit rate is high, since there are many target pixels for the pixel shift processing, the processing load becomes high. On the other hand, when the shift width is large, since the amount of change in the pixel value is large, the influence on the appearance of the image becomes large.

[0033] For example, when the camera 100 is a fixed surveillance camera, it is important that the content of the image is not impaired. In such a case, a high hit rate (for example, 30%) is set and a low shift width (for example, 1) is set. Also, when the camera 100 is an in-vehicle camera, real-time performance is important. In such a case, a low hit rate (for example, 0.3%) is set and a high shift width (for example, 24) is set.

[0034] The encoder 220 is a specific example of the encoding unit 12 described above and is also referred to as an encoding unit. The encoder 220 encodes the imaging video by the camera 100. When pixel shift processing is performed, the encoder 220 encodes the imaging video on which the pixel shift processing has been performed. The encoder 220 performs encoding according to the bit rate and the frame rate determined by the adaptive network control unit 250 described later.

[0035] The transmission unit 230 is a specific example of the transmission unit 13 described above and is also referred to as a TCP / IP communication unit. The transmission unit 230 transmits the imaging video encoded by the encoder 220 to the receiving device 300.

[0036] The communication throughput prediction unit 240 is a specific example of the prediction unit 14 described above, and predicts the future communication throughput (bandwidth prediction). The communication throughput prediction unit 240 receives the effective throughput from the receiving device 300 and predicts the communication throughput.

[0037] The adaptive network control unit 250 optimizes the video bit rate and frame rate according to the communication throughput prediction by the communication throughput prediction unit 240. The adaptive network control unit 250 may perform an allocation such as a bit rate for each of the plurality of cameras 100. The adaptive network control unit 250 outputs the target bit rate of the video to the encoder.

[0038] The receiving device 300 includes a receiving unit 310 and a playback unit 320. The receiving unit 310 is also referred to as a TCP / IP communication unit and receives the captured video from the transmitting terminal 200. The playback unit 320 is also referred to as a decoding / display unit, decodes the received captured video, and displays it on a display device such as a display.

[0039] When the bit rate of the captured video is decreased, the video distribution system 1000 performs data filling by pixel shift processing. Therefore, even when the data amount after encoding the captured video becomes extremely small, it is possible to prevent the predicted value from decreasing and to prevent a decrease in the accuracy of the bandwidth prediction. In addition, since the video distribution system 1000 slightly changes the captured image, it is possible to suppress a sense of incongruity and data loss when viewing the captured video.

[0040] Next, with reference to FIG. 3, the conditions for performing the pixel shift processing will be described. FIG. 3 is a schematic diagram schematically showing the time change of the target bit rate determined by the adaptive network control unit 250 and the bit rate of the most recently encoded captured video (also referred to as the effective bit rate).

[0041] The horizontal axis of FIG. 3 indicates time [t], and the vertical axis indicates throughput [Mbps]. The time change of the target bit rate is indicated by a dashed line, and the time change of the actual bit rate is indicated by a solid line. In reality, when the actual bit rate decreases, the target bit rate also decreases, but for the sake of clarity, it is assumed that the target bit rate does not decrease.

[0042] The transmitting terminal 200 executes pixel shift processing based on the comparison result between the bit rate (actual bit rate) of the most recently encoded captured video and the target bit rate. Specifically, the transmitting terminal 200 performs pixel shift processing when the following two conditions (A) and (B) are satisfied. (A) is that the bit rate (actual bit rate) of the most recently encoded captured video is less than a threshold value (for example, 1.0 Mbps). (B) is that the bit rate (actual bit rate) of the most recently encoded captured video is less than the product of the target bit rate and a predetermined ratio (for example, 0.5). When such conditions are satisfied, the transmitting terminal 200 performs pixel shift processing, increases the data amount up to near the threshold value as indicated by the upward arrow, and prevents the predicted value of the bandwidth from decreasing.

[0043] Next, with reference to FIGS. 4 and 5, the results of the verification performed by the inventor will be described. The resolution of the video was set to 1280×720, the hit rate was set to 0.3%, and the shift width was set to 24. The condition for performing pixel shift processing was that the actual bit rate was 2.0 Mbps or less and 50% or less of the target bit rate. In the captured video for verification, frames consisting only of black images were inserted during the period from 15 seconds to 35 seconds after the start.

[0044] FIG. 4 is a graph showing the time variations of the band prediction value (predicted value of communication throughput) and the effective throughput when pixel shifting processing is not performed. The horizontal axis represents time, and the vertical axis represents throughput. The section indicated by T represents the section in which frames of black images are inserted. As the measured throughput decreases, the band prediction value also significantly decreases. Also, after the section indicated by T ends, it takes time for the band prediction value to return to its original value.

[0045] FIG. 5 is a graph showing the time variations of the band prediction value and the effective throughput when pixel shifting processing is performed. The horizontal axis represents time, and the vertical axis represents throughput. The section indicated by T represents the section in which frames of black images are inserted. When pixel shifting processing is performed, since the decrease in the effective throughput is suppressed, the decrease in the band prediction value is also suppressed. Also, after the section indicated by T ends, the band prediction value quickly returns to its original value.

[0046] The video distribution system according to Embodiment 2 increases the data amount without significantly degrading the appearance of the captured video. Therefore, the video distribution system according to Embodiment 2 can maintain the accuracy of band prediction even in an environment where the data amount of the video is extremely small.

[0047] As described above, the embodiments of the present disclosure have been described in detail. However, the present disclosure is not limited to the above-described embodiments, and modifications and changes made to the above embodiments without departing from the spirit of the present disclosure are also included in the present disclosure.

Description of Reference Numerals

[0048] 1: Video distribution device 11: Pixel shift unit 12: Encoding unit 13: Transmission unit 14: Prediction unit 1000: Video distribution system 100: Camera 200: Transmission terminal 210: Pixel shift unit 220: Encoder 230: Transmission unit 240: Communication throughput prediction unit 250: Adaptive network control unit 300: Receiver 310: Receiving unit 320: Reproduction unit 400: Network

Claims

1. When the data amount after encoding of the captured video becomes small, pixel shifting means for changing the pixel values of the pixels in the captured video, encoding means for encoding the captured video, transmission means for transmitting the encoded captured video, prediction means for predicting the communication throughput using the effective throughput of the transmission of the captured video, comprising the pixel shifting means selects a number of the pixels according to the hit rate and changes the pixel values of the selected pixels according to the shift width, a video distribution device.

2. the pixel shifting means when the pixel value of the selected pixel is smaller than the threshold value, adds the shift width to the pixel value, when the pixel value of the selected pixel is equal to or greater than the threshold value, subtracts the shift width from the pixel value, The video distribution device according to claim 1.

3. the encoding means encodes the captured video according to a target bit rate based on the prediction result of the prediction means, the pixel shifting means changes the pixel values of the pixels in the captured video based on the comparison result between the bit rate of the most recently encoded captured video and the target bit rate, The video distribution device according to claim 1 or 2.

4. the pixel shifting means when the bit rate of the most recently encoded captured video is equal to or less than a predetermined ratio of the target bit rate and is equal to or less than the threshold value, changes the pixel values of the pixels in the captured video, The video distribution device according to claim 3.

5. a video distribution device, a receiving device, comprising the video distribution device when the data amount after encoding of the captured video becomes small, pixel shifting means for changing the pixel values of the pixels in the captured video, encoding means for encoding the captured video, transmission means for transmitting the encoded captured video to the receiving device, prediction means for predicting the communication throughput using the effective throughput of the transmission of the captured video, comprising the pixel shifting means selects a number of the pixels according to the hit rate and changes the pixel values of the selected pixels according to the shift width, a video distribution system.

6. A computer when the data amount after encoding of the captured video becomes small, changes the pixel values of the pixels in the captured video, encodes the captured video, transmits the encoded captured video, Predicting the communication throughput using the effective throughput of the transmission of the captured video; including; wherein changing the pixel values includes selecting a number of the pixels according to a hit rate and changing the pixel values of the selected pixels according to a shift width; Video distribution method.

7. Causing a computer to when the data amount of the captured video after encoding becomes small, perform a process of changing the pixel values of the pixels in the captured video, a process of encoding the captured video, a process of transmitting the encoded captured video, and a process of predicting the communication throughput using the effective throughput of the transmission of the captured video; and causing the computer to execute; wherein the process of changing includes selecting a number of the pixels according to a hit rate and changing the pixel values of the selected pixels according to a shift width; Program.

8. Pixel shift means for changing the pixel values of the pixels in the captured video when the data amount of the captured video after encoding becomes small; comprising; wherein the pixel shift means selects a number of the pixels according to a hit rate and changes the pixel values of the selected pixels according to a shift width; Video distribution apparatus.

Images