Encoding device, encoding method, encoding program, decoding device, decoding method, decoding program, streaming system, and streaming method

By generating and sharing a base stream with differential bitstreams, existing HEVC decoders can reconstruct SHVC-encoded video efficiently, reducing hardware requirements and costs.

JP7577807B2Active Publication Date: 2024-11-05KK TOSHIBA
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2023135546
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2023-08-23
Publication Date
2024-11-05
Estimated Expiration
2035-09-16

AI Technical Summary

Technical Problem

Existing decoders incompatible with SHVC-encoded data require dedicated hardware, limiting the ability of existing HEVC decoders to decode scalably expanded and low-bit-rate video data.

Method used

An encoding device generates a base stream shared with the decoder and transmits only differential bitstreams, allowing existing HEVC decoders to reconstruct the original video quality using pre-shared base streams and received differential bitstreams.

Benefits of technology

Enables high bandwidth utilization efficiency and reduces the need for dedicated SHVC decoders, enabling use of general-purpose HEVC decoders and lowering system costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007577807000001
    Figure 0007577807000001
  • Figure 0007577807000002
    Figure 0007577807000002
  • Figure 0007577807000003
    Figure 0007577807000003
Patent Text Reader

Abstract

To enable data that is subjected to scalable extension and low-band transmission to be decoded by an existing decoder.SOLUTION: A determination unit determines whether or not a base stream which is a reference of encoding and decoding and relevant to an input image to be shared with a decoding device is stored in a storage unit. If it is determined that the base stream relevant to the input image is stored in the storage unit, a base stream generation unit performs encoding processing on the base stream that is stored in the storage unit, and if it is determined that the base stream relevant to the input image is not stored in the storage unit, generates the base stream by performing encoding processing on the input image. A difference generation unit generates a difference bit stream being the difference between the base stream and the input image. A communication control unit transmits the base stream to the decoding device to share it with the decoding device, and then transmits the difference bit stream to the decoding device.SELECTED DRAWING: Figure 10
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an encoding device, an encoding program, and a streaming system. [Background technology]

[0002] Nowadays, SHVC encoding technology (scalable extension of HEVC) is known, which uses HDTV video transmission and can be expanded to super-resolution images such as 4K resolution at low bit rates. HDTV is an abbreviation for "High-definition television." SHVC is an abbreviation for "Scalable High-efficiency Video Coding." HEVC is an abbreviation for "High-efficiency Video Coding."

[0003] In addition to the inter-frame and intra-frame prediction used in HEVC, this SHVC encoding technology uses inter-layer prediction to predict 4K images from HDTV images, reducing the amount of data allocated to image areas where motion causes blur and allocating more data to in-focus image areas with little motion, improving subjective image quality. In addition, by removing noise components that are difficult to compress from 4K images in advance using spatio-temporal image processing, it is possible to suppress the loss of perceived resolution at low bit rates. Therefore, even if the additional bit rate is set to 10 Mbps, the perceived resolution of 4K images can be maintained. For example, a bit rate of about 30 Mbps was required to distribute 4K resolution images, but by using HEVC encoding technology, the required additional transmission bandwidth can be reduced to one-third, about 10 Mbps.

[0004] In a narrowband video transmission system using SHVC, a reference image is shared in advance between the sending side and the receiving side. The sending side sends out only the difference from the reference image as encoded data. The receiving side decodes and displays the received encoded data using the previously shared reference image. As a result, the encoded data contains only the difference from the reference image, reducing the amount of data and enabling video streaming with good bandwidth efficiency.

[0005] However, in a narrowband video transmission system using SHVC, in order for the receiving side to decode the video signal transmitted from the transmitting side, a dedicated SHVC-compatible decoder is required on the receiving side, making playback difficult with existing HEVC decoders. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2005-110244 A Summary of the Invention [Problem to be solved by the invention]

[0007] The problem to be solved by the present invention is to provide an encoding device, an encoding program, and a streaming system that enable scalably expanded data transmitted at a low bit rate to be decoded by an existing decoder. [Means for solving the problem]

[0008] According to an embodiment, the encoding device has a determination unit that determines whether a base stream, which is a reference for encoding and decoding and is a base stream shared with the decoding device and is related to an input image, is stored in the storage unit. The base stream generation unit encodes the base stream stored in the storage unit when the determination unit determines that the base stream related to the input image is stored in the storage unit, and encodes the input image to generate a base stream when the determination unit determines that the base stream related to the input image is not stored in the storage unit. The difference generation unit generates a difference bit stream that is the difference between the base stream generated by the base stream generation unit and the input image. The communication control unit controls the communication unit to transmit the base stream to the decoding device, thereby sharing the base stream with the decoding device, and thereafter controls the communication unit to transmit the difference bit stream generated by the difference generation unit to the decoding device. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a system configuration diagram of a streaming system according to the first embodiment. [Diagram 2] FIG. 2 is a hardware configuration diagram of a server device provided in the streaming system according to the first embodiment. [Diagram 3] FIG. 3 is a hardware configuration diagram of a client device provided in the streaming system according to the first embodiment. [Figure 4] FIG. 4 is a functional block diagram of the encoder of the server device. [Diagram 5] FIG. 5 is a functional block diagram of a decoder of the client device. [Figure 6] FIG. 6 is a flowchart for explaining the operation of generating a base stream to be shared with a client device. [Figure 7]FIG. 7 is a flowchart illustrating an operation of generating a differential bitstream based on a base stream and transmitting the differential bitstream to a client device. [Figure 8] FIG. 8 is a flowchart for explaining an operation of storing the base stream transmitted from the server device in the client device. [Figure 9] FIG. 9 is a flowchart illustrating an operation of decoding a bitstream transmitted from a server device. [Figure 10] FIG. 10 is a functional block diagram of an encoder in a streaming system according to the second embodiment. [Figure 11] FIG. 11 is a flowchart illustrating an operation of generating a base stream to be shared with a client device in the streaming system according to the second embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an operation of generating a differential bit stream based on a base stream and transmitting the differential bit stream to a client device in the streaming system according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, a streaming system according to an embodiment will be described in detail with reference to the drawings.

[0011] (overview) The streaming system of the embodiment is, for example, a system for transmitting and receiving surveillance video. The transmitting encoder generates and stores a base stream, such as an intra-picture NAL unit, which is a standard for encoding and decoding, for each screen pattern. NAL is an abbreviation of "Network Abstraction Layer". The generated base stream is also stored in the receiving decoder, and the base stream is shared between the encoder and the decoder. The base stream is stored in the receiving decoder when, for example, the receiving decoder is shipped from the factory or when it is initially set up. Alternatively, the base stream is dynamically updated asynchronously or synchronously during actual stream transmission.

[0012] When encoding an input screen, the encoder encodes the input video using a base stream shared with the decoder, and generates a differential bitstream such as a P picture NAL unit based on the encoding state. The bitstream transmitted to the decoder is only the differential bitstream. This reduces the amount of transmitted data. The differential bitstream transmitted to the decoder may include information indicating whether the base stream was used.

[0013] The decoder on the receiving side reconstructs a stream equivalent (same image quality and resolution) to the stream that the encoder was originally going to generate, using the base stream shared in advance and the differential bit stream received. In other words, the decoder reconstructs a bit stream in a state in which a screen corresponding to the differential bit stream reproduced using the base stream shared in advance and the differential bit stream received is encoded. The decoder then decodes and displays the reconstructed stream. This enables video streaming with good bandwidth utilization efficiency in screen transfer. Also, a dedicated decoder is not required. Also, the amount of reconstruction processing can be reduced compared to transcoding. Therefore, a receiving device (decoder) can be realized with a general-purpose decoder chip and an inexpensive CPU, and the cost of the streaming system can be reduced.

[0014] (First embodiment) FIG. 1 shows a system configuration diagram of a streaming system according to a first embodiment. As shown in FIG. 1, the streaming system according to the first embodiment includes a server device 1 and a client device 2. The server device 1 and the client device 2 are connected to each other via a network 3 such as the Internet. The server device 1 performs scalable extension processing on a predetermined video stream using an SHVC encoding technique to reduce the bit rate of a high-resolution video signal, such as a so-called 4K video signal or an 8K video signal, while maintaining the high-resolution feel of the signal, and distributes the signal via the network 3. SHVC is an abbreviation for "Scalable High-efficiency Video Coding." The client device 2 is a general HEVC decoder, and decodes and plays back an SHVC-encoded video signal without having a special configuration, as described later. The decoded video signal is displayed on a monitor device or recorded on a recorder device or the like. HEVC is an abbreviation for "High-efficiency Video Coding."

[0015] Fig. 2 shows a hardware configuration diagram of the server device 1. As shown in Fig. 2, the server device 1 includes a CPU 11, a ROM 12, a RAM 13, an HDD (hard disk drive) 14, and a communication unit 15. The CPU 11 to the communication unit 15 are connected to each other via a bus line 16. CPU is an abbreviation for "Central Processing Unit". ROM is an abbreviation for "Read Only Memory". RAM is an abbreviation for "Random Access Memory".

[0016] An encoding program for encoding a video stream is stored in the HDD 14. The CPU 11 operates according to the encoding program to function as an encoder 17, and SHVC-encodes the video stream and distributes it. The encoding program may be stored in another storage unit such as the ROM 12 or the RAM 13.

[0017] Fig. 3 shows a hardware configuration diagram of the client device 2. As shown in Fig. 3, the client device 2 includes a CPU 21, a ROM 22, a RAM 23, an HDD (hard disk drive) 24, and a communication unit 25. The CPU 21 to the communication unit 25 are connected to each other via a bus line 26.

[0018] The HDD 24 stores a decoding program for decoding the video stream that is SHVC encoded and transmitted from the server device 1. The CPU 21 operates according to the decoding program to function as a decoder 27 and decodes the video stream received from the server device 1. The decoding program may be stored in another storage unit such as the ROM 22 or the RAM 23.

[0019] In this example, the encoder 17 and the decoder 27 are assumed to be realized by software. However, all, any one or more of the encoder 17 and the decoder 27 may be realized by hardware such as an integrated circuit (IC).

[0020] The encoding program and the decoding program may be provided by being recorded in a computer-readable recording medium such as a CD-ROM or a flexible disk (FD) in an installable or executable format file. The encoding program and the decoding program may be provided by being recorded in a computer-readable recording medium such as a CD-R, a DVD, a Blu-ray Disc (registered trademark), or a semiconductor memory. DVD is an abbreviation for "Digital Versatile Disk." The encoding program and the decoding program may be provided by being installed via a network such as the Internet. The encoding program and the decoding program may be provided by being pre-installed in a ROM or the like in the device.

[0021] Fig. 4 is a functional block diagram of the encoder 17 of the server device 1. The encoder 17 is an example of an encoding device. As shown in Fig. 3, the encoder 17 has a pre-processing unit 31, a storage control unit 32, a first transmission control unit 33, a second buffer control unit 34, a second encoding unit 35, an adjustment unit 36, and a second transmission control unit 37. The pre-processing unit 31 has a first buffer control unit 38 and a first encoding unit 39. As described above, all or part of the pre-processing unit 31 to the second transmission control unit 37 may be configured by hardware.

[0022] The pre-processing unit 31 is a processing unit that functions when the above-mentioned base stream is generated in advance and stored in the decoder, for example, when the receiving client device 2 is shipped from the factory or when the client device 2 is initially set up. Therefore, the pre-processing unit 31 is not required when the base stream on the decoder side is dynamically updated asynchronously or synchronously during actual stream transmission. Although the pre-processing unit 31 is illustrated in FIG. 4 as an independent processing unit, the pre-processing unit 31 can be omitted by substituting the second buffer control unit 34 for the first buffer control unit 38 and the second encoding unit 35 for the first encoding unit 39.

[0023] The storage control unit 32 controls storage of the base stream, which is generated by the pre-processing unit 31 and is to be shared between the client devices 2 (decoder 27), in a storage unit such as the RAM 13. The first transmission control unit 33 controls the communication unit 15 to transmit the base stream stored in the RAM 13 to the client device 2, and shares the base stream with the decoder 27.

[0024] The second buffer control unit 34 buffers the input video stream (input bitstream) in a storage unit such as the RAM 13. The second encoding unit 35, which is an example of a difference generating unit, generates a difference bitstream that is the difference between the base stream stored in the RAM 13 and the buffered video stream. This generates a scalable extended and low bitrate bitstream (a bitstream that has been subjected to SHVC encoding).

[0025] The adjustment unit 36 ​​adjusts the difference bit stream by, for example, adding identification information or related information of the base stream to be referenced. A second transmission control unit 37, which is an example of a communication control unit, controls the communication unit 15 to transmit the adjusted difference bit stream to the client device 2.

[0026] Fig. 5 is a functional block diagram of the decoder 27 of the client device 2. The decoder 27 is an example of a decoding device. As shown in Fig. 5, the decoder 27 has an adjustment processing unit 41 and a decoding processing unit 42. The adjustment processing unit 41 has a first reception control unit 43, a storage control unit 44, a second reception control unit 45, and a configuration adjustment unit 46. The decoding processing unit 42 has a decoding unit 47 and a buffer control unit 48. As described above, all or part of the first reception control unit 43 to the buffer control unit 48 may be configured by hardware.

[0027] The first reception control unit 43 controls the communication unit 25 to receive a base stream from the server device 1, which is generated in advance or dynamically updated as described above. The storage control unit 44 controls storage of the received base stream in a storage unit such as the RAM 23. The second reception control unit 45 controls the communication unit 25 to receive a differential bit stream from the server device 1. The configuration adjustment unit 46 uses the base stream shared in advance and the received differential stream to reconstruct a bit stream equivalent (with the same image quality and resolution) to the bit stream that was originally intended to be generated by the encoder 17.

[0028] The decoder performs normal HEVC decoding processing on the reconstructed bitstream. The buffer control unit 48 buffers the decoded bitstream in a storage unit such as the RAM 23, and sequentially reads and outputs the bitstream to a monitor device, a recording device, or the like.

[0029] Next, the flow chart of Fig. 6 shows a flow of operations for generating a base stream to be shared with the client device 2. For example, at the timing of pre-processing such as when the client device 2 serving as the receiving side is shipped from the factory, at the time of initial setup, or at the time of actual stream transmission, the first buffer control unit 38 of the pre-processing unit 31 shown in Fig. 4 buffers the pre-inputted picture (pre-inputted bitstream) in the RAM 13.

[0030] The pre-input picture may be a captured image, a picture obtained from a camera input, or a picture that has been subjected to a predetermined processing process, etc. In addition, the pre-input picture is preferably a picture that has a quality that allows it to be used as a reference picture when generating a difference bitstream, and that has high similarity to the picture from which the difference is generated.

[0031] The first encoding unit 39 reads the buffered pre-input pictures from the RAM 13 and generates a base stream for SHVC. The base stream may be a single type or multiple types. The base stream may be the whole or a part of the encoded bit stream. An example of a partial base stream is an I-picture alone of an elementary stream.

[0032] 6, the storage control unit 32 acquires the base stream in step S1, and determines in step S2 whether storage is necessary for adjustment processing such as adding identification information or related information. If storage is necessary (step S2: Yes), the process proceeds to step S3, and if storage is not necessary (step S2: No), the process proceeds to step S4.

[0033] In step S3, the adjustment unit 36 ​​performs adjustment processing such as adding identification information or related information to the base stream. Also in step S3, the storage control unit 32 temporarily stores in the RAM 23 the base stream that has been subjected to the adjustment processing.

[0034] Next, in step S4, the first transmission control unit 33 determines whether or not it is necessary to output (transmit) the base stream to the client device 2. If output is necessary (step S4: Yes), the first transmission control unit 33 controls the communication unit 15 in step S5 to transmit the base stream to the client device 2. On the other hand, if output is not necessary (step S4: No), the first transmission control unit 33 ends the process of the flowchart in FIG. 6 without transmitting the base stream.

[0035] The timing at which the first transmission control unit 33 transmits the base stream to the client device 2 may be, for example, when the storage control unit 32 notifies the client device 2 that a new base stream has been stored. Alternatively, the timing may be when an instruction to output the base stream is provided from a user or the system. Alternatively, the timing may be a predetermined timing, or when a transmission request is received from the client device 2. The transmission speed of the base stream is arbitrary, and may be a low bit rate transmission speed according to the communication line of the client device 2, or a high bit rate transmission speed in order to shorten the output time.

[0036] Next, the flow chart of Fig. 7 shows the flow of operations for generating a differential bit stream based on a base stream and transmitting the differential bit stream to the client device 2. In step S11, the second buffer control unit 34 shown in Fig. 4 obtains a picture to be encoded and buffers it in the RAM 13 or the like. In step S12, the second encoding unit 35 determines via the storage control unit 32 whether or not a base stream to be referred to when encoding an input picture is stored in the RAM 13.

[0037] If the base stream to be referenced is not stored in the RAM 13 (step S12: No), the second encoding unit 35 performs SHVC encoding on the input picture without referencing the base stream in step S14, and proceeds to step S15. On the other hand, if the base stream to be referenced is stored in the RAM 13 (step S12: Yes), the second encoding unit 35 performs SHVC encoding processing in step S13, referring to the base stream, to generate a difference bit stream that is the difference from the input picture, and proceeds to step S15.

[0038] In step S15, the adjustment unit 36 ​​performs an adjustment process to add identification information or related information of the base stream to be referenced to the difference bit stream generated by the SHVC encoding process. In step S16, the storage control unit 32 determines whether or not to store the bit stream generated by the SHVC encoding process in the RAM 13. If the bit stream is to be stored (step S16: Yes), the storage control unit 32 stores the bit stream generated by the SHVC encoding process in the RAM 13 in step S17. At this time, the storage control unit 32 stores the newly generated bit stream in the RAM 13, or updates the bit stream stored in the RAM 13 with the new bit stream. Alternatively, the storage control unit 32 stores only the difference between the bit stream stored in the RAM 13 and the new bit stream. If such a process by storage is not required (step S16: No), the process proceeds to step S18.

[0039] In step S18, the second transmission control unit 37 controls the communication unit 15 to transmit the differential bit stream and the like to the client device 2, and ends the process of the flowchart in Fig. 7. In the case of the streaming system of the first embodiment, the base stream is transmitted to the client device 2 first, and thereafter, only the differential bit stream, which is the difference from the base stream, is transmitted to the client device 2. This makes it possible to achieve a low bit rate and effectively utilize the transmission band.

[0040] Next, the flow chart of Fig. 8 shows the flow of the operation of storing the base stream transmitted from the server device 1 on the client device 2 side. In step S21, the first reception control unit 43 shown in Fig. 5 controls the communication unit 25 to acquire the base stream transmitted from the server device 1. In step S22, the storage control unit 44 determines whether or not the acquired base stream needs to be stored.

[0041] Specifically, the storage control unit 44 stores the acquired base stream in a storage unit such as the RAM 23 when a user or the system instructs output processing of the base stream, or when a predetermined timing is reached. Alternatively, the storage control unit 44 stores the acquired base stream in a storage unit such as the RAM 23 when a storage instruction is received from the server device 1, or when a storage instruction is received from the configuration adjustment unit 46. The input speed of the base stream is arbitrary, and may be a low bit rate input speed according to the communication line of the server device 1, or may be a high bit rate input speed to shorten the input time.

[0042] If it is determined in step S22 that storage is necessary, then in step S23, the storage control unit 44 controls storage of the acquired base stream in the RAM 23. This allows the base stream to be shared between the server device 1 and the client device 2. If it is determined in step S22 that storage is not necessary, the storage control unit 44 ends the process of the flowchart in FIG. 8 as it is.

[0043] 9 shows a flow of decoding the bitstream transmitted from the server device 1. In step S31, the second reception control unit 45 controls the communication unit 25 to acquire the bitstream transmitted from the server device 1. In step S32, the configuration adjustment unit 46 determines whether or not a base stream is required to decode the bitstream.

[0044] If the base stream is necessary (step S32: Yes), in step S33, the configuration adjustment unit 46 acquires the base stream pre-stored in the RAM 23, and proceeds to step S34. On the other hand, if the base stream is not necessary (step S32: No), the configuration adjustment unit 46 proceeds to step S34.

[0045] In step S34, the configuration adjustment unit 46 uses the previously shared base stream and the received subsequent differential bit stream to reconstruct a bit stream equivalent (same image quality, same resolution) to the bit stream that was originally intended to be generated by the encoder 17 of the server device 1. In other words, the configuration adjustment unit 46 reconstructs a bit stream in a state in which a screen corresponding to the differential bit stream reproduced using the previously shared base stream and the received differential bit stream is encoded. In step S35, the decoding unit 47 performs normal HEVC decoding processing on the reconstructed bit stream to decode it. Then, the buffer control unit temporarily stores the decoded bit stream in the RAM 23, and sequentially reads it out and outputs it to a monitor device, a recording device, or the like.

[0046] The streaming system of the first embodiment can achieve streaming with good bandwidth utilization efficiency in bitstream transfer. Also, a normal HEVC decoder can decode a bitstream that has been SHVC encoded, without using a decoder dedicated to SHVC. Also, the amount of reconstruction processing can be reduced compared to transcoding. Therefore, the client device 2 can be realized with a general-purpose decoder chip and an inexpensive CPU, and the cost of the streaming system can be reduced.

[0047] (Second embodiment) Next, a streaming system according to a second embodiment will be described. The streaming system according to the first embodiment is an example of transmitting a bit stream of moving images, whereas the streaming system according to the second embodiment is an example of transmitting a bit stream of still images (pictures). Note that the first embodiment and the second embodiment differ only in this respect. Therefore, hereinafter, only the differences will be described, and duplicated explanations will be omitted.

[0048] Fig. 10 is a functional block diagram of the encoder 17 of the streaming system of the second embodiment. In Fig. 10, the same reference numerals are used for the parts showing the same operations as in Fig. 4. As shown in Fig. 10, the encoder 17 of the streaming system of the second embodiment has, in addition to the functions of Fig. 4, a picture storage control unit 51 that controls storage in the RAM 13 of pictures buffered by the first buffer control unit 38 and the second buffer control unit 34. Also, it has a determination unit 52 that determines (selects) pictures to be encoded from among the pictures stored in the RAM 13.

[0049] The flow chart in Fig. 11 shows a flow of an operation for generating a base stream to be shared with the client device 2. For example, at the timing of pre-processing such as when the client device 2 serving as the receiving side is shipped from a factory, at the time of initial setup, or at the time of actual stream transmission, the first buffer control unit 38 of the pre-processing unit 31 shown in Fig. 10 buffers a picture that has been input in advance (pre-input picture) in step S41.

[0050] The pre-input picture may be a captured image, a picture obtained from a camera input, or a picture that has been subjected to a predetermined processing process, etc. In addition, the pre-input picture is preferably a picture that has a quality that allows it to be used as a reference picture when generating a difference bitstream, and that has high similarity to the picture from which the difference is generated.

[0051] In step S42, the picture storage control unit 51 cooperates with the determination unit 52 to determine whether or not the pre-inputted picture needs to be stored. If storage is required (step S41: Yes), in step S43, the picture storage control unit 51 controls storage of the pre-inputted picture in the RAM 13, and proceeds to step S44. On the other hand, if storage is not required (step S41: No), the picture storage control unit 51 proceeds to step S44.

[0052] In step S44, the first encoding unit 39 reads the pre-input picture from the RAM 13 and generates a base stream for SHVC. The base stream may be of a single type or of multiple types. The base stream may be the whole or a part of the encoded bit stream. An example of a partial base stream is an I-picture alone of an elementary stream.

[0053] Next, in step S45, the storage control unit 32 determines whether or not the base stream needs to be stored for adjustment processing such as adding identification information or related information, etc. If storage is required (step S45: Yes), the process proceeds to step S46, and if storage is not required (step S45: No), the process proceeds to step S47.

[0054] In step S46, the storage control unit 32 controls the storage of the base stream in the RAM 13, and the adjustment unit 36 ​​performs adjustment processing such as adding identification information or related information to the base stream, if necessary. Then, in step S47, the first transmission control unit 33 controls the communication unit 15 to transmit the base stream to the client device 2, and the processing of the flowchart in Fig. 6 ends. As a result, the base stream is shared between the server device 1 and the client device 2.

[0055] The timing at which the first transmission control unit 33 transmits the base stream to the client device 2 may be, for example, when the storage control unit 32 notifies the client device 2 that a new base stream has been stored. Alternatively, the timing may be when an instruction to output the base stream is provided from a user or the system. Alternatively, the timing may be a predetermined timing, or when a transmission request is received from the client device 2. The transmission speed of the base stream is arbitrary, and may be a low bit rate transmission speed according to the communication line of the client device 2, or a high bit rate transmission speed in order to shorten the output time.

[0056] Next, the flowchart in Fig. 12 shows the flow of operations for generating a differential bitstream based on a base stream and transmitting it to the client device 2. In step S51, the second buffer control unit 34 shown in Fig. 10 obtains an input picture to be encoded and performs buffering processing in the RAM 13 or the like. In step S52, the picture storage control unit 51 cooperates with the determination unit 52 to determine whether or not a picture (base stream) to be referenced that is related to the input picture exists (whether or not it is stored in the RAM 13).

[0057] If a picture to be referenced exists (step S52: Yes), the second encoding unit 35 SHVC-encodes the picture to be referenced instead of the input picture, and generates a base stream. On the other hand, if a picture to be referenced does not exist (step S52: No), the second encoding unit 35 SHVC-encodes the input picture, and generates a base stream. Note that the picture storage control unit 51 may cooperate with a determination unit 52 that determines the picture to be encoded to determine whether or not the input picture needs to be stored, and if necessary, store the input picture in the RAM 13 or the like.

[0058] Next, in step S55, the adjustment unit 36 ​​performs an adjustment process of adding identification information or related information of the base stream to be referenced to the bit stream generated by SHVC encoding. In step S56, the storage control unit 32 determines whether or not to store the bit stream generated by SHVC encoding in the RAM 13. If it is to be stored (step S56: Yes), the storage control unit 32 stores the bit stream generated by the SHVC encoding process in the RAM 13 in step S57. At this time, the storage control unit 32 stores the newly generated bit stream in the RAM 13, or stores the bit stream stored in the RAM 13 by updating it with the new bit stream. Alternatively, the storage control unit 32 stores only the difference between the bit stream stored in the RAM 13 and the new bit stream. Note that if such a process by storage is not required (step S56: No), the process proceeds to step S58.

[0059] In step S58, the second transmission control unit 37 controls the communication unit 15 to transmit the bit stream and the like to the client device 2, and ends the process of the flowchart in Fig. 12. In the case of the streaming system of the second embodiment, a base stream which is a reference picture is also transmitted to the client device 2 first, and thereafter, only a difference picture which is a difference from the base stream is transmitted to the client device 2. This makes it possible to achieve a lower bit rate, and to obtain the same effects as those of the first embodiment described above, such as efficient use of the transmission band.

[0060] In the second embodiment, the decoder 27 of the client device 2 also uses a prestored base stream and the received input pictures to reconstruct a bit stream equivalent (with the same image quality and resolution) to the bit stream that was originally intended to be generated by the encoder 17. Then, the decoder 27 plays back the reconstructed stream for display or the like.

[0061] Although the embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and gist of the invention. [Explanation of symbols]

[0062] 1. Server device 2. Client Device 3. Network 11 CPU 12 ROM 13 RAM 14 HDD 15 Communications Department 17 Encoder 21 CPU 22 ROM 23 RAM 24 HDD 25 Communications Department 27 Decoder 31 Pre-processing section 32 Memory control unit 33 First transmission control section 34 Second buffer control section 35 Second Encoding Section 36 Adjustment part 37 Second transmission control section 39 Encoding section 38 First buffer control section 41 Adjustment processing section 42 Decoding processing section 43 First reception control section 44 Memory control unit 45 Second reception control section 46 Configuration adjustment section 47 Decoding section 48 Buffer control section 51 Picture storage control unit 52 Judgment section

Claims

1. a difference generating unit that generates a difference bit stream that is a difference between a base stream shared in advance with a decoding device and an input image; a communication control unit that controls a communication unit to transmit the differential bit stream generated by the differential generation unit to the decoding device; the base stream is generated based on a pre-input picture, which has a quality that can be used as a reference picture when generating the differential bitstream, is an image that is highly similar to an input image from which a difference is generated, and is an image that is generated without using the input image; the communication control unit controls the communication unit not to transmit a bitstream generated based on the pre-input picture to the decoding device, and to transmit the difference bitstream generated by the difference generation unit to the decoding device. Encoding device.

2. A difference generating unit that generates a difference bit stream that is a difference between a base stream shared in advance with a decoding device and an input image; a communication control unit that controls a communication unit to transmit the differential bit stream generated by the differential generation unit to the decoding device; an adjustment unit that adds identification information of the base stream to the difference bit stream; the base stream is generated based on a pre-input picture, which is an image quality that can be used as a reference image when generating the differential bitstream, has high similarity to an input image from which a difference is generated, and is generated without using the input image. Encoding device.

3. An encoding method executed by an encoding device, comprising: a difference generating step of generating a difference bit stream which is a difference between a base stream shared in advance with a decoding device and an input image; a communication control step of controlling a communication unit to transmit the difference bit stream generated in the difference generating step to the decoding device; the base stream is generated based on a pre-input picture, which has a quality that can be used as a reference picture when generating the differential bitstream, is an image that is highly similar to an input image from which a difference is generated, and is an image that is generated without using the input image; The communication control step controls the communication unit so as not to transmit a bit stream generated based on the pre-input picture to the decoding device, and to transmit the difference bit stream generated in the difference generating step to the decoding device. The encoding method.

4. An encoding method executed by an encoding device, comprising: a difference generating step of generating a difference bit stream which is a difference between a base stream shared in advance with a decoding device and an input image; a communication control step of controlling a communication unit to transmit the differential bit stream generated in the difference generating step to the decoding device; an adjustment step of adding identification information of the base stream to the difference bitstream, the base stream is generated based on a pre-input picture, which is an image quality that can be used as a reference image when generating the differential bitstream, has high similarity to an input image from which a difference is generated, and is generated without using the input image. The encoding method.

5. Computer, a difference generating unit that generates a difference bit stream that is a difference between a base stream shared in advance with a decoding device and an input image; a communication control unit that controls a communication unit to transmit the differential bit stream generated by the differential generation unit to the decoding device; the base stream is generated based on a pre-input picture, which has a quality that can be used as a reference picture when generating the differential bitstream, is an image that is highly similar to an input image from which a difference is generated, and is an image that is generated without using the input image; the communication control unit controls the communication unit not to transmit a bitstream generated based on the pre-input picture to the decoding device, and to transmit the difference bitstream generated by the difference generation unit to the decoding device. Encoding program.

6. A computer comprising: a difference generating unit that generates a difference bit stream that is a difference between a base stream shared in advance with a decoding device and an input image; a communication control unit that controls a communication unit to transmit the differential bit stream generated by the differential generation unit to the decoding device; an adjustment unit that adds identification information of the base stream to be referenced to the differential bitstream; the base stream is generated based on a pre-input picture, which is an image quality that can be used as a reference image when generating the differential bitstream, has high similarity to an input image from which a difference is generated, and is generated without using the input image. Encoding program.

7. a configuration adjustment unit that reconstructs a bit stream equivalent to the bit stream that was to be generated by the encoding device, using a base stream shared in advance with an encoding device and a difference bit stream that is a difference between the base stream and an input image and is transmitted from the encoding device; a decoding unit that executes a decoding process corresponding to an encoding process applied to the reconstructed bit stream, the base stream is generated based on a pre-input picture, which is an image quality that can be used as a reference image when generating the differential bitstream, has high similarity to an input image from which a difference is generated, and is generated without using the input image. Decoding device.

8. A decoding method executed by a decoding device, comprising: a configuration adjustment step of reconstructing a bitstream equivalent to the bitstream that was to be generated by the encoding device, using a base stream shared in advance with an encoding device and a difference bitstream that is a difference between the base stream and an input image and is transmitted from the encoding device; A decoding step of performing a decoding process corresponding to an encoding process performed on the reconstructed bitstream, the base stream is generated based on a pre-input picture, which is an image quality that can be used as a reference image when generating the differential bitstream, has high similarity to an input image from which a difference is generated, and is generated without using the input image. How to decode.

9. Computer, a configuration adjustment unit that reconstructs a bit stream equivalent to the bit stream that was to be generated by the encoding device, using a base stream shared in advance with an encoding device and a difference bit stream that is a difference between the base stream and an input image and is transmitted from the encoding device; a decoding unit that executes a decoding process corresponding to the encoding process applied to the reconstructed bit stream; the base stream is generated based on a pre-input picture, which is an image quality that can be used as a reference image when generating the differential bitstream, has high similarity to an input image from which a difference is generated, and is generated without using the input image. Decoding program.

10. A streaming system comprising an encoding device and a decoding device, The encoding device comprises: a difference generating unit that generates a difference bit stream that is a difference between a base stream shared in advance with the decoding device and an input image; a communication control unit that controls a communication unit to transmit the differential bit stream generated by the differential generation unit to the decoding device; The decoding device comprises: a configuration adjustment unit that reconstructs a bit stream equivalent to the bit stream that was intended to be generated by the encoding device, using the base stream and the differential bit stream transmitted from the encoding device; a decoding unit that executes a decoding process corresponding to an encoding process applied to the reconstructed bit stream, the base stream is generated based on a pre-input picture, which is an image quality that can be used as a reference image when generating the differential bitstream, has high similarity to an input image from which a difference is generated, and is generated without using the input image. Streaming system.

11. A streaming method executed in a streaming system including an encoding device and a decoding device, comprising: a difference generating step of generating a difference bit stream which is a difference between a base stream shared in advance between the encoding device and the decoding device and an input image; a communication control step of controlling a communication unit in the encoding device so as to transmit the differential bit stream generated in the difference generating step to the decoding device; a configuration adjustment step in which the decoding device reconstructs a bitstream equivalent to the bitstream that was intended to be generated by the encoding device, using the base stream and the differential bitstream transmitted from the encoding device; a decoding step in which the decoding device executes a decoding process corresponding to an encoding process applied to the reconstructed bitstream, the base stream is generated based on a pre-input picture, which is an image quality that can be used as a reference image when generating the differential bitstream, has high similarity to an input image from which a difference is generated, and is generated without using the input image. Streaming methods.

Citation Information

Patent Citations

  • Real time data reproduction device and system therefor

    JP1999150720A

  • Method for coding moving picture

    JP2003333602A

  • System and method for multimedia streaming service

    JP2005110244A

  • Image generation apparatus and image reproducing apparatus

    JP2010011154A

  • Stream transmission system, transmitting device, receiving device, and stream transmission method

    JP2014053949A