Compressed data communication system, compressed data communication method, and computer program

Abstract

To provide a compressed data communication system capable of preventing the omission of a frame when an error occurs, and reducing any influence on the band of communication.SOLUTION: A compressed data communication system includes first video image encoding means for encoding one frame in a first video image as an I frame and encoding one or more frames in the first video image as a P frame, second video image encoding means for encoding a frame in a second video image having a lower resolution than the first video image as the P frame in the first video image with reference to the I frame, and transmission means for transmitting data encoded by the first and second video image encoding means.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to a compressed data communication system, a compressed data communication method, a computer program, and the like. [Background technology]

[0002] In video distribution, video is transmitted by compressing the data volume using video coding methods such as MPEG (Moving Picture Experts Group) and H.264 / AVC (Advanced Video Coding). However, errors on the transmission path can cause information to be lost, and the video receiving side cannot reconstruct the video, resulting in a phenomenon known as frame dropping, where individual frames of the video are lost.

[0003] For example, in Patent Document 1, when a retransmission request is sent from the receiving side due to an error on the transmission path, the missing frame data is retransmitted and the image coding rate is reduced to ensure data capacity for retransmission.

[0004] In Patent Document 2, when an error on the transmission path is detected at the receiving end, a reference frame transmission request signal is sent, and the transmitting end encodes the next frame after receiving the request as the reference frame.

[0005] In Patent Document 3, a normal stream and a low bit-rate stream are multiplexed and transmitted. The receiving side decodes the two streams, removes frames with errors in the normal stream, and replaces them with the corresponding frames from the low bit-rate stream.

[0006] Furthermore, the Versatile Video Coding (VVC) coding method (hereinafter referred to as VVC) is known as a coding method for compressing and recording moving images. In VVC, a technique called Reference Picture Resampling (RPR) (hereinafter referred to as RPR) is introduced to improve coding efficiency.

[0007] RPR is a technology that allows an image with a different resolution from the image to be decoded to be used as a reference image, making it possible to change the resolution even in the case of inter-frame compression. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2001-268058 [Patent Document 2] Japanese Patent Application Laid-Open No. 2008-131143 [Patent Document 3] Special Publication No. 2010-514346 Summary of the Invention [Problem to be solved by the invention]

[0009] In the above Patent Document 1, when an error is detected on the transmission path and data is retransmitted, data of the same size as the original missing data is retransmitted, which increases the load on the transmission path and necessitates reducing the image coding rate to ensure sufficient data capacity for retransmission.

[0010] The above-mentioned Patent Document 2 has a problem in that all data of frames from the missing frame until the arrival of the next reference frame is discarded.

[0011] In the above-mentioned Patent Document 3, the receiving side continues to decode both the normal stream and the low bit rate stream, which allows frame replacement in the event of an error, but this increases the decoding load on the receiving side, and also poses the problem that when the received data is saved, the replaced frame portion must be re-encoded.

[0012] An object of the present invention is to provide a compressed data communication system that can prevent frame loss when an error occurs and has little effect on the communication band. [Means for solving the problem]

[0013] The compressed data communication system according to claim 1 comprises: a first video encoding means for encoding one frame in a first video as an I-frame and encoding one or more frames in the first video as P-frames; a second video encoding means for encoding a frame in a second video having a resolution lower than that of the first video as the P frame by referencing the I frame in the first video; a transmitting means for transmitting the data encoded by the first and second video encoding means; The present invention is characterized by having the following. [Effects of the Invention]

[0014] According to the present invention, it is possible to prevent frame loss when an error occurs, and to realize a compressed data communication system that has little effect on the communication band. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a functional block diagram illustrating an example of the configuration of a compressed data communication device 100 according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a functional block diagram illustrating an example of the configuration of an encoding unit 160 according to the first embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a stream configuration according to the first embodiment. [Figure 4] FIG. 2 is a diagram showing a filter coefficient table 1 according to the first embodiment. [Figure 5] FIG. 10 is a diagram showing a filter coefficient table 2 according to the first embodiment. [Figure 6] FIG. 3 is a diagram showing a filter coefficient table 3 according to the first embodiment. [Figure 7] 1(A) to 1(C) are simplified diagrams of a thinning method according to the first embodiment. [Figure 8] 10A and 10B are simplified diagrams of a method for thinning out one row according to the first embodiment. [Figure 9]10 is a flowchart showing an example of the processing flow of the coded stream generating unit 170 according to the first embodiment. [Figure 10] 10 is a flowchart showing an example of the processing flow of a retransmission processing unit 190 in the first embodiment. [Figure 11] 1A to 1D are diagrams showing examples of coded streams according to the first embodiment. [Figure 12] 10 is a flowchart showing an example of the processing flow of the coded stream generating unit 170 according to the second embodiment. [Figure 13] FIG. 13 is a functional block diagram illustrating an example of the configuration of a coded stream receiving device 1300 according to a second embodiment. [Figure 14] 13 is a flowchart showing an example of the processing flow of an error recovery processor 1350 according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. In each drawing, the same members or elements are designated by the same reference numerals, and duplicate descriptions will be omitted or simplified.

[0017] (Embodiment 1) Fig. 1 is a functional block diagram showing an example of the configuration of a compressed data communication device 100 according to a first embodiment of the present invention. Note that some of the functional blocks shown in Fig. 1 are realized by causing a CPU or the like serving as a computer included in the compressed data communication device 100 to execute a computer program stored in a memory serving as a storage medium.

[0018] However, some or all of these functions may be implemented by hardware, which may be a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP).

[0019] 1 may not be contained in the same housing, but may be configured as separate devices connected to each other via signal paths. The above explanation regarding FIG. 1 also applies to FIG. 2.

[0020] Compressed data communication device 100 and a compressed data receiving device constitute a compressed data communication system. Reference numeral 101 denotes a CPU as a computer. Reference numeral 102 denotes a ROM (non-volatile memory) such as an EEPROM or flash memory. Reference numeral 103 denotes a RAM (volatile memory) such as an SRAM or DRAM.

[0021] Computer programs for realizing the functions according to this embodiment and data used when executing the computer programs are stored in the ROM 102. These programs and data are appropriately loaded into the RAM 103 via the bus 110 under the control of the CPU 101 and executed by the CPU 101.

[0022] The imaging unit 120 includes a focus lens 121, a fixed lens 122, an aperture 123, and an image sensor 124. The focus lens 121 is moved along the optical axis by a lens driving unit 125. The aperture 123 is driven by the lens driving unit 125 to change the aperture size.

[0023] The image sensor 124 photoelectrically converts light that has passed through the focus lens 121 and the aperture 123 to generate an analog image signal. The generated analog image signal is subjected to sampling processing such as correlated double sampling, and then input to the camera signal processing unit 130.

[0024] The camera signal processing unit 130 converts the analog image signal into a digital image signal by A / D conversion, and then performs various digital image processing on the signal. The various digital image processing processes include offset processing, gamma correction processing, gain processing, RGB interpolation processing, noise reduction processing, contour correction processing, color correction processing, light source type determination processing, etc. The video signal after the digital image processing is stored in the RAM 103 via the bus 110.

[0025] The motor control unit 140 controls the lens driving unit 125. The video signal stored in the RAM 103 from the camera signal processing unit 130 undergoes resolution conversion in the scaling unit 150, and then undergoes compression processing by the encoding unit 160, generating a bitstream that is stored in the RAM 103 via the bus 110. In this embodiment, the image compression by the encoding unit 160 is performed based on the VVC standard, but is not limited to this.

[0026] The coded stream generating unit 170 controls whether the compression process in the coding unit 160 is to be an intra-frame process in which no inter-frame reference is made, or an inter-frame process in which inter-frame reference is made.

[0027] The IP communication unit 180 is connected to a network 181 via a LAN. The bit stream stored in the RAM 103 is distributed from the IP communication unit 180 through the network 181 to a compressed data receiving device.

[0028] When an error occurs in the bit stream distributed by the IP communication unit 180, the retransmission processing unit 190 retransmits a part of the bit stream stored in the RAM 103 to the compressed data receiving device via the IP communication unit 180 in response to a retransmission request sent from the compressed data receiving device.

[0029] 2 is a functional block diagram showing an example of the configuration of the encoding unit 160 according to embodiment 1. Reference numeral 200 denotes an image analysis unit that analyzes the angle of view value of an input frame and outputs the analysis results as image analysis information, and also outputs tile images that are combined with tile information for dividing the image into spatial regions based on image characteristics and external inputs.

[0030] Reference numeral 210 denotes an RPR control information generator, which generates information on the scaling ratio and offset position of a motion vector required for decoding using RPR (Reference Picture Resampling).

[0031] A prediction unit 220 performs intra-frame prediction, or inter-frame prediction, on tile-based image data to generate predicted image data. The prediction unit 220 also calculates and outputs a prediction error from the input image data and the predicted image data. It also outputs prediction information (such as a prediction mode and motion vectors) necessary for prediction, along with the prediction error.

[0032] A transform / quantization unit 230 performs orthogonal transform on the prediction errors output from the prediction unit 220 in units of blocks to calculate transform coefficients, performs quantization, and calculates quantized coefficients. A dequantization / inverse transform unit 231 dequantizes the quantized coefficients output from the transform / quantization unit 230 to regenerate transform coefficients, and further performs inverse orthogonal transform on them to regenerate prediction errors.

[0033] Reference numeral 250 denotes a frame memory that stores reconstructed image data. Reference numeral 240 denotes an image reconstruction unit that generates predicted image data by appropriately referencing the frame memory 250 based on the prediction information output from the prediction unit 220, and generates and outputs reconstructed image data from the input prediction error and predicted image data.

[0034] An in-loop filter unit 251 performs in-loop filtering such as deblocking filtering and sample adaptive offset on the reconstructed image, and outputs the filtered image.

[0035] An entropy coding unit 260 encodes the quantization coefficients output from the transform / quantization unit 230 and the prediction information output from the prediction unit 220 to generate and output coded data.

[0036] A bitstream generation unit 270 generates header code data by encoding the outputs from the image analysis unit 200 and the RPR control information generation unit 210. Furthermore, the bitstream generation unit 270 combines the header code data with the code data output from the entropy encoding unit 260 to generate and output a bitstream.

[0037] Next, the image encoding operation in the encoding unit 160 will be described below. In this embodiment, moving image data is input in frame units. The image analysis unit 200 receives image data for one frame and calculates a field of view change value. When an arbitrary frame is set as a reference frame, the field of view change value is the ratio of the field of view of the reference frame to the frame to be encoded.

[0038] Next, the RPR control information generator 210 signals by setting sps_ref_pic_resampling_enabled_flag of the SPS (Sequence Parameter Set) to 1 to indicate that RPR is to be used.

[0039] The number of pixels in the input frame is also calculated, and the number of pixels in the vertical direction and the horizontal direction for luminance are stored as pps_pic_width_in_luma_samples and pps_pic_height_in_luma_samples of the PPS (Picture Parameter Set), respectively.

[0040] The prediction unit 220 cuts the image data input from the image analysis unit 200 into a plurality of blocks and performs prediction processing on a block-by-block basis. As a result of the prediction processing, prediction errors are generated and input to the transformation and quantization unit 230. The prediction unit 220 also generates prediction information and outputs it to the image reproduction unit 240 and the entropy coding unit 260.

[0041] Here, we will explain in more detail the prediction process executed by the prediction unit 220 and the prediction information output from the prediction unit 220. In image coding techniques such as VVC, in order to reduce the amount of data in the coded bitstream while maintaining the image quality of the reproduced image, a prediction process is performed in which pixels of a coded block are used to predict pixels of the block to be coded.

[0042] Prediction processes include intra-prediction, which uses pixels from blocks that have already been coded within the same frame, and inter-prediction, which uses pixels from blocks in different coded frames. VVC also standardizes a technology called RPR, which allows decoding even when the resolution of the coded frame to be referenced differs from the resolution of the frame to be coded.

[0043] Here, as an explanation of RPR, inter prediction in the case where the resolution of a reference encoded frame and the resolution of a current encoding frame are different will be further explained.

[0044] 3 is a diagram showing an example of a stream configuration in the first embodiment, showing an example where the resolution is changed midway through the stream using RPR. Frames 301 to 306 are arranged in chronological order. Frame 301 is an I frame consisting only of intra-references, and frames 302 to 306 are P frames that include a reference to the previous frame. Here, frames 301 to 303 make up the first video.

[0045] At the timing of frames 303 to 304, the image is reduced to two-thirds vertically and horizontally. In terms of specific pixel count, for example, frames 301 to 303 are 1920 x 1080 pixels, while frames 304 to 306 are 1280 x 720 pixels. Here, frames 304 to 306 make up the second video.

[0046] In locations where a change in resolution occurs, the reference encoded frame is scaled (enlarged or reduced) to the same resolution as the frame to be encoded, and inter prediction is performed. An example of a method for scaling the reference encoded frame will be described below. For simplicity, only luminance values ​​will be described. A description of chrominance will be omitted, as it can be performed in a similar manner taking into account the number of samples.

[0047] (Step 1) The scaling ratios in the vertical and horizontal directions are calculated as scalingRatio[0] and scalingRatio[1], respectively. Hereinafter, scalingRatio[0] and scalingRatio[1] will be collectively referred to as scalingRatio[x].

[0048] The scalingRatio[x] is determined by the ratio between the size of the scaling window of the reference encoded frame and the scaling window of the encoding target frame. In this embodiment, the RPR control information generator 210 acquires the above ratio as RPR control information.

[0049] (Step 2) Next, an interpolation filter to be used for scaling is determined. That is, for example, coefficients of the interpolation filter are selected depending on the value of scalingRatio[x]. Fig. 4 shows filter coefficient table 1 of the first embodiment, Fig. 5 shows filter coefficient table 2 of the first embodiment, and Fig. 6 shows filter coefficient table 3 of the first embodiment.

[0050] If scalingRatio[x] is 1.75 times or more, the coefficients of the interpolation filter are used, for example, as shown in Table 1 in Figure 4. If scalingRatio[x] is less than 1.75 times but is 1.25 times or more, the coefficients are used, for example, as shown in Table 2 in Figure 5. If scalingRatio[x] is less than 1.25 times, the coefficients are used, for example, as shown in Table 3 in Figure 6.

[0051] The coefficients of the interpolation filter in each table are determined by the sample position p to be calculated. Tables 1 to 3 in Figures 4 to 6 are examples of filters defined for luminance expansion / contraction in the VVC standard. Table 1 is created based on the cutoff frequency when the ratio is 2x, and Table 2 is created based on the cutoff frequency when the ratio is 1.5x.

[0052] p is an integer ranging from 0 to 15, and is the numerator value when the smallest sample unit is divided into 1 / 16th units. For example, if sample point A and sample point A+1 are divided into 16, the filter coefficients for the third sample point (A+3 / 16, p=3) are fL[3][i]=[-4, -1, 16, 29, 23, 7, -4, -2], referring to Table 1.

[0053] Hereinafter, when the filter coefficients obtained in this way are used in the horizontal direction, they will be written as fLH[p][i] (=fL[p][i]), and when they are used in the vertical direction, they will be written as fLV[p][i] (=fL[p][i]).

[0054] (Step 3) Next, the reference image generated from the target encoding frame is resampled according to scalingRatio[x] so that it has the same resolution as the target encoding frame. For example, the position of each pixel when the target encoding frame is scaled by scalingRatio[x] is calculated with 1 / 16 pixel accuracy.

[0055] Also, the reference image is interpolated 16 times using the filter obtained in step 2. For example, if scalingRatio[x]≧1.25, a new sampling point (x3+px, y3+py) is obtained using the following equations 1 and 2.

[0056] Here, we use the notation that the coordinates of the target pixel in the reference image are (xi, yi), the coordinates of the adjacent pixel to the left are (x(i-1), yi), and the coordinates of the adjacent pixel below are (xi, y(i-1)).

[0057] Also, px and py are integers modulo 16 (divisor), and are indices of coordinates obtained by dividing the coordinates of adjacent pixels in the horizontal and vertical directions by 16, respectively, and L(x, y) represents the luminance value of coordinates (x, y). a is a normalization constant.

[0058]

number

[0059]

number

[0060] Here, fLH[p][i] and fLV[p][i] are generated from Table 2. Furthermore, yn is the coordinate value from y0 to y7. The samples generated in this manner are called upsampled images. This upsampled image is thinned out, and resampled from the reference image to the same resolution as the frame to be coded, and then a reference image is generated.

[0061] 7(A) to 7(C) are simplified diagrams of a thinning method according to embodiment 1, and Figures 8(A) and 8(B) are simplified diagrams of a thinning method for one line according to embodiment 1. The thinning method is the same for both the vertical and horizontal directions, so for simplicity, only the horizontal direction will be described.

[0062] In Fig. 7(A), 701 is a reference image, and in Fig. 7(B), 702 is a frame to be coded, which is composed of pixel blocks such as 703. Note that pixel values ​​are defined for each pixel block. In this case, the reference image 701 is a 6x6 image, and the frame to be coded 702 is a 4x4 image.

[0063] The origin is the upper left vertex of the entire image, and the coordinates of each pixel value are the coordinates of the upper left vertex of the pixel block. Here, the coordinate values ​​of each pixel in the encoding frame are multiplied by scalingRatio[x] (2 / 3 in the example of Figure 7).

[0064] Then, for the x coordinate of the encoding target frame 702 being (0, 1, 2, 3), the coordinate values ​​(0, 3 / 2, 6 / 2, 9 / 2) can be calculated and stored in, for example, a coordinate array H[x].

[0065] 7C, 704 denotes an image obtained by resampling the reference image to the number of pixels of the frame to be encoded. The pixel values ​​of each pixel block of the resampled image 704 may be constructed using pixel values ​​from the upsampled image that correspond to the coordinates of the enlarged image.

[0066] An example of the construction method will now be described with reference to Figure 8. 801 in Figure 8(A) shows only the pixel block at y=0 in the resampled image 704. Similarly, 804 in Figure 8(B) shows only the pixel block at y=0 in the reference image 701.

[0067] Here, the luminance value of the reference image 701 is Y[x], and the luminance value of the resampled image 704 is Y'[x]. Note that x is the x-coordinate value of the luminance value. Then, the luminance value of the coordinate position x of the resampled image 704 can be obtained by finding the coordinate position of the reference image 701 with reference to H[x] in the coordinate array 803. When only the x-coordinate is considered, it can be written as in Equation 3 below.

[0068]

number

[0069] For example, the coordinate of the desired brightness value for pixel block 802 is 1, so set x=1, find coordinate array H[1]=3 / 2, and obtain the value of Y[3 / 2]. 805 is a diagram that clearly shows the sampling points obtained by interpolating the coordinate values ​​1 and 2 of 804 using the filter described above.

[0070] Since the reference image has been interpolated in advance in 1 / 16 increments as shown in 805, it is sufficient to obtain the luminance value of the coordinate corresponding to 3 / 2 (=1 + 8 / 16) of 806. In this way, the reference image after resampling at the same resolution can be constructed by thinning out the pixel values ​​of the upsampled reference image except for those corresponding to the enlarged pixel positions of the frame to be coded.

[0071] Inter-prediction is a process in which, when the pixels of an encoded frame or the number of pixels of an encoded frame differ from the number of pixels of the frame to be encoded, pixels of the block to be encoded are predicted by referring to the pixels of the resampled image constructed using the above method.

[0072] For simplicity, the coded frame and the resampled image are collectively referred to as a target image for inter prediction. For example, if there is no motion between the coded frame to be referenced and the target image for inter prediction, pixels of the target block for coding are predicted using pixels at the same positions in the target image for inter prediction.

[0073] In such a case, the prediction information includes a motion vector (0, 0), which indicates no motion. On the other hand, if there is motion between frames for the block to be coded, the prediction information includes the motion vector (MVx, MVy).

[0074] Figure 9 is a flowchart showing an example of the processing flow of the encoded stream generation unit 170 according to the first embodiment, Figure 10 is a flowchart showing an example of the processing flow of the retransmission processing unit 190 according to the first embodiment, and Figures 11(A) to (D) are figures showing examples of encoded streams according to the first embodiment.

[0075] The operations of the steps in the flowcharts of FIGS. 9 and 10 are performed sequentially by a CPU or the like serving as a computer within the compressed data communication device 100 executing a computer program stored in memory.

[0076] Next, the compressed data communication method according to this embodiment will be described using the flowcharts of FIGS. 9 and 10 and the coded stream examples of FIGS. 11(A) to 11(D).

[0077] In step S900 of Fig. 9, the process branches depending on whether the next frame is an I frame or a P frame. If it is an I frame, in step S910, the encoding unit 160 compresses the image, and the compressed image is temporarily stored in RAM 103, and then in step S920, it is transmitted from the IP communication unit 180. This corresponds to I1 and I2 in Fig. 11(A).

[0078] If it is determined in step S900 to be a P frame, in step S930, the encoding unit 160 compresses the image using the difference from the immediately preceding frame, and the compressed image is temporarily stored in RAM 103. Thereafter, in step S940, it is transmitted from the IP communication unit 180. This corresponds to P1 to P6 in FIG. 11(A). The circular arrows indicate how the difference is used by referring to the immediately preceding frame.

[0079] Steps S900 to S940 function as a first video encoding step (first video encoding means) that encodes one frame in the first video as an I frame and encodes one or more frames in the first video as P frames.

[0080] In step S950, the scaling unit 150 creates a reduced image, and then in step S960, the encoding unit 160 compresses the image using the difference from the most recent I-frame. The frame reference here uses the RPR described above. P1' to P6' in Figure 11(B) correspond to the processing in step S960, and the frame reference is indicated by the straight arrows.

[0081] Here, step S960 functions as a second video encoding step (second video encoding means) that encodes a frame in a second video having a lower resolution than the first video as a P frame that references an I frame in the first video.

[0082] The compressed data created in step S960 is not immediately transmitted but is stored in RAM 103.

[0083] In step S970, it is determined whether the series of compression processes has ended, and if it is still ongoing, the process returns to step S900.

[0084] When the data in Figure 11(A) explained in the flow of Figure 9 is transmitted, for example, an error occurs in the frame transmission of P3, as shown in Figure 11(C). Because an error occurs in P3, an error also occurs in P4, which references P3. Here, a retransmission request for P3 is sent from the receiving side to the transmitting side.

[0085] 10 shows an example of the flow of processing performed by retransmission processing unit 190 when a retransmission request is received. In step S1000, it is determined whether the retransmission request occurred in an I frame or a P frame.

[0086] If it is an I frame, in step S1010, the I frame data stored in RAM 103 is retransmitted from IP communication unit 180. If it is a P frame, in step S1020, the low-resolution P' data created in step S960 is read from RAM 103 along with the P frame in which the error occurred, and transmitted from IP communication unit 180.

[0087] Next, in step S1030, the subsequent frame is checked, and if it is a P frame, the process returns to step S1020 to continue transmitting the corresponding P' data.

[0088] Here, steps S1000 to S1030 function as a transmitting step (transmitting means) for transmitting data encoded by the first and second video encoding means. In this manner, in this embodiment, when an error occurs in transmitting a P frame in the first video and a retransmission request is received, the P frame in the second video is transmitted.

[0089] Returning to the example of Figure 11, P3 and P4 in Figure 11(C), which were lost due to a transmission error, are interpolated by transmitting P3' and P4' as shown in Figure 11(D) after a retransmission request. In this embodiment, P3' and P4' use I1 as the reference frame, so they can be directly replaced with P3 and P4, and can also be stored without re-encoding. Furthermore, because the resolution is low, the amount of data is small, and the communication required for retransmission has little impact on the bandwidth.

[0090] That is, according to this embodiment, when an error occurs on the transmission path, it is possible to replace the frames up to the next I frame with lower resolution images, preventing frames from being completely dropped. It is also possible to save the replaced portion as a continuous stream without re-encoding it.

[0091] In addition, because low-resolution data is retransmitted only when a frame drop occurs and a retransmission request is received, increases in traffic can be kept to a minimum. Also, because low-resolution images are used for interpolation, there is little impact on communication bandwidth.

[0092] Note that the interpolated low-resolution frames (P3', P4', etc.) may be temporally distant from the reference I-frame, so depending on the decoder implementation, there is a risk that the decoded I-frame results may be discarded rather than retained.

[0093] To prevent this discarding, when creating compressed data in step S930, not only the immediately preceding P frame but also the immediately preceding I frame may be defined as a dummy reference frame. That is, when encoding a P frame in the first video by the first video encoding means, the immediately preceding P frame and the immediately preceding I frame may be referenced.

[0094] In this case, multiple frames are defined as reference frames, so they may be treated as B frames. Alternatively, a Long Term Reference frame may be specified in the header of the P frame.

[0095] (Embodiment 2) In the first embodiment, a low-resolution P frame is transmitted only during retransmission when an error occurs in communication, but in the second embodiment, a low-resolution P' frame is always transmitted. Note that explanations of figures common to the first embodiment will be omitted.

[0096] Fig. 12 is a flowchart showing an example of the processing flow of the coded stream generation unit 170 according to embodiment 2. Note that the operation of each step in the flowchart of Fig. 12 is performed sequentially by a CPU or the like serving as a computer in the compressed data communication device 100 executing a computer program stored in a memory.

[0097] In the flowchart of Fig. 12, steps with the same reference numerals as those in Fig. 9 perform the same operations, and therefore description thereof will be omitted. In Fig. 12, step S1200 is inserted between step S950 and step S960 in Fig. 9.

[0098] In step S1200, the low-resolution data P' created in step S950 and stored in RAM 103 is read and transmitted from IP communication unit 180. That is, in the second embodiment, the second video is also transmitted when the encoded first video is transmitted.

[0099] Fig. 13 is a functional block diagram showing an example of the configuration of a coded stream receiving device 1300 according to embodiment 2. Note that some of the functional blocks shown in Fig. 13 are realized by causing a CPU 1310 or the like serving as a computer included in the coded stream receiving device 1300 to execute a computer program stored in a memory serving as a storage medium.

[0100] However, some or all of these functions may be implemented by hardware. Examples of hardware that can be used include dedicated circuits (ASICs) and processors (reconfigurable processors, DSPs). Furthermore, the functional blocks shown in Fig. 13 do not have to be built into the same housing, and may be configured as separate devices connected to each other via signal paths.

[0101] The coded stream receiving device 1300 functions as a compressed data receiving device. The coded stream receiving device 1300 as a compressed data receiving device constitutes a compressed data communication system together with the compressed data communication device 100.

[0102] In FIG. 13, 1310 is a CPU, 1311 is a ROM (non-volatile memory) such as an EEPROM or flash memory, and 1312 is a RAM (volatile memory) such as an SRAM or DRAM.

[0103] Computer programs for realizing the functions according to the second embodiment and data used when the computer programs are executed are stored in a ROM 1311. These computer programs and data are loaded into a RAM 1312 via a bus 1301 as needed under the control of a CPU 1310, and are executed by the CPU 1310.

[0104] The IP communication unit 1320 is connected to a network 1321 via a LAN. A bit stream delivered from the network 1321 is stored in the RAM 1312 through the IP communication unit 1320. The IP communication unit 1320 functions as a receiving means for receiving the first video and the second video transmitted from the IP communication unit 180, which serves as a transmitting means of the compressed data communication device 100.

[0105] The bitstream stored in the RAM 1312 is decoded by a decoding unit 1330, enlarged or reduced by a scaling unit 1340, and then displayed on an external display 1361 via a display unit 1360. If a communication error is detected in the IP communication unit 1320, an error recovery process is performed by an error recovery processing unit 1350, and the result is stored in the RAM 1312.

[0106] Fig. 14 is a flowchart showing an example of the processing flow of the error recovery processor 1350 of embodiment 2. Note that the operation of each step in the flowchart of Fig. 14 is performed sequentially by a CPU or the like serving as a computer in the coded stream receiving device 1300 executing a computer program stored in a memory.

[0107] 14 shows an example of the flow of error recovery processing performed by error recovery processor 1350 when a communication error is detected in IP communication unit 1320. In step S1400, it is determined whether the frame in which the error occurred is an I frame or a P frame. If it is an I frame, in step S1410, IP communication unit 180 sends a retransmission request for the I frame data to the transmitting side.

[0108] If it is a P frame, in step S1420, an operation of replacing the P frame in which the error occurred with lower resolution P' data corresponding to the P frame in which the error occurred is executed on RAM 1312. Step S1420 functions as an error recovery processing step (error recovery processing means) that, if an error is detected in the transmission of a P frame in the first video, replaces the P frame in which the error was detected with a P frame in the second video.

[0109] Next, in step S1430, the next frame is checked, and if it is a P frame, the process returns to step S1420 to continue replacing it with the corresponding P' data. This interpolated data can also be saved without re-encoding.

[0110] As described above, in the second embodiment, a bit stream at a lower resolution is transmitted in advance at the same time as the bit stream at the original resolution is transmitted, so that if an error occurs during transmission of a P frame, it is possible to interpolate with data at a lower resolution without delay.

[0111] The present invention has been described above in detail based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications and combinations of the above embodiments are possible based on the spirit of the present invention, and these are not excluded from the scope of the present invention.

[0112] The present invention also includes those that realize the functions of the above embodiments using, for example, at least one processor such as a CPU, memory, or circuit (for example, ASIC). Also, multiple processors may be used to perform distributed processing.

[0113] In order to realize part or all of the control in the above embodiments, a computer program that realizes the functions of the above embodiments may be supplied to a compressed data communication system or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the compressed data communication system or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. The present invention also includes the following combinations.

[0114] (Configuration 1) A compressed data communication system comprising: a first video encoding means for encoding one frame in a first video as an I frame and encoding one or more frames in the first video as P frames; a second video encoding means for encoding frames in a second video having a lower resolution than the first video as P frames that reference the I frames in the first video; and a transmission means for transmitting data encoded by the first and second video encoding means.

[0115] (Configuration 2) The compressed data communication system according to Configuration 1, wherein the transmitting means transmits the P frame in the second video when an error occurs in transmitting the P frame in the first video and a retransmission request is received.

[0116] (Configuration 3) The compressed data communication system according to configuration 1 or 2, wherein the transmitting means transmits the second video when transmitting the encoded first video.

[0117] (Configuration 4) A compressed data communication system according to any one of configurations 1 to 3, characterized in that the first video encoding means, when encoding the P frame in the first video, refers to the immediately preceding P frame and the most recent I frame.

[0118] (Configuration 5) A compressed data communication system according to configuration 4, characterized in that when referring to the immediately preceding P frame and the most recent I frame, the P frame is treated as a B frame or a Long Term Reference frame is specified in the header of the P frame.

[0119] (Configuration 6) A receiving means for receiving the first video and the second video transmitted from the transmitting means; and an error recovery processing means for, when an error is detected in the transmission of the P frame in the first video, replacing the P frame in which the error was detected with the P frame in the second video.

[0120] (Method) A compressed data communication method comprising: a first video encoding step of encoding one frame in a first video as an I frame and encoding one or more frames in the first video as P frames; a second video encoding step of encoding frames in a second video having a lower resolution than the first video as P frames that reference the I frames in the first video; and a transmission step of transmitting data encoded by the first and second video encoding means.

[0121] (Program) A computer program for controlling each means of the compressed data communication system according to any one of configurations 1 to 6 by a computer. [Explanation of symbols]

[0122] 100: Compressed data communication device 150: Zoom section 160: Encoding section 170: Encoded stream generation unit 190: Retransmission processing unit

Claims

1. a first video encoding means for encoding one frame in a first video as an I frame and encoding one or more frames in the first video as P frames; a second video encoding means for encoding a frame in a second video having a resolution lower than that of the first video as the P frame by referencing the I frame in the first video; a transmitting means for transmitting the data encoded by the first and second video encoding means; 1. A compressed data communication system comprising:

2. 2. The compressed data communication system according to claim 1, wherein the transmitting means transmits the P frame in the second video when an error occurs in transmitting the P frame in the first video and a retransmission request is received.

3. 2. The compressed data communication system according to claim 1, wherein said transmitting means transmits said second video when transmitting said encoded first video.

4. 2. The compressed data communication system according to claim 1, wherein the first video encoding means refers to the immediately preceding P frame and the immediately preceding I frame when encoding the P frame in the first video.

5. 5. The compressed data communication system according to claim 4, wherein when referring to the immediately preceding P frame and the immediately preceding I frame, the P frame is treated as a B frame or a Long Term Reference frame is specified in the header of the P frame.

6. a receiving means for receiving the first image and the second image transmitted from the transmitting means; 2. The compressed data communication system according to claim 1, further comprising: an error recovery processing means for, when an error is detected in the transmission of the P frame in the first video, replacing the P frame in which the error was detected with the P frame in the second video.

7. a first video encoding step of encoding one frame in the first video as an I frame and encoding one or more frames in the first video as P frames; a second video encoding step of encoding a frame in a second video having a resolution smaller than that of the first video as the P frame referring to the I frame in the first video; a transmitting step of transmitting the data encoded by the first and second video encoding means; A compressed data communication method comprising:

8. A computer program for controlling each means of the compressed data communication system according to any one of claims 1 to 6 by a computer.

Images