Method for analyzing changes in a video stream for bitrate-constrained, low-latency video coding, and associated device

Abstract

The invention relates to a method for analyzing the change between two frames of an uncompressed video stream (201), comprising estimating a shift and a statistical deviation between all or part of said frames in at least one direction, and using the calculated shift(s) and statistical deviation(s) to detect (301) a significant change between said frames. It also includes a method for encoding an uncompressed video stream, wherein the information relating to a significant change is used to adapt the uncompressed video stream or the encoding parameters (203) of the uncompressed video stream, in order to avoid bitrate spikes.The invention also relates to a device implementing one or the other of the methods, a complete transmission system (300), a computer program product comprising program instructions for executing the steps of the methods according to the invention, and a recording medium comprising the computer program product. Figure for the abstract: Fig. 3.

Description

Title of the invention: Method for analyzing changes in a video stream for bitrate-constrained, low-latency video coding, and associated device technical field

[0001] The invention is in the field of video coding, and relates more particularly to the problem of transmitting compressed videos over communication links with constrained bandwidths and low latency requirements.

[0002] A typical application of the invention is remote driving, that is, the remote operation of any type of mobile device: car, train, boat, drone, robot, balloon, etc. In these critical applications, images of the device's environment must be transmitted to a remote operator with the lowest possible latency. Depending on the type of mobile device, its speed, or its distance, the communication capabilities of the transmission link used to transmit a video stream to remote equipment may be limited.

[0003] However, the invention applies generally to any type of application where a video is transmitted over a communication link with a limited bandwidth, and where latency must be controlled. Previous technique

[0004] Video compression reduces the amount of information in a video stream that needs to be exchanged between a transmitting and a receiving device. Current standards, such as AVC (Advanced Video Coding), HEVC (High Efficiency Video Coding), and many others, use predictive transmission schemes. The frames of a video stream are divided into blocks, and these blocks are predicted from other blocks belonging either to the current frame or to previously encoded / decoded frames. The residuals, i.e., the residual differences between the predicted blocks and the blocks to be transmitted, are transmitted after encoding, for example, by a discrete cosine transform (DCT), and quantization.Thus, prediction makes it possible to take advantage of the temporal continuity between images in a video stream, where the differences between two successive images generally consist of small image shifts, or of the frequency continuity within a video image, where some parts can be made up of identical blocks, to limit the amount of data to be transmitted.

[0005] In predictive encoders, compression uses reference frames and predicted frames. Reference frames (I-frames) are keyframes encoded independently of other frames in the video stream. This is called intra-frame prediction, or intra-frame prediction. The blocks of the reference frames are predicted with respect to other previously encoded blocks of the same frame, thus exploiting the frequency redundancy of the blocks.

[0006] The compression also uses images predicted by inter-image prediction, or P images (in English, Predicted). These images are predicted from the reference images (I images) and / or other predicted images (P images), thus exploiting the temporal redundancy of the blocks.

[0007] Other types of images exist, such as bidirectional images (B-images), which use inter-prediction with respect to images prior to and / or subsequent to the current image, or IDR (Instantaneous Decoder Refresh) images, which use intra-prediction and serve to clear the encoder's memory to prevent the propagation of prediction errors. Although described hereafter only in relation to I-images and P-images, the invention applies regardless of the frame configuration of the predictive encoder used.

[0008] Figure 1a illustrates the principles of predictive coding for images in a video stream. Successive frames are grouped into subsets called GOPs (Group Of Pictures). GOPs generally begin with an I frame, followed by a series of predicted P frames. Many GOP schemes are possible, as video compression standards can generally switch between several transmission schemes depending on the type of video stream to be compressed.

[0009] Prediction makes it possible to significantly reduce the amount of data required to transmit a block. In [Fig. 1a], the heights of the frame representations are representative of the size of the encoded frame. Since P frames use inter-frame prediction, their size is significantly smaller than that of I frames, which use intra-frame prediction.

[0010] The size of the I and P frames is also related to the quantization level of the residuals. The quantization levels are chosen according to the target bitrates for the compressed video. In the context of a bitrate-constrained link, quantization factors are sized so that the bitrate of the compressed video is as close as possible to the bitrate capacity of the link over which the compressed video is transmitted, in order to optimize the quality of the transmitted video stream. Since the compression ratio of a video stream is uncertain, some video encoders have the ability to dynamically adjust the quantization values ​​of the video streams.

[0011] A problem arises when an abrupt scene change occurs in the video stream, upstream of a frame predicted by prediction relative to a previous frame (P or B frame). Such a change can occur, for example, in the case of remote control of a vehicle entering a tunnel, in the case of an abrupt change in the orientation of the camera acquiring the video stream, in the case of a camera change, etc. Temporal prediction is then no longer effective on the frame following the scene change. Hereafter, a significant change is defined as a change between two frames that renders inter-frame prediction suboptimal, or a change sufficient to cause an increase in the instantaneous bit rate of the encoded stream exceeding an acceptable threshold, this threshold being related to the transmission link bit rate and the permissible latency.

[0012] Fig.lb illustrates the impact of a significant change in the predictive coding of Fig.la for a state-of-the-art coder not dynamically adjusting the bitrate of the compressed video stream.

[0013] In [Fig. 1b], the change occurs at time 101, upstream of a frame P. Due to the scene change, the current frame P is unable to identify suitable blocks for inter-frame prediction in the previously encoded images. Thus, the amount of residual blocks to be transmitted increases significantly, so that the image P can be much larger than that of an image I when the change involves a significant portion of the image. The instantaneous bit rate of the compressed video then increases, which, when the communication link has a limited bit rate, results in an increase in the amount of data stored in the buffers of the equipment providing the data link, and generates an increase in latency on the communication link. If the buffers of the transmission equipment are limited, this phenomenon can lead to memory saturation and data loss.To avoid increased transmission latency and buffer saturation, it is possible to remove portions of the video stream to be transmitted during its broadcast, but this causes interruptions in the video stream which impair its visual quality.

[0014] If the current frame is an I frame, using intra prediction, the significant change 101 has no effect on the instantaneous rate of the video stream.

[0015] Patent application EP 3,843,409 A1I describes a method for absorbing instantaneous data rate peaks during the transmission of a video stream over a limited-rate link. This solution consists of monitoring variations in the buffer level storing the video stream before its transmission, and when the data rate increases, duplicating one or more frames in the video stream. The duplicated frames are encoded very efficiently by inter-frame coding, which causes a decrease in the instantaneous data rate to compensate for the peak. However, this solution has the drawback to operate a posteriori in response to an increase in instantaneous throughput and requires a feedback loop between the compressed video stream transmission equipment and the video encoder.

[0016] Figure 1a illustrates the impact of a significant change on the predictive coding of Figure 1a for a state-of-the-art encoder that dynamically adjusts the bitrate of the compressed video stream. In the event of a peak in instantaneous bitrate, such an encoder adjusts the quantization factors of subsequent frames to achieve a target bitrate. In this case, following the significant scene change occurring at 101, the size of the frame P following the change increases, leading to a substantial increase in the instantaneous bitrate. To meet the bitrate constraint, the encoder adjusts the quantization factor of the following frames. This mechanism results in a degradation of the quality of the transmitted video stream after the change.

[0017] Finally, there are recursive encoders, which re-encode the video stream with different encoding parameters when the maximum target bitrate is exceeded, in order to guarantee compliance with a bitrate constraint. However, this mechanism is very computationally expensive since the video stream may have to be encoded several times, and is poorly suited for live streaming.

[0018] There is therefore a need for a simple method to identify significant scene changes between frames of an uncompressed video stream, in order to take this change into account when encoding the video stream, particularly in the context of broadcasting the compressed video stream with a constrained bitrate and low latency requirement. Thus, the video encoder is able to adapt its encoding to take into account the abrupt image variation detected. Summary of the invention

[0019] To this end, the present invention describes a method for analyzing the change between two images of an uncompressed video stream, comprising estimating an offset and a statistical deviation between all or part of said images in at least one direction, and using the offset(s) and statistical deviation(s) calculated to detect a significant change between said images.

[0020] According to one embodiment of the method for analyzing the change between two images of an uncompressed video stream according to the invention, the estimation of a statistical deviation and an offset between all or part of the images in at least one direction comprises, for each direction considered: - a first step of applying a Mojette transform to each of the images in order to determine, for each image, a vector corresponding to a projection of the image in the direction considered, - a second step of calculating a cross-correlation between the vectors obtained by applying a Mojette transform to the two images in said direction during the first step of the process, then measuring a shift from the results of the cross-correlations, - a third step of measuring a statistical difference between the vectors obtained by applying a Mojette transform to the two images in the direction considered during the first step of the process, after compensation of the associated offset calculated during the second step of the process.

[0021] According to different possible embodiments of the method for analyzing the change between two images of an uncompressed video stream according to the invention, said at least one direction may include one or more directions chosen from a horizontal direction and a vertical direction.

[0022] According to one embodiment of the method for analyzing the change between two images of an uncompressed video stream according to the invention, the use of the offset(s) and the statistical deviation(s) calculated to detect a significant change between said images comprises: - the comparison of the offset(s) or a combination of offsets with one or more thresholds, and - the comparison of the statistical deviation(s) or a combination of statistical deviations with one or more thresholds.

[0023] Advantageously, at least one of the shift(s) or combination of shifts and the statistical deviation(s) or combination of statistical deviations are compared with a plurality of thresholds in order to quantify the magnitude of said change.

[0024] The invention also relates to a method of encoding an uncompressed video stream composed of a succession of images, comprising, for at least one pair of images among said image succession, a step of analyzing the change between two images of an uncompressed video stream as described above, and a predictive coding step of the uncompressed video stream in which, when a significant change is detected during the change analysis step, the uncompressed video stream or coding parameters of the video stream are modified so as to mitigate an increase in the bit rate of the video stream after predictive coding related to said significant change.

[0025] According to one embodiment, the adaptation of the coding parameters comprises one or more of the following elements: - modify a quantization step of predictive video coding, - reduce a bitrate instruction applied to predictive video coding, - use an intra-image coding mode, - start a new image grouping (GOP), - replace the oldest image with the newest image.

[0026] According to a particular embodiment of the method for encoding an uncompressed video stream according to the invention, the images of said image pair are successive images of the uncompressed video stream, and the modification of the uncompressed video stream includes copying the older image in place of the newer image.

[0027] The invention also relates to a device comprising computing means configured to implement a method according to the invention, and to a video stream transmission system comprising: - means of acquiring an uncompressed video stream, - computing means configured to implement a method for encoding an uncompressed video stream according to an embodiment of the invention on the uncompressed video stream to obtain a compressed video stream, - means for transmitting the compressed video stream, - means for acquiring the compressed video stream, - means for decoding the compressed video stream to obtain an uncompressed video stream, and - means of displaying the decompressed video stream.

[0028] It also relates to a computer program comprising program code instructions for executing the steps of a process according to an embodiment of the invention when said program is executed on a computer, and to a computer program product comprising program code instructions recorded on a computer-readable medium, for implementing the steps of a process according to an embodiment of the invention when said program is executed on a computer. Brief description of the drawings

[0029] The invention will be better understood and other features, details and advantages will become clearer from the following description, given by way of non-limiting reason, and from the accompanying figures, given by way of example.

[0030] [Fig. la] Fig. la] schematizes the principles of predictive coding for images of a video stream.

[0031] [Fig.lb] Fig.lb illustrates the impact of a significant change in the predictive coding of Fig.la for a state-of-the-art coder not dynamically adjusting the bitrate of the compressed video stream.

[0032] [Fig. le] Fig. le] illustrates the impact of a significant change on the predictive coding of Fig. la for a state-of-the-art coder dynamically adjusting the bitrate of the compressed video stream.

[0033] [Fig.2] Fig.2 represents a system for transmitting a video stream over a link with a constrained rate according to the state of the art.

[0034] [Fig.3] Fig.3 represents a system for transmitting a video stream over a data rate constrained link according to the invention.

[0035] [Fig.4] The [Fig.4] is a synoptic diagram illustrating the estimation of a shift and a statistical gap in a method of analyzing the change between two images of an uncompressed video stream according to an embodiment of the invention.

[0036] [Fig.5] Fig.5 represents, in block diagram form, the principles of determining an offset and a statistical gap between two images in a method of analyzing the change between two images of an uncompressed video stream according to an embodiment of the invention.

[0037] [Fig.6] The [Fig.6] represents a synoptic diagram of a method for encoding a video stream according to an embodiment of the invention.

[0038] [Fig.7a] Fig.7a illustrates the impact of a significant change on the predictive coding of Fig. 1a for the coding of an uncompressed video stream according to an embodiment of the invention in which, following the detection of a significant change, the encoder uses an intra-prediction to encode the current frame.

[0039] [Fig.7b] Fig.7b illustrates the impact of a significant change on the predictive coding of Fig.1a for the coding of an uncompressed video stream according to an embodiment of the invention in which, following the detection of a significant change, the encoder uses an intra-prediction to encode the current frame and where the quantization values ​​of the P frames of the GOP following the current image are adjusted so as to compensate for the increase in bit rate related to the scene change.

[0040] [Fig.7c] Fig.7c illustrates the impact of a significant change on the predictive coding of Fig.1a for the coding of an uncompressed video stream according to an embodiment of the invention in which, following the detection of a significant change, the encoder uses an intra-prediction to encode the current frame 702 and where the quantization value of the current image is adjusted so as to compensate for the increase in bit rate related to the scene change.

[0041] [Fig.7d] Fig.7d illustrates the impact of a significant change on the predictive coding of Fig.1a for the coding of an uncompressed video stream according to an embodiment of the invention in which, following the detection of a significant change, the current picture grouping (GOP) is interrupted, and a new picture grouping is started.

[0042] [Fig. 7e] Fig. 7e illustrates the impact of a change of average magnitude on the Predictive coding of [Fig. 1a] for encoding an uncompressed video stream according to an embodiment of the invention in which, following the detection of a significant change, the encoder adjusts the quantization value of the current image so as to compensate for the increase in bitrate related to the scene change

[0043] Identical references may be used in different figures when they refer to identical or comparable elements. Description of the implementation methods

[0044] Fig. 2 represents a 200 system for transmitting a video stream over a data rate constrained link according to the state of the art.

[0045] The system comprises: - means 201 for acquiring an uncompressed video stream, typically a camera, - 202 means for compressing and transmitting the video stream. These means are typically one or more computing devices such as a microprocessor, a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), an ASIC (Application-Specific Integrated Circuit), or any combination of these means. They are configured to implement: • a predictive video encoder 203, for example an HEVC encoder, allowing the generation of a compressed video stream 204 from the uncompressed video stream 201, • Means for transmitting the compressed video stream, configured to store the compressed video stream, divide it into frames 205 (or PDUs, abbreviation for Protocol Data Unit), the size of which depends on the capacity and characteristics of the communication link over which the compressed video stream is transmitted, and to implement the physical layer 206 processing necessary for shaping the PDUs for transmission over the communication link 207, typically a wireless radio link or a wired link; - Means 208 for receiving and decompressing the video stream received over the radio link 207. These means are typically one or more computing devices such as a microprocessor, a DSP, an FPGA, an ASIC, or any combination thereof, configured to implement: • a device for receiving the compressed video stream, configured to implement the physical layer 209 processing necessary for receiving the PDUs 210, and to reform the compressed video stream 211 from the PDUs, • a predictive video decoder 212, allowing the uncompressed video stream to be reformed from the compressed video stream 211, - means 213 for processing, displaying or recording the uncompressed video stream.

[0046] In the representation of [Fig.2], the compression and emission means 202 The means for receiving and decompressing the uncompressed video stream include the video encoder and the transmission equipment. Alternatively, these two functions can be implemented by two separate pieces of equipment connected by any link (wired, radio, optical, etc.). The same applies to the means for receiving and decompressing the compressed video stream, where the two functions can be implemented in separate pieces of equipment.

[0047] Figure 3 represents an embodiment of a video stream transmission system 300 over a data rate-constrained link according to the invention. This system comprises, in addition to the elements of Figure 2, a device 301 configured to detect significant changes between frames, which interfaces with the video encoder function 203. This device makes it possible to efficiently and inexpensively detect and quantify a change between two frames of the uncompressed video stream. It can be implemented either on the same computing means 302 as the encoder 203 or on separate computing means interfacing with the video encoder 203.

[0048] The invention relates to a method for analyzing the change between two images, ideally applicable to each pair of successive images in the video stream. The method is based on estimating a statistical difference and a shift between the two images, either in only one direction (for example, horizontally or vertically) or in two orthogonal directions. These estimated values ​​make it possible to determine whether a change between two images is significant or not, and possibly to quantify it more precisely. The remainder of the description will focus solely on the application case in which the statistical difference and the shift are estimated in two orthogonal directions. Embodiments limited to analysis in the horizontal domain only, or vertical domain only, respectively, can be easily derived from the two-directional embodiment described.

[0049] Alternatively, in order to reduce the amount of calculations performed, the process can be implemented on non-successive images, for example by estimating the statistical difference and the offset between two images separated by N images of the video stream, with N greater than or equal to two.

[0050] The invention therefore relates to a method for analyzing the change between two images of a video stream before its compression. This method comprises estimating a shift and a statistical difference between all or part of said images in at least one direction, and comparing the shift(s) and statistical difference(s) with thresholds in order to detect a significant change between said images.

[0051] Figure 4 is a block diagram illustrating the estimation of a shift and a statistical deviation in a method for analyzing the change between two images of an uncompressed video stream according to an embodiment of the invention. This estimation includes a first step 401 of applying a Mojette transform to each of the images in order to determine, for each image and for each direction considered, a vector corresponding to a projection of the image in that direction.

[0052] The Mojette transform is a discrete exact application of the Radon transform. It has the advantage of being very simple to implement since it uses only integer additions and subtractions. Applied to an image in the horizontal or vertical direction, it consists of summing the pixels in each row or column of the image. Thus, the output of the Mojette transform applied to an image in a given direction is a vector whose size corresponds to the size of the image in the orthogonal direction.

[0053] The Mojette transform can be applied to the whole of the image considered, or to only a portion of the images, which further limits the computational complexity required to implement the invention and makes it possible not to consider the edges of the image, which may disappear from one image to another as a result of the movement of the camera acquiring the video stream.

[0054] Estimating the offset and statistical deviation of the method for analyzing the change between two images of an uncompressed video stream according to an embodiment of the invention then comprises a second step 402 of calculating, for each direction considered, a cross-correlation between the vectors obtained by applying a Mojette transform during the first step 401 of the method, and then measuring an offset associated with each direction from the results of the cross-correlation(s). The offset corresponds to the maximum of the cross-correlation, obtained either directly on the discretized values ​​or, for greater accuracy, by polynomial approximation or by an equivalent mathematical method. Thus, at the end of the second step 402, an offset between the two images was measured in each of the directions considered, which corresponds to the equivalent of the estimated displacement between two images in the direction considered.

[0055] Estimating the statistical shifts and deviations of the change analysis method between two images of an uncompressed video stream according to an embodiment of The invention finally includes a third step 403 of compensating the offset calculated for each direction during the second step of the process in the corresponding vectors obtained during the first step 401 of the process, and then of measuring, for each direction considered, a statistical difference between the compensated vectors of the corresponding offset.

[0056] Thus, the vectors obtained by the Mojette transforms are shifted so as to be aligned before measuring their statistical deviation.

[0057] The measurement of the statistical difference can be carried out, for example, by integrating the differences between the vectors considered.

[0058] The offset (or offsets) and the statistical deviation (or statistical deviations) measured by the change analysis method between two images of an uncompressed video stream according to the invention make it possible to quantify the similarity between two images, and to adapt the behavior of the video stream encoder according to the temporal continuity between the images.

[0059] Figure 5 illustrates, in block diagram form, the principles of determining the offset and statistical difference between two images in a method for analyzing the change between two images of an uncompressed video stream according to an embodiment of the invention. In this embodiment, the measurements are taken along two orthogonal directions (horizontal and vertical).

[0060] The input for the measurements consists of two images 501 and 511 of an uncompressed video stream, typically but not necessarily successive images. A Mojette transform 502 / 512 is performed on each of the images in the horizontal and vertical directions. The results from each Mojette transform are a vector associated with the horizontal direction 503 / 513, and a vector associated with the vertical direction 504 / 514. A cross-correlation 505 / 515 is performed between the vectors associated with the same direction. A horizontal offset AX 506 is calculated from the cross-correlation performed on the horizontal vectors 503 and 513, typically by noting the position of the correlation peak. Similarly, a vertical offset AY 516 is calculated from the cross-correlation performed on the vertical vectors 504 and 514.

[0061] The horizontal vectors 503 and 513 are realigned by compensating for the horizontal offset between the two images, for example by applying an offset 507 equal to AX or -AX to one of the horizontal vectors to obtain a compensated horizontal vector 508, and adjusting the size of vectors 503 and 508 accordingly. Similarly, the vertical vectors 504 and 514 are realigned by compensating for the vertical offset between the two images, for example by applying an offset 517 equal to AY or -AY to one of the vertical vectors to obtain a compensated vertical vector 518, and adjusting the size of vectors 504 and 518 accordingly.

[0062] Finally, a horizontal statistical deviation oX 509 is calculated between the realigned horizontal vectors 503 and 508. Similarly, a vertical statistical deviation oX 519 is calculated between the realigned vertical vectors 504 and 518. This statistical deviation can be calculated in different ways, such as by summing the absolute differences.

[0063] The values ​​of the offsets AX and AY and the statistical deviations oX and oY characterize and quantify the changes between the two images. They can be compared with thresholds in order to determine the strategy to be adopted for the video coding of the stream.

[0064] The invention relates to the method of analyzing the change between two images of an uncompressed video stream described above, but also to a method of encoding a video stream using the method of analyzing the change between two images according to the invention.

[0065] Figure 6 shows a block diagram of a method for encoding a video stream according to an embodiment of the invention. The method takes as input an uncompressed video stream comprising a succession of images, and successively performs: - a step 602 for analyzing the change between two images of the video stream. This step can be applied systematically to all successive pairs of images of the uncompressed video stream 601, or to a subsampling of the images of the video stream; - a step 603 of encoding the images of the uncompressed video stream 601 by a conventional predictive video coding process, such as for example HEVC encoding, in which, when a significant change is detected, the video stream is modified or coding parameters of the video stream are modified in order to mitigate an instantaneous increase in encoder bit rate due to the scene change between the two images. This modification is therefore temporary.

[0066] The output of the method for encoding a video stream is a 604 video stream compressed in such a way as to avoid instantaneous spikes in bit rate that can lead to an increase in latency, frame loss and / or a decrease in image quality in prior art devices when the video stream varies abruptly.

[0067] Step 602 of the change analysis between two images of the video stream may include raising an alert when a significant change is detected. For this purpose, the calculated offsets AX and / or AY and the statistical deviations oX and / or oY may each be compared to a threshold, and an alert raised when one or more of these indicators exceeds the threshold. Crossing a threshold means that there is a sufficient difference between the current image and the previous image to trigger a parameter change before the current image is encoded. Threshold values ​​are to be determined on a case-by-case basis depending on the size of the images considered, the capabilities of the video encoder, the target bit rate, latency constraints, the strategy for adapting the encoder parameters implemented and the capabilities of the data link on which the video stream is transmitted.

[0068] Alternatively, when statistical shifts and deviations are calculated in two directions, the thresholds can be applied to the sum of squares (or absolute values) of the shifts and / or to the sum of squares (or absolute values) of the statistical deviations, which makes it possible to take into account the displacement / statistical deviation simultaneously in both directions.

[0069] Advantageously, step 602 of analyzing the change between two images of the video stream can include quantifying the magnitude of the change between the two images. This quantification can be achieved by comparing one or more of the calculated statistical offsets and deviations, combinations of offsets, and combinations of statistical deviations, with a plurality of thresholds to determine the magnitude of the change.For example, the sum of the absolute values ​​of the statistical deviations oX and oY can be compared with two thresholds to determine whether the statistical deviation between the two images is large (characteristic of a significant change between the two images despite motion compensation performed by compensating for the AX and / or AY offsets), medium (characteristic of a significant change, but where a large part of the image is contained within the other image), or small (characteristic of little or no scene change between the two images). This quantification of the magnitude of the change allows the video encoder to adapt its strategy for handling scene changes.

[0070] According to various embodiments of the invention, the modification of the video stream coding parameters carried out when a significant change between the current image and a previous image is detected may include, when the current image uses inter-frame prediction: - adapting the quantization step used to encode the current image, in order to reduce its size. Several levels of quantization can be considered, particularly when the magnitude of the change between images is quantized. Unlike the state of the art illustrated in [Fig. 1b] where an instantaneous data rate peak related to a scene change is compensated by a decrease in the data rate of subsequent images, the quantization step of the image where the scene change occurs can be increased, thus limiting the instantaneous data rate peak. Advantageously, the quantization step of subsequent images can also be modified; - when the video encoder has a bitrate setting, reducing this setting (temporarily) is equivalent to increasing the quantization step; - force the current image to be encoded using an intra-image prediction mode (image I). Indeed, when temporal continuity is limited, encoding the current image using intra-image coding is generally more efficient than inter-image coding in this specific case; - start a new GOP, thus abandoning the current image grouping. Since GOPs generally start with an image predicted by intra-frame prediction, whose size is not dependent on the temporal continuity between images in the video stream, this amounts to forcing the encoding of the current image to be done by intra-frame prediction while postponing the deadline for the next intra-frame.

[0071] According to another embodiment in which the images considered are successive images, when a significant change is detected between the current image and the previous image, the current image is replaced by the previous image in the video stream. This duplication of the oldest image has the effect of delaying the arrival of the scene change, but has no impact on the video latency. Since the new current image is a copy of the previous image, it can be encoded very efficiently, which creates a momentary drop in the instantaneous bitrate, allowing it to absorb all or part of the instantaneous bitrate spike that will occur during the encoding of the next image following the scene change.

[0072] These different embodiments are not incompatible. In particular, when the magnitude of the change is quantified, the policy for adjusting the video stream coding parameters can combine these different embodiments, for example by encoding the current image with intra-frame prediction when the statistical difference between the images is very small, by adjusting the current image quantization parameters when the statistical difference is medium, and by doing nothing when the statistical difference is large.

[0073] Fig. 7a illustrates the impact of a significant change on the predictive coding of Fig. 1a for the coding of an uncompressed video stream according to an embodiment of the invention in which, following the detection of a significant change, the encoder uses an intra-frame prediction to encode the current frame 701. This embodiment makes it possible to reduce the size of the current image compared to the use of inter-frame prediction, illustrated in Fig. 1b, and therefore to limit the instantaneous bit rate peak of the video encoder related to the scene change.

[0074] Figure 7b illustrates the impact of a significant change in the predictive coding of Figure 1a for the coding of an uncompressed video stream according to an embodiment of the invention wherein, following the detection of a significant change, the encoder uses an intra-prediction to encode the current 702 frame and wherein the quantization values ​​of the P frames of the GOP following the current image are adjusted so as to compensate for the increase in bit rate related to the scene change.

[0075] Figure 7c illustrates the impact of a significant change on the predictive coding of the [Fig.la] for the coding of an uncompressed video stream according to an embodiment of the invention in which, following the detection of a significant change, the encoder forces the use of an intra prediction to encode the current frame 703 and where the quantization value of the current image is adjusted so as to compensate for the increase in bit rate related to the scene change.

[0076] Figure 7d illustrates the impact of a significant change on the predictive coding of Figure 1a for encoding an uncompressed video stream according to an embodiment of the invention in which, following the detection of a significant change, the current picture grouping (GOP) is interrupted, and a new picture grouping is started. This embodiment makes it possible to use intra-frame prediction for the current frame, and thus to limit the increase in the peak bitrate. When the scene change occurs towards the end of the GOP, slightly anticipating the start of the next GOP has very little impact on the average encoder bitrate. The instantaneous bitrate is then less affected by the scene change than if the method were not implemented. When the scene change occurs at the beginning of the GOP, starting a new GOP, and therefore having two close I frames, can impact the bitrate.It is then possible to modify the quantization step for the following frame(s) in order to smooth the increase in bitrate, or to choose a different implementation.

[0077] Figure 7e illustrates the impact of a significant change of medium magnitude (rather that strong) on ​​the predictive coding of the [Fig. 1a] for the coding of an uncompressed video stream according to an embodiment of the invention in which, following the detection of an average change, the encoder adjusts the quantization value of the current image 704 so as to compensate for the increase in bit rate related to the scene change.

[0078] The method for analyzing the change between two images according to the invention makes it possible to identify abrupt scene changes that impair the compression efficiency of the inter-image prediction. This detection, performed upstream of the encoder, allows the encoder to be configured accordingly in order to dampen the effect of this change on the video encoder's bitrate, and thus avoid uncontrolled bitrate spikes, which are undesirable when transmission capacity is limited.

[0079] Another aspect of the invention is that its computational complexity is very low, since it relies on a Mojette transform whose implementation does not require than adders and subtractors. Furthermore, it can be implemented independently of the video encoder, and thus applies to any configurable predictive video encoder. Finally, the invention requires no modification of the decoder for implementation.

[0080] The invention relates to a method for analyzing the change between two images, and also to a device configured to implement the method. This device takes as input an uncompressed video stream 201, and delivers information representative of the presence of a change, which may be: - a boolean indicating the presence or absence of a significant change, - information relating to the extent of the change between the two images considered, or - configuration parameters of a video encoder, such as quantization step values, bitrate values, instructions regarding the type of predictive coding to use, instructions regarding resetting a group of images.

[0081] Alternatively, the device can implement the change analysis method between two images according to the invention on successive image pairs, and be configured to modify the video stream by copying the older image into the newer image when a significant change is observed. In this case, it is possible, but not essential, to provide additional information.

[0082] The invention also relates to a device 302 configured to implement a method for analyzing the change between two images according to an embodiment of the invention, and to encode the uncompressed video stream, adapting the encoder parameters and / or the uncompressed video stream when a significant change between the two images is detected. Advantageously, the device may also include means for transmitting the compressed video stream over a transmission link.

[0083] The invention also relates to a video stream transmission system (300) comprising: - means for acquiring a video stream, such as one or more cameras, - computing means configured to implement a method for encoding an uncompressed video stream according to an embodiment of the invention on the video stream to obtain a compressed video stream, - means of transmitting the compressed video stream, i.e., radiocommunication equipment configured to transmit the compressed video stream to another radiocommunication equipment over a wired or wireless radio link, - means of acquiring the compressed video stream, - means of decoding the compressed video stream to obtain a decompressed video stream, - means of displaying the decompressed video stream.

[0084] Finally, the invention relates to a computer program and a computer program product comprising program code instructions for executing a method for analyzing the change between two images of an uncompressed video stream according to the invention, optionally associated with program code instructions concerning the encoding of the video stream and instructions for modifying the video stream or the parameters of the encoding of the video stream when a significant change is detected.

Claims

1. Demands Method for analyzing the change between two images (501, 511) of an uncompressed video stream (201), said method comprising estimating a shift (506, 516) and a statistical deviation (509, 519) between all or part of said images in at least one direction, and using the calculated shift(s) and statistical deviation(s) to detect a significant change between said images; in which the estimation of a statistical deviation and a shift between all or part of said images (501, 511) in at least one direction comprises, for each direction considered: - a first step (401) of applying a Mojette transform (502, 512) to each of said images in order to determine, for each image, a vector (503, 513, 504, 514) corresponding to a projection of the image in said direction, - a second step (402) of calculating a cross correlation (505, 515) between the vectors obtained by applying a Mojette transform to the two images during the first step (401) of the process, then measuring a shift (506, 516) from the results of said cross correlations, - a third step (403) of measuring a statistical deviation (509, 519), said process being characterized in that the statistical deviation is measured between the vectors obtained by applying a Mojette transform to the two images during the first step (401) of the process, after compensation (507, 517) of the associated offset calculated during the second step (402) of the process; the use of the offset(s) and statistical deviation(s) calculated to detect a significant change between said images including: - the comparison of the offset(s) or a combination of offsets with one or more thresholds, and - the comparison of the statistical deviation(s) or a combination of statistical deviations with one or more thresholds.

2. A method for analyzing the change between two images of an uncompressed video stream according to claim 1, comprising at least one of the following provisions: - said offset corresponds to the maximum of said cross correlation; - said compensation is carried out in the vectors.

3. Method for analyzing the change between two images of an uncompressed video stream according to any one of the preceding claims, wherein said at least one direction comprises one or more directions selected from a horizontal direction and a vertical direction.

4. A method for analyzing the change between two images of an uncompressed video stream according to any one of the preceding claims, wherein at least one of the offset(s) or combination of offset(s) and the statistical deviation(s) or combination of statistical deviation(s) are compared with a plurality of thresholds in order to quantify the magnitude of said change.

5. A method for encoding (203) an uncompressed video stream (601) composed of a succession of images, said method being characterized in that it comprises, for at least one pair of images (501, 511) among said image succession, a step (602) for analyzing the change between two images of an uncompressed video stream according to any one of claims 1 to 4, and a step (603) for predictive coding of the uncompressed video stream (203) in which, when a significant change (101) is detected during the change analysis step (602), the uncompressed video stream or video stream coding parameters are modified so as to mitigate an increase in the bit rate of the video stream after predictive coding (604) related to said significant change.

6. A method for encoding an uncompressed video stream according to claim 5, wherein the adaptation of the encoding parameters comprises one or more of the following: - modifying a quantization step of the predictive video encoding, - reduce a bitrate instruction applied to predictive video coding, - use an intra-frame coding mode, - start a new frame grouping (GOP), - replace the oldest frame (501) with the newest frame (511).

7. A method of encoding an uncompressed video stream according to any one of claims 5 and 6, wherein the images of said image pair (501, 511) are successive images of the uncompressed video stream (201), and wherein the modification of the uncompressed video stream includes copying the older image in place of the newer image.

8. Device (301) comprising computing means configured to implement a method according to any one of claims 1 to 7.

9. A video stream transmission system (300) comprising: - means for acquiring an uncompressed video stream (201), - computing means configured to implement a method for encoding an uncompressed video stream according to any one of claims 5 to 7 on said uncompressed video stream to obtain a compressed video stream (204), - means for transmitting the compressed video stream (206), - means for acquiring the compressed video stream (209), - means for decoding (212) the compressed video stream to obtain a decompressed video stream (213), and - means for displaying the decompressed video stream.

10. Product computer program comprising program code instructions for implementing the steps of the process according to any one of claims 1 to 7 when said computer program is executed on a computer.

11. Computer-readable recording medium on which a computer program product according to claim 10 is recorded.

Images