Method and device for modifying at least one sequence of audiovisual content.

Abstract

The invention relates to a method for modifying audiovisual content, said method being implemented by a modification device and characterized in that it comprises: - a step of calculating a first average of a first plurality of modification indicators, each indicator being associated with an image from a first temporal sequence of images of said audiovisual content and determined by comparing all or part of said image, called the reference image, with all or part of another image, called the target image, from said first sequence, and when the value of said first average exceeds a threshold; - a step of replacing, within said audiovisual content, said first temporal sequence of images with a second temporal sequence of images. Figure 2 (for the abstract)

Description

Title of the invention: Method and device for modifying at least one sequence of audiovisual content.

[0001] 1. Scope of the invention

[0002] The invention relates to the field of telecommunications and more particularly to a method of improving the accessibility of audiovisual content.

[0003] 2. Prior Art

[0004] The accessibility of audiovisual sequences is a constantly evolving field, aiming to ensure that audiovisual content (for example, a film) is accessible to all users, including those with visual, auditory, motor, or cognitive disabilities. Numerous technologies and methods are currently used to make audiovisual content accessible. Most of them enable a user to better understand the audiovisual content through the addition of contextual elements.

[0005] One can cite for example subtitling which allows, by displaying text at the bottom of a screen, a real-time transcription of a dialogue or audio description which provides, through an additional audio track, a narration allowing the description of important visual elements, such as actions, expressions and scenes.

[0006] To improve the accessibility of audiovisual content, it is also possible to modify the rendering parameters of all or part of said content by using, for example, at the level of a screen, a contrast or brightness adapted to a particular context (for example, to a user's disability).

[0007] It must be noted that all these methods do not allow for real-time adaptation / modification of the original audiovisual content (i.e., audio and / or video) to a particular visual impairment.

[0008] 3. Description of the invention

[0009] The invention improves upon the prior art and, to this end, proposes a method for modifying audiovisual content, said method being implemented by a modification device and characterized in that it comprises:

[0010] - a step of calculating a first average of a first plurality of indicators of modification, each indicator being associated with an image from a first temporal sequence of images of said audiovisual content and determined by comparing all or part of an image, called the reference image, with all or part of another image, called the target image, from said first sequence and

[0011] when the value of said first average exceeds a threshold,

[0012] - a substitution step, within said audiovisual content, of said first temporal sequence of images by a second temporal sequence of images.

[0013] Advantageously, according to the invention, such a method makes it possible to modify audiovisual content, and more particularly its video portion, when the average value of a plurality of modification indicators associated with images in a temporal sequence of images of the audiovisual content exceeds a predetermined threshold. Specifically, a modification indicator (for example, a percentage) is calculated by comparing two images in the temporal sequence, for example, two adjacent images. The modification indicator can be determined by comparing the image with the highest temporal value to the image with the lowest temporal value, or vice versa. In the present case, the method performs this operation a plurality of times with pairs of images belonging to the temporal sequence of images and then calculates an average of all the modification indicators determined / calculated.

[0014] This average makes it possible to quantify the speed of the visual evolution of a video scene in audiovisual content. For example, when the average of the modification indicators is high (the images have been significantly modified throughout the time sequence of images), this means that the scene is fast-paced (for example, representative of an action scene). Then, depending on the calculated average value, the method either replaces the time sequence of images (first time sequence of images as defined in the invention) in the audiovisual content with a new time sequence of images (second time sequence of images as defined in the invention), for example, one adapted to a user's visual impairment.

[0015] Audiovisual content means any type of content capable of reproducing moving images forming a video, with or without sound. An example of video content is an avi or mpeg file.

[0016] According to a particular embodiment of the invention, a method as described above is characterized in that the step of determining at least one said change indicator comprises: - a step of determining a first area of ​​interest included by said reference image and a second area of ​​interest included by said target image, said first and second areas having the same dimensions and the same origin within the same reference frame; - a step of dividing said first and second areas of interest respectively into a first and a second ordered sequence of display units; - a step of calculating a ratio between the number of display units contained in one of said sequences of display units and the number of display units of the same index which differ between said first and second ordered sequence of display units.

[0017] This embodiment of the invention makes it possible to determine a change indicator for an image of audiovisual content. Initially, the method obtains a first area of ​​interest from an image in a temporal sequence of images (first temporal sequence of images within the meaning of the invention) and a second area of ​​interest with equivalent characteristics (same dimensions, same origin for the same reference frame) from another image in the temporal sequence of images. Subsequently, the method divides the first and second areas of interest into a first and second ordered sequence of display units of the same size / dimension. It should be noted that the display units can correspond to a pixel or a set of pixels of a representation of the areas of interest on a device capable of reproducing the area of ​​interest (for example, a screen, a video projector, etc.).

[0018] Once the two ordered sequences of display units are obtained, the method compares them to determine the number of display units with the same index (i.e., the same rank / having the same position within the ordered sequence of units) that differ between the two ordered sequences of units. Then, the method calculates a ratio (or a percentage) between the number of display units contained in one of said sequences of display units and the number of display units with the same index that differ between the two ordered sequences of units.

[0019] According to a particular embodiment of the invention, a method as described above is characterized in that the second time sequence is obtained within said audiovisual content.

[0020] This embodiment of the invention makes it possible to obtain the second time sequence directly from the audiovisual content, for example via a dedicated audiovisual track / sequence.

[0021] According to a particular embodiment of the invention, a method as described above is characterized in that the value of a second average of a second plurality of change indicators calculated for said second time sequence of images is lower than said first average, said first and second average being calculated with change indicators determined via pairs of reference image, target image of the same temporality.

[0022] This embodiment of the invention makes it possible to insert into the audiovisual content, in place of the first temporal sequence of images, a second temporal sequence of images whose average indicators of The modification determined under the same conditions as the average determined for the first temporal sequence of images (pairs of images with the same temporality) is smaller. In other words, the second temporal sequence of images corresponds to a video scene of audiovisual content whose visual evolution speed is slower than that corresponding to the first temporal sequence of images, i.e., a slower scene.

[0023] Indeed, people with visual impairments may have difficulty following a TV program that features fast-paced action. To better follow the program and understand the story, they need a slow-motion effect for each fast-paced scene to give them time to identify details that would not be perceived at normal speed.

[0024] According to a particular embodiment of the invention, a method as described above is characterized in that said second time sequence of images is obtained as a function of said first time sequence of images.

[0025] This embodiment of the invention makes it possible to generate a second time sequence of images based on the first time sequence of images, for example, by adding new images generated by morphing and / or blending methods to the first time sequence of images. These new images are generated using all or part of a plurality of images from the first time sequence of images. The second time sequence of images thus corresponds to the same video scene as the first time sequence of images, but slowed down.

[0026] According to a particular embodiment of the invention, a method as described above is characterized in that said second time sequence of images is obtained as a function of a user parameter.

[0027] This embodiment of the invention makes it possible to adapt the speed of the visual evolution of the video scene corresponding to the second temporal sequence of images to a user's disability. The scene will be faster or slower depending on the user's visual and / or cognitive abilities.

[0028] According to a particular embodiment of the invention, a method as described above is characterized in that when the duration of said first time sequence of images differs from that of said second time sequence of images, a first audio sequence of said audiovisual content of the same temporality as said first time sequence of images is substituted within said audiovisual content by a second audio sequence having the same duration as that of said second time sequence of images.

[0029] This embodiment of the invention makes it possible to adapt the audio track of the audiovisual content (synchronization) to the new video content which incorporates the second A new audio sequence of images, the same duration as the second image sequence, is generated and inserted into the audiovisual content in place of the original audio sequence of the same duration as the first image sequence. The generation of this new audio sequence is achieved, for example, by adding gaps at key moments in the original audio sequence, which can correspond to specific sound events such as musical beats, one or more spoken words, sound effects, etc.

[0030] Note that the various modes or embodiments mentioned above can be added independently or in combination with each other to the modification process defined above.

[0031] The invention also relates to a device for modifying audiovisual content characterized in that it comprises:

[0032] - a module for calculating a first average of a first plurality of indicators of modification, each indicator being associated with an image from a first temporal sequence of images of said audiovisual content and determined by comparison of all or part of an image, called the reference image, with all or part of another image, called the target image, from said first sequence;

[0033] - a replacement module, within said audiovisual content, for said first temporal sequence of images by a second temporal sequence of images.

[0034] The term module can refer to a software component, a hardware component, or a set of hardware and software components. A software component itself corresponds to one or more computer programs or subprograms, or more generally to any element of a program capable of implementing a function or set of functions as described for the modules concerned. Similarly, a hardware component corresponds to any element of a hardware assembly capable of implementing a function or set of functions for the module concerned (integrated circuit, smart card, memory card, etc.).

[0035] The invention also relates to a terminal or a server comprising a modification device as described above.

[0036] The invention also relates to a computer program comprising instructions for implementing the above method according to any one of the particular embodiments described above, when said program is executed by a processor. The method can be implemented in various ways, including in hardwired or software form. This program can use any programming language and be in the form of source code, object code, or code intermediate between source and object code, such as in a partially compiled form, or in any other desirable form.

[0037] The invention also relates to a computer-readable recording or information medium containing instructions for a computer program as described above. The aforementioned recording media can be any entity or device capable of storing the program. For example, the medium may include a storage means, such as a ROM, for example a CD-ROM or a microelectronic circuit ROM, or a magnetic recording means, for example a hard drive. Furthermore, the recording media may be a transmissible medium such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio, or by other means. The programs according to the invention can, in particular, be downloaded from a network such as the Internet.

[0038] Alternatively, the recording media may correspond to an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the process in question.

[0039] This device and modification system and this computer program have characteristics and advantages similar to those described above in relation to the modification process.

[0040] 4. List of figures

[0041] Other features and advantages of the invention will become more apparent upon reading the following description of particular embodiments, given by way of simple illustrative and non-limiting examples, and the accompanying drawings, among which:

[0042] [Fig.la], [Fig.lb], [Fig.le] Figures la, 1b and le schematically represent a video sequence in which a part is replaced according to a particular embodiment,

[0043] [Fig.2] Fig.2 represents the different stages of the modification process according to a particular embodiment of the invention,

[0044] [Fig. 3a], [Fig. 3b] Figures 3a and 3b represent a method of calculating the indicators modifications according to a particular embodiment of the invention

[0045] [Fig.4] Fig.4 represents in a simplified manner a modification device according to a particular embodiment of the invention.

[0046] 5. Description of an embodiment of the invention

[0047] The invention will now be described using the example of the distribution of audiovisual content displayed on a television screen in a user's home. However, the invention can find application in other contexts. For example, the invention can be used in mobile devices via the user's mobile terminal. The modification process is performed by a modification device, for example, integrated into the television or a television decoder connected to the television. More generally, the the process can be executed by any type of connected terminal / server capable of obtaining audiovisual content and broadcasting it to television for playback.

[0048] Fig. 1a represents a video 100 broadcast to the user and displayed on a television screen. In this example, the video comprises a video component 101 and an audio component 102.

[0049] In a first step DETER / 200 shown in [Fig. 2], the audiovisual content 100 is analyzed to determine a plurality of modification indicators. Specifically, the process analyzes the audiovisual content in its entirety or by time block / time range of a few milliseconds, seconds, or minutes, the set of blocks potentially corresponding to all or part of the audiovisual content 100.

[0050] According to a particular embodiment, the analysis is carried out on time blocks of the audiovisual content 100 corresponding to the different temporal states of a sliding time window for example of a few milliseconds, seconds or minutes.

[0051] Figure [Fig. 1b] illustrates the sequence 100 in which a portion 105 corresponding to a sequence, a time block, or a predefined time range is analyzed by the method. Instants 103 and 104 correspond respectively to the beginning and end of the time range 105. It should be noted that the size of the time sequence of images 105 can be determined by the method according to one or more parameters and / or be variable or fixed over time.

[0052] Figures 3a and 3b illustrate an example of a method for calculating the change indicators for the image sequence / time range 105. More specifically, [Fig. 3a] illustrates the image time sequence 105 in more detail. In this example, the image time sequence 105 comprises the images in to in+5 and an audio component 300. When the method obtains this sequence, it determines / calculates a plurality of change indicators (ratio, percentage, etc.) associated with the image time sequence 105, more precisely with the images in the image time sequence 105. Each indicator is calculated by comparing two different images (a reference image and a target image) from the time sequence 105, for example, two adjacent images. The method then repeats this calculation a plurality of times.

[0053] For example, the method can calculate five indicators representing the level of modification between the images in and in+ i, in + [ and in+ 2, in +2 and in+ 3, in +3 and in+ 4, in +4 and in+5. Thus, each image will be associated with an indicator.

[0054] Alternatively or cumulatively, the method can calculate indicators for which an image of sequence 105, for example in, is compared to a plurality other images of the sequence 105, for example in +iàin +5. Thus, in this example, 5 indicators will be associated with the image in (the indicator of modification of the image in versus in+ 15in versus in+ 2in versus in+ 3in versus in+ 4, and in versus in+5.

[0055] Alternatively or cumulatively the method can calculate indicators for all pairs of different images of sequence 105.

[0056] The comparison between two images of sequence 105 is now described with reference to [Fig. 3b]. First, the method obtains a first region of interest from one image of sequence 105, for example, image in (the first temporal sequence of images within the meaning of the invention), and a second region of interest with equivalent characteristics (same dimensions, same origin for the same reference frame) from another image of sequence 105, for example, image in +b. In our example, the first and second regions of interest correspond respectively to the entirety of images in and in +b. Second, the method similarly divides the first and second regions of interest into a first and second ordered sequence of display units of the same size / dimension (301). An ordered sequence of display units can, for example, correspond to all the rows and / or columns of pixels constituting an image placed end to end.Figure [Fig. 3b] illustrates this segmentation for the image in with an ordered sequence of 301 display units comprising the pixels Pnl, Pn2, Pn3, etc.

[0057] Note that the display units can correspond to a pixel or a plurality of pixels corresponding to a graphic representation of an area of ​​interest at the level of a device capable of reproducing the area of ​​interest (for example a screen, a video projector, etc.).

[0058] Once the two ordered sequences of display units are obtained, the method compares them to determine the number of display units with the same index (i.e., the same rank / having the same position within the ordered sequence of units) that differ between the two ordered sequences of units. The method then calculates a ratio (or a percentage) between the number of display units contained in one of said sequences of display units and the number of display units with the same index that differ between the two ordered sequences of units.

[0059] Alternatively, the method can, for example, compare in real time each display unit of the image in with a display unit of the same size and index from the image in +b. The method then calculates a ratio (or a percentage) between the number of display units contained in the image in or in +i and the number of display units of the same index that differ between the images in and in +b.

[0060] During the CALC / 201 step shown in [Fig.2], the process calculates an average of the plurality of change indicators associated with sequence 105. This average allows us to quantify the speed of visual evolution in a video scene of audiovisual content. For example, when the average of the change indicators is high (the images have changed significantly throughout the 105-frame time sequence), this means that the scene is fast-paced (for example, representative of an action scene).

[0061] During the SUB / 202 step shown in [Fig.2] and [Fig.lb] / lc, the process substitutes or not, depending on the value of the calculated average, the time sequence of images 105 (first time sequence of images in the sense of the invention) with a new time sequence of images 106 (second time sequence of images in the sense of the invention) in the audiovisual content 100.

[0062] The temporal sequence 106 is for example adapted to a user's visual impairment.

[0063] According to a particular embodiment, sequence 106 is obtained within the audiovisual content 100 (for example, at the beginning or end) in a dedicated chapter of the audiovisual content 100. This allows for faster adaptation of the audiovisual content 100 to a user's disability, since the replacement sequence 106 is already generated and present in the audiovisual content 100.

[0064] According to a particular embodiment, the replacement sequence 106 is a sequence whose average modification indicators, calculated under the same conditions as those calculated for sequence 105, are lower. Thus, the time sequence of images 106 corresponds to a video scene of the audiovisual content 100 whose visual evolution rate is lower than that corresponding to the time sequence of images 105. This embodiment makes it possible to obtain audiovisual content with a slower scene 106, for example, adapted to a user's visual impairment.

[0065] According to a particular embodiment, sequence 106 is obtained from sequence 105. Sequence 106 is, for example, generated by adding new images to sequence 105 using methods called morphing and / or blending of all or part of a plurality of images from said first temporal sequence of images. Sequence 106 thus corresponds to the slowed-down video scene of sequence 105.

[0066] According to a particular embodiment, the sequence 106 is obtained based on a user parameter. In this embodiment, the number of images inserted within the sequence 105 is, for example, a function of a parameter. This parameter may be related to the degree of a user's disability.

[0067] According to a particular embodiment, when sequence 106 is longer than sequence 105, the audio component of sequence 105 is modified so that its duration corresponds to that of sequence 106. This component The modified audio is then substituted for the audio component of sequence 105 within the audiovisual content. The generation of the modified audio component is achieved, for example, by adding silences at key moments in the original audio sequence, which can correspond to specific sound events such as musical beats, one or more lines of dialogue, sound effects, etc.

[0068] Alternatively or cumulatively, the addition of blanks can be carried out at the end and / or at the beginning of the sequence.

[0069] Figure 4 illustrates the hardware architecture of a device S configured to implement the modification process according to a particular embodiment of the invention. In the embodiment described herein, this device has the hardware architecture of a computer. It includes, in particular, a processor PROC1, a random access memory MV1, a read-only memory MEM1, and a non-volatile flash memory MF1. Such means are known per se and are not described in further detail here. The read-only memory constitutes a storage medium according to the invention, readable by the processor PROC1, on which a computer program PG1 according to the invention is stored. This program comprises instructions for implementing the steps of the modification process as described above when the program is executed by the processor PROC1.At initialization, the code instructions of the computer program PG1 are, for example, loaded into memory before being executed by the PROC1 processor. The PROC1 processor of the processing unit UT1 implements, in particular, the steps of the modification process according to any one of the specific embodiments described in relation to figures 1a, 1b, 1e, 2 and 3, according to the instructions of the computer program PG1.

[0070] The device S also includes a DETER1 module capable of determining change indicators associated with a time sequence of images. For this purpose, this module can rely on a comparison module (not shown) capable of comparing two images from the time sequence of images. Alternatively, this comparison module is capable of comparing an image from the time sequence of images with an image from the audiovisual content that includes the time sequence of images.

[0071] The device S further includes a CALC1 module capable of calculating an average of the change indicators determined by the DETER1 module.

[0072] The device S also includes a SUBI module capable of substituting, within an audiovisual content, a temporal sequence of images with another according to the value of the average calculated by the CALC1 module.

[0073] It goes without saying that the embodiment described above has been given purely as an example and is in no way limiting, and that many modifications may can be easily implemented by a person skilled in the art without going outside the scope of invention.

Claims

Demands

1. A method for modifying audiovisual content, said method being implemented by a modification device and characterized in that it comprises: - a calculation step (CALC) of a first average of a first plurality of modification indicators, each indicator being associated with an image of a first time sequence of images of said audiovisual content and determined (DETER) by comparison of all or part of said image, called reference image, with all or part of another image, called target image, of said first sequence and when the value of said first average exceeds a threshold, - a substitution step (SUB), within said audiovisual content, of said first time sequence of images by a second time sequence of images.

2. A method according to claim 1 characterized in that the determination of at least one said change indicator comprises: - a step of determining a first area of ​​interest included by said reference image and a second area of ​​interest included by said target image, said first and second areas having the same dimensions and the same origin within the same reference frame; - a step of dividing said first and second areas of interest respectively into a first and a second ordered sequence of display units; - a step of calculating a ratio between the number of display units contained in one of said sequences of display units and the number of display units of the same index that differ between said first and second ordered sequences of display units.

3. Method according to claim 1 characterized in that the second time sequence is obtained within said audiovisual content.

4. A method according to claim 1 characterized in that the value of a second average of a second plurality of change indicators calculated for said second time sequence of images is lower than said first average, said first and second averages being calculated with change indicators determined via pairs of reference image, target image of the same temporality.

5. A method according to claim 1 characterized in that said second time sequence of images is obtained as a function of said first time sequence of images.

6. Method according to claim 1 characterized in that said second time sequence of images is obtained as a function of a user parameter.

7. The method according to claim 1 characterized in that when the duration of said first time sequence of images differs from that of said second time sequence of images, a first audio sequence of said audiovisual content of the same temporality as said first time sequence of images is substituted within said audiovisual content by a second audio sequence having the same duration as that of said second time sequence of images.

8. Device for modifying audiovisual content characterized in that it comprises: - a calculation module (CALC1) for a first average of a first plurality of modification indicators, each indicator being associated with an image of a first temporal sequence of images of said audiovisual content - a module for determining said plurality of modification indicators by comparing all or part of an image, called the reference image, with all or part of another image, called the target image, of said first sequence - a substitution module (SUBI), within said audiovisual content, of said first temporal sequence of images by a second temporal sequence of images.

9. Terminal or server characterized in that it includes a modification device according to claim 8.

10. Computer program comprising instructions for carrying out the method according to any one of claims 1 to 7, when the program is executed by a processor.

Images