Shared restitution of multimedia content and individually enhanced

Abstract

The invention aims to enhance the individual experience of shared viewing of audiovisual content displayed by a common device (20) to multiple viewers, typically a television. This individual enhancement is achieved independently for each viewer using personal equipment (30), tailored to their specific disability, so as not to cause inconvenience to other viewers. Each viewer with a disability uses their personal equipment to create a mirrored audiovisual session of the shared device's audiovisual session, obtain a track corresponding to their disability—typically subtitles for the hearing impaired or an audio description—and play this track back to them only, independently of the other viewers. [Fig. 2]

Description

Title of the invention: Shared and individually augmented multimedia content playback Scope of the invention

[0001] The present invention relates to the field of broadcasting and playback of multimedia content, typically audiovisual. Previous techniques

[0002] It is common for people to share audiovisual content together. This is particularly true when watching a film on family television or at the cinema. This shared or collective consumption of content has significant social importance: it allows for exchange between viewers.

[0003] Audiovisual, or audio-video, content is generally composed of a plurality of components, typically channels or tracks. The main components include a video component and an audio component. Other general components may be provided, such as audio tracks in different languages ​​or one or more subtitle tracks (depending on the languages ​​available). Other additional components exist to accommodate viewers with disabilities, in particular one or more audio description tracks (depending on the languages ​​available), one or more subtitle tracks for the hearing impaired (depending on the languages ​​available) or for dyslexic viewers, a sign language track, etc.

[0004] The shared or collective consumption of audiovisual content in the presence of a viewer with a disability requires the activation of an audio description track or subtitles for the hearing impaired, or other options depending on the disability. These tracks can also be combined when viewers have different disabilities. Indeed, consumer television equipment is not equipped with multiple outputs allowing, for example, separate output of the main audio track and the audio description track.

[0005] Also, the activation of these tracks degrades the multimedia experience of other viewers, in particular those without a disability or not with the same disability.

[0006] In this context, document FR2981782 proposes equipping a cinema with a central beacon configured to capture an audio signal from the audiovisual content and send, synchronously, a digital data stream corresponding to the audio signal, typically dialogue during a film screening. The stream, received by a user's personal terminal, is transcribed into an audio output for the user.

[0007] This solution is not satisfactory however because it requires in particular the costly acquisition of a central beacon.

[0008] There is therefore a need for a technical solution using consumer equipment to improve the shared consumption of audiovisual content, reducing the inconvenience to neighbouring viewers. Description of the invention

[0009] A conventional method for reproducing audiovisual content composed of multiple components includes the following steps: to establish an initial audiovisual session between equipment shared by multiple viewers and a content server to obtain components of audiovisual content, and to reproduce, using common equipment, a video component of the audiovisual content with a common display device and an audio component of the audiovisual content with a common sound device and optionally a subtitle component.

[0010] The invention, however, provides for the following steps: to establish a second audiovisual session between a personal device belonging to one of the viewers and the content server in order to obtain at least one so-called "additional" component of the same audiovisual content, and to render the additional component of the audiovisual content using a rendering device specific to the personal equipment, so that the rendering of the additional component increases, in a synchronized manner for the viewer of the personal equipment, the rendering of the video or audio component by the common display or sound device.

[0011] Also, a method according to the invention includes the following steps: to establish a second audiovisual session between a viewer's personal equipment and a content server to obtain at least one so-called "additional" component of audiovisual content that is also the subject of a first audiovisual session established between equipment shared by several viewers and the content server, and whose video and audio components are reproduced, by the shared equipment, respectively using a shared display device and a shared sound device, and to render the additional component of the audiovisual content using a rendering device specific to the personal equipment, so that the rendering of the additional component increases, in a synchronized manner for the viewer of the personal equipment, the rendering of the video or audio component by the common display or sound device.

[0012] The term "augment" means that the video or audio component common to all viewers is supplemented, for example by superimposition (sound or visual), by the additional component. Also, since this rendering of the additional component is performed by the personal device, only the individual viewer benefits from this enhancement, without inconveniencing other viewers. The rendering mechanism specific to the personal device is therefore adapted to this enhancement, in that it allows the merging of the video or audio component with the additional component, for example, an audio merging using open-back headphones or a visual merging using augmented reality glasses. It should be noted that if additional subtitles supplement the displayed standard subtitles, they a fortiori enhance the video on which the standard subtitles are displayed.

[0013] Using a second audiovisual session for the same audiovisual content with the content server eliminates the need for additional equipment, such as the central beacon of the state of the art. Indeed, this second session, mirroring the first session in that it allows access to the same audiovisual content, enables the individual viewer to access the specific components that are relevant to them, for example, depending on a disability they have.

[0014] The approach according to the invention can be applied to a multitude of personal devices associated with a multitude of respective spectators wishing to access different additional data, for example depending on their respective disabilities or wishes.

[0015] The invention also relates to personal equipment comprising a self-returning device and configured to: to establish a second audiovisual session with a content server to obtain at least one so-called "additional" component of audiovisual content that is also the subject of a first audiovisual session established between equipment shared by several viewers and the content server, and whose video and audio components are reproduced by the shared equipment, respectively using a shared display device and a shared sound device, and to render the additional component of the audiovisual content using a rendering device specific to the personal equipment, so that the rendering of the additional component increases, in a synchronized manner for the viewer of the personal equipment, the rendering of the video or audio component by the common display or sound device.

[0016] The invention also relates to an audiovisual system comprising: a content server, equipment shared by multiple spectators and comprising a common display device and a common sound device, the shared equipment being configured to establish the first audiovisual session with the content server for to obtain components of audiovisual content, and to render a video component of the audiovisual content using the common display device and an audio component of the audiovisual content using the common sound device, and at least one personal device for one of the viewers as above, i.e. having its own rendering device, the personal device being configured to establish a second audiovisual session with the content server to obtain at least one so-called "additional" component of the same audiovisual content, and to render the additional component of the audiovisual content using its own rendering device, so that the rendering of the additional component increases, in a synchronized manner for the viewer of the personal device, the rendering of the video or audio component by the common display or sound device.

[0017] The system has the same advantages as those of the process mentioned above.

[0018] Optional features of embodiments of the invention are defined in the attached claims. Some of these features are explained below with reference to a system, while they can be transposed into process features.

[0019] In one embodiment, the video and audio components are not reproduced by the personal equipment. In other words, the additional component is different from these components, and only this additional component (possibly with one or more other additional components) is reproduced by the personal equipment. Therefore, the viewer possessing this equipment also consumes the shared reproductions by the shared equipment in order to fully enjoy the audiovisual content.

[0020] In one embodiment, the method comprises: the establishment of multiple audiovisual sessions between several personal devices belonging to different viewers and the content server to obtain audiovisual content data, and the rendering, using a rendering device specific to each personal device, of an additional component of the audiovisual content so that the rendering of the additional component increases, in a synchronized manner for the viewer of the personal device concerned, the rendering of the video or audio component by the common display or sound device.

[0021] Multiple viewers may have distinct disabilities or preferences, so that each can access the corresponding data (additional component) during shared consumption of the audiovisual content, without inconvenience to the other viewers. Of course, two viewers with the same disability can each access the same content separately and independently. additional component on their own personal equipment.

[0022] In one embodiment, the additional component is a subtitling component for the hearing impaired, displayed to the viewer using personal equipment, superimposed on the video component displayed by the common display device. This display can be provided either in addition to existing standard subtitles (i.e., by displaying only a difference in subtitles) or as a replacement for such standard subtitles (in which case the latter are, for example, hidden).

[0023] In particular, the rendering device specific to the personal equipment is one of the following: a virtual reality headset or glasses, an augmented reality headset or glasses, or a holographic device. These devices easily allow the additional component to be superimposed onto the common video. Alternatively, the viewer can use a remote screen, for example, a tablet or a smartphone.

[0024] In one embodiment, the additional component is an audio description component provided to the viewer of the personal equipment, superimposed on the audio component provided by the common sound device.

[0025] In particular, the playback device specific to the personal equipment is one of the following: open-back headphones, a hearing aid, a bone conduction audio device, or an augmented reality headset or glasses equipped with an open-back audio system. These devices easily allow the additional component to be superimposed onto the common audio.

[0026] In one embodiment, the additional component is one of the following: a subtitle component for the hearing impaired, an audio description component, a sign language component, a subtitle component for dyslexic viewers, the audio component in which case the playback by the playback device specific to the personal equipment applies at least one audio processing (for example, increased sound volume or a sound boost of only the dialogues, modification of the frequency curve to compensate for deficiencies on certain frequencies, etc.) compared to the playback by the common sound device, the video component in which case the playback by the playback device specific to the personal equipment applies at least one video processing compared to the playback by the common display device.Such video processing might, for example, involve applying a different contrast or a different color display. Audio and video processing are therefore distinct from those used for playback by common display and audio devices.

[0027] In one embodiment, establishing a second audiovisual session comprises one of the following steps: - to initiate, using personal equipment, a discovery procedure to locate shared equipment on a local network and obtain from the shared equipment identification information for the first audiovisual session, - to enter, by the viewer concerned on their personal equipment, identification information for the first audiovisual session, - connect personal equipment to the content server using the same user account as the one used on the shared equipment to obtain the audiovisual content components, followed by a step consisting of requesting the content server to establish a mirror audiovisual session of the first audiovisual session.

[0028] Thus, the second audiovisual session launched mirrors the first audiovisual session, in that it relates to the same audiovisual content. Furthermore, the identification of the first audiovisual session allows for synchronization between the two sessions.

[0029] The present invention also relates to a computer program comprising instructions for implementing the above method, when this program is executed by a processor. This program may use any programming language (for example, an object-oriented language or other), and may be in the form of interpretable source code, partially compiled code, or fully compiled code.

[0030] The invention also relates to a non-transient recording medium readable by a computer on which a program is recorded for the implementation of the above method, when this program is executed by a processor.

[0031] At least part of the methods according to the invention can be implemented by computer. Consequently, the present invention can take the form of a completely hardware embodiment, a completely software embodiment (comprising firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all of which can be collectively referred to herein as a "circuit," "module," or "system." Furthermore, the present invention can take the form of a computer program product embedded in any tangible medium having computer-usable program code embedded in the medium.

[0032] Since the present invention can be implemented in software, the present invention can be incorporated in the form of computer-readable code for provision to a programmable device on any suitable medium. A tangible or non-transient medium may include a storage medium such as a hard disk drive, a magnetic tape device, or a semiconductor memory device and the like. A transient medium may include a signal such as a electrical signal, electronic signal, optical signal, acoustic signal, magnetic signal or electromagnetic signal, for example a microwave or RF (radio frequency) signal. Brief description of the drawings

[0033] Other features, details, and advantages of the invention will become apparent upon reading the detailed description below. This description is purely illustrative and should be read in conjunction with the accompanying drawings, on which: Fig. 1

[0034] [Fig. 1] illustrates a multimedia or audiovisual system for implementing this process according to embodiments of the invention; and Fig. 2

[0035] [Fig.2] illustrates, using a flowchart, general steps of a process according to embodiments of the invention. Detailed description

[0036] The present invention aims to enhance the individual viewing experience of audiovisual content displayed by a device common to several viewers, typically a TV displaying the images of the audiovisual content and broadcasting the associated sound at ambient volume for all viewers, or even with "classic" subtitles (depending on the language selected). The individual enhancement of the experience is achieved independently for each viewer concerned, using personal equipment, so as not to cause any inconvenience to other viewers sharing the audiovisual content on the TV.

[0037] A viewer may be affected in that he has a disability and can only fully enjoy the audiovisual content if additional information is presented to him, typically subtitles for the hearing impaired, an audio description, a boost in sound volume or dialogues or any other audio processing of the main audio track (for example, adapting the frequency curve to his hearing impairment), any other display that helps readability for a visually or hearing impaired person, subtitles for dyslexics, a display of sign language, etc.

[0038] Also, each spectator concerned uses personal equipment, for example a smartphone, a tablet, AR augmented reality glasses, a VR virtual reality headset, a smartwatch, a holographic device, a contact lens type device (or AR glasses) with retinal projection, a retinal or ocular implant, etc., giving him this additional information according to his disability.

[0039] In this way, the consumption experience, which remains shared, is individually adapted to each person's disability, without degrading the consumption experience of others. All viewers thus fully enjoy the audiovisual content, with or without additional information depending on their respective disabilities, allowing for exchanges between them during the viewing.

[0040] A method according to the invention for the restitution of audiovisual content made up of multiple components then comprises the following steps.

[0041] On the one hand, a personal device belonging to one of the spectators establishes a second audiovisual session with a content server to obtain at least one so-called "additional" component of audiovisual content, the video and audio components of which, or even subtitles, are obtained from the content server by a device common to the spectators through a first audiovisual session, and are rendered, on the common device, by a common display device with regard to the video component and by a common sound device with regard to the audio component.

[0042] On the other hand, the personal equipment reproduces the additional component using a reproduction device specific to the personal equipment, so that the reproduction of the additional component increases, in a synchronized manner for the viewer of the personal equipment, the reproduction of the video or audio component by the common display or sound device.

[0043] Fig. 1 illustrates a multimedia or audiovisual system 1 for implementing this process according to embodiments of the invention.

[0044] System 1 comprises a content server 10, a collective content playback device 20 for a plurality of viewers A, B, C, D, and one or more personal devices 30 specific to one or more of the viewers respectively. In the example in the figure, only viewers B, C, and D are equipped with such personal devices for an implementation of the invention: device 30B for viewer B, device 30C for viewer C, and device 30D for viewer D.

[0045] The server 10 is configured to provide digital audiovisual content to requesting devices. This could be, for example, a multimedia server, typically for video-on-demand (VOD) content, storing this content, encrypted or unencrypted, in dedicated storage spaces. Alternatively, it could be a live streaming server. The multimedia server 10 thus includes application software that retrieves the audiovisual content and makes it available locally or over a network for streaming.

[0046] Each audiovisual content consists of a video component (or track or object) and one or more audio components (or tracks or objects, Dolby AC-4 object type - trade name) depending on the available languages. According to a language configuration chosen by a user, the video component and the audio component of the language chosen form the two main components of the audiovisual content,

[0047] The audiovisual content may include complementary or additional information forming other components (or tracks or objects) such as classic subtitles (possibly in different languages), subtitles for the hearing impaired, an audio description, a sign language track (generally video or metadata enabling the generation of 3D objects, typically an avatar), subtitles for dyslexic viewers, any other type of data helping readability for a viewer with a visual, auditory or cognitive disability.

[0048] Although the Figure represents a single server 10, multiple servers may be provided to collaborate for the provision of the same VOD service or broadcasting of television content or live content.

[0049] The collective equipment 20 is equipment requesting the server 10 to obtain audiovisual content, and in particular the main components (audio, video), or even additional components (for example, classic subtitling).

[0050] The shared equipment may be local to the server 10, either on the same local network, or directly connected to it by a dedicated computer link (for example USB). Local use corresponds, for example, to the case of a NAS (Network-Attached Storage) multimedia server in a home or in a cinema.

[0051] Alternatively, the shared equipment 20 can be located remotely from the server 10, connected to it by a communication network 40 such as the Internet (via a local wireless network, a local wired network, or a mobile phone network with data). This remote case corresponds, for example, to conventional streaming servers (VOD or television programs), which are particularly well-suited to content consumption within a household.

[0052] Equipment 20 is said to be collective because the content playback carried out by it benefits, collectively, the plurality of spectators A, B, C, D. For this playback, the collective equipment 20 includes a display device called "common" 21 and a sound device called "common" 22, either directly integrated into the collective equipment 20, or remote and connected to it by conventional (audio, video) links.

[0053] The collective equipment 20 can take various forms: a connected television, a multimedia center such as Apple TV (trade name), a TV decoder or Internet box ("set-top box" in English), a computer, a digital tablet, etc.

[0054] The display device 21 can be of any type, typically the screen integrated into the shared equipment 20 or a remote display screen, or a 2D or 3D video projector, a film projector, or even a holographic projector. The display device 21 reproduces, under the control of the shared equipment 20, the video component of the audiovisual content requested by a primary user. Incidentally, a component of classic subtitles can also be rendered by the display device 21 if the user requests it.

[0055] The sound device 22 typically consists of one (or more) loudspeakers either integrated into the shared equipment 20 or located remotely from it. The sound device 22 reproduces, under the control of the shared equipment 20, the audio component of the audiovisual content requested by the primary user.

[0056] The main audio and video components of the audiovisual content, and possibly a component of traditional subtitles, are thus delivered by the shared equipment 20 to all viewers A, B, C, and D. Some or all of the viewers have a disability requiring the presentation of additional information to these components, in a synchronized manner, to enable them to fully view the audiovisual content. This information is presented to them, as explained below, using personal devices 30, connected locally to the shared equipment 20 via a local network 50 (typically a Wi-Fi network, or alternatively a wired LAN). The personal devices 30 and the shared equipment 20 thus share a common clock (of the local network 50), enabling synchronization as described below.Personal devices 30 can access the content server either directly via the local network 50 (via a gateway to the Internet, if applicable), or indirectly via the shared equipment 20.

[0057] Viewer A has no disability and fully enjoys the audiovisual content provided by the common equipment 20. This may be the main user who starts playing the audiovisual content (a film in French for example, or in the original version with subtitles) on the common equipment 20.

[0058] Viewer B has a visual impairment requiring the use of audio description. Their personal device 30B may be a tablet, a smartphone, a smartwatch, or AR glasses with an integrated audio system, allowing the execution of a multimedia application as described below. To reduce inconvenience to other viewers, the listening system integrated into or connected (wired or wirelessly, typically via Bluetooth - registered trademark) to the personal device 30B, i.e., the audio description sound playback device, may include a conventional non-waterproof earphone (in "transparent" mode to allow maximum external sound transmission), a hearing aid, "open" headphones (i.e., with an open shell allowing sound to pass in and out), a bone conduction audio device, or the audio system of the AR headset or glasses.In general, any audio device that allows the viewer to maintain a connection with the outside auditory world can be used. Thus, the reproduced audio description track can be superimposed on the main audio track of the content. diovisual, for viewer B only.

[0059] Viewer C has a hearing impairment requiring the use of subtitles for the hearing impaired, or alternatively, a (video or synthesized) component of sign language. Their personal equipment 30C may be a tablet, smartphone, smartwatch, VR headset, AR glasses, retinal implant, or contact lenses with a retinal projector, enabling the execution of a multimedia application as described below. The display system integrated into or connected (wired or wirelessly, typically via Bluetooth - registered trademark) to the personal equipment 30C, i.e., the visual display device for subtitles for the hearing impaired, may include the integrated screen of the tablet / smartphone / smartwatch, the projector of the AR glasses (or headset), or a holographic projector or device connected to the personal equipment 30C.Thus, the subtitles for the hearing impaired (or sign language component) provided by the personal device 30C complement or "augment" the video provided by the shared device 20 (via its display device 21). Devices such as VR headsets, AR glasses, holographic devices, retinal implants, and contact lenses allow these subtitles for the hearing impaired to be superimposed onto the video provided by the shared device 20. Regarding the sign language component, it can be video-based or metadata-based, enabling the generation and animation of 3D objects, such as avatars. The resulting video (video or avatar) can be added via a hologram displaying signs next to the shared display device 21 (e.g., the TV), or added directly to the main video, or indirectly by masking a portion of the video and displaying a hologram over the mask.

[0060] When the video played back by the common equipment 20 includes standard subtitles, subtitles for the hearing impaired can be displayed by overlaying or masking the standard subtitles. Alternatively, the subtitles for the hearing impaired can be superimposed on the video in a location separate from that of the standard subtitles.

[0061] Viewer D may have a slight hearing impairment requiring only specific audio processing of the main audio track, for example, increased volume of the entire track or of the dialogue only (if the dialogue is encoded as separate audio objects), and / or modification of the frequency response curve to compensate for the viewer's hearing loss at certain frequencies. The personal equipment 30D may be similar to the personal equipment 30B, for reproducing the main audio track modified by the audio processing.

[0062] Of course, only some of these spectators with disabilities can be present. Other types of spectators may also exist, including:

[0063] - a spectator with reduced visual impairment requiring visual rendering adapted, for example, by using enhanced contrast and / or a specific colorimetric display. Personal equipment 30C (visual rendering) can be used, preferably AR headsets or glasses, to replace the main video with the one rendered by this personal equipment.

[0064] - a dyslexic viewer wishing to watch the audiovisual content (for example (in the original version) with dyslexic subtitles. The personal equipment 30C (visual restoration) can be used again to supplement the video restored by the shared equipment 20.

[0065] - a spectator with any other type of disability, including a cognitive disability, for which either the main tracks undergo appropriate processing, or one or more additional tracks are restored.

[0066] A viewer without a disability (for example A) may wish to view standard subtitles independently. They can then use personal equipment similar to the 30C equipment (visual display).

[0067] Furthermore, a viewer may wish to combine several of the needs mentioned above, and consequently use their personal equipment for the reproduction of several so-called additional components.

[0068] Figure 2 illustrates, using a flowchart, the general steps of a process according to embodiments of the invention. This process allows the reproduction of audiovisual content made up of multiple components for shared or collective consumption of the content adapted to the disability of each viewer.

[0069] The Figure process is implemented at the level of a personal equipment 30 for the spectator concerned.

[0070] As a preliminary step, the audiovisual content is launched by the main user (viewer A) on the shared equipment 20. To do this, the user connects to an audiovisual content delivery service (e.g., a VOD service) using a user account. The user selects audiovisual content in the conventional manner, leading to the establishment of an initial audiovisual session between the shared equipment 20 and the content server 10 to obtain the audiovisual content, i.e., its main audio and video components and, optionally, a standard subtitling component.

[0071] The multimedia (or audiovisual) session links these two devices for the provision of audiovisual content. It can be identified directly by a unique session number known to the content server or indirectly by an identifier of the common device 20 (for example IP address or MAC address) or of the primary user (for example user account).

[0072] The components obtained are reproduced on the common equipment 20, typically the video component of the audiovisual content using the common display device 21 and the audio component of the audiovisual content using the common sound device 22. The spectators A, B, C, D share this audiovisual content together.

[0073] Spectator B, C, or D then wishes to obtain additional information taking into account their disability. Several spectators can proceed in the same way.

[0074] To do this, in step 200, he starts his personal equipment 30 and launches the multimedia application it contains, allowing access to the content provision service from the content server 10.

[0075] At step 210, the personal equipment 30 discovers the first ongoing session, providing the shared audiovisual content.

[0076] In one embodiment, an action by the viewer on their personal device 30 (for example, a "sync" button) triggers a discovery procedure to locate the shared device 20 in which a session is in progress. A discovery protocol such as mDNS or Bonjour (trade name) can be used. Through this discovery protocol, the various devices on the local network 50 describe their capabilities and identifiers or URLs enabling communication between devices using a communication protocol (for example, https). The personal device 30 thus automatically retrieves the session identifier of the first ongoing session or any corresponding identifier.

[0077] In another embodiment, the personal device 30 may request the relevant viewer (the user of this device) to enter the session ID of the first current session or any corresponding identifier. This identifier may be obtained from the shared device 20: a number may be displayed on the screen 21 and then entered by the viewer on their personal device 30; alternatively, a QR code or equivalent may be displayed on the screen 21 and scanned by the personal device 30. The personal device 30 thus obtains the session ID of the first current session or any corresponding identifier.

[0078] In another embodiment, the spectator in question logged in at step 200 with the same user account as on the common equipment 20. Typically, two distinct profiles of the same user account can be used respectively on the personal equipment 30 and on the common equipment 20. This information makes it possible to indirectly identify the first session in progress.

[0079] Once the first audiovisual session has been identified (directly or indirectly), the personal device 20 obtains, at step 220, information representative of the disability of the viewer concerned. This information makes it possible to determine the component(s) of the shared audiovisual content that the personal device will reproduce. for the viewer in question. For example, this information could simply indicate "visual impairment" (which might include an audio description component) or "hearing impairment" (which might include subtitles for the hearing impaired). Alternatively, it could directly indicate the desired component(s), for example, "audio description" or "subtitles for the hearing impaired."

[0080] In one embodiment, the viewer enters this information in a dedicated interface provided by the multimedia application running on the personal equipment 30. In another embodiment, this information is predefined in the user account or in the profile used on the personal equipment 30.

[0081] The order of steps 210 and 220 can be reversed.

[0082] At step 230, a second audiovisual session is launched between the personal equipment 30 and the content server 20 using the identification obtained in step 210, in order to obtain components, called "additional", of the same audiovisual content as that shared on the common equipment 20.

[0083] To do this, the personal device 30 can request the content server 20 to establish a mirror audiovisual session of the first identified audiovisual session, indicating, where applicable, either the type of disability or the desired component(s). Alternatively, the desired disability or additional component(s) can be retrieved by the content server from the user account or profile used to establish the second audiovisual session.

[0084] The “mirror” session is understood in the sense that it relates to the same audiovisual content, even if the component(s) transmitted are not necessarily the same, and that means of synchronization between the two sessions are put in place.

[0085] In a "local synchronization" embodiment, the common equipment 20 exposes an HTTPS "status" method which provides, upon each call from a personal device 30, the following information: the content played in the main session (for example, an identifier), the playback state (playing, paused), the playback position, and optionally a timecode (or timestamp). The personal device can thus regularly (for example, every 1 / 10 of a second) call this method and proceed to play the components of the second audiovisual session in a synchronized manner (according to the playback position of the main session). In one embodiment, the common equipment 20 can also notify each personal device in real time of each change of state. This allows the personal devices 30 to synchronize their respective playbacks on the local network as quickly as possible.

[0086] In a "server synchronization" embodiment, the common equipment 20 publishes, in real time on the content server, all of its states and state changes. The content server can then transmit, still in real time (for example via websocket), these states and state changes to the personal devices 30 in order to allow them to synchronize (the second session) with the audiovisual content shared on the common device 20. Alternatively, the personal devices 30 can regularly call the content server to obtain the state changes, and thus proceed with their synchronization.

[0087] Once the second audiovisual session is established, the personal equipment 30 receives, at step 240, one or more components of the shared audiovisual content.

[0088] The transmitted component(s) may be limited to the "additional" components to be reproduced, typically those corresponding to the disability indicated in step 220. Streaming techniques ("master playlist," "media playlist," or "manifest" in HLS ("HTTP Live Streaming" by Apple - trade name) or "MPD" (for "Media Presentation Description") in MPEG-DASH) allow, for example, the personal device 30 to request only one or more tracks of the audiovisual content. In a more bandwidth-intensive variant, the entire audiovisual content is transmitted in step 240 (the choice of "additional" components to be reproduced being made by the personal device).

[0089] The "additional" component(s), corresponding to the disability of the spectator concerned, are then returned to him by his personal equipment 30 at step 250, allowing him to increase the audiovisual content shared with this additional information, typically audio description or subtitles for the hearing impaired, without disturbing other spectators.

[0090] This playback is performed in synchronization with the common equipment 20, meaning that the data of this / these additional component(s) corresponding to a given playback position (or timestamp / timecode) are played back by the personal equipment 30 at the same time as the common equipment 20 plays back the data of the main audio and video components corresponding to the same playback position (or timestamp / timecode). In other words, the personal equipment 30 is synchronized in timecode with the audiovisual content played on the common equipment 20.

[0091] This completes the process of [Fig.2].

[0092] In one embodiment, the personal device 30 can control the playback (pause, rewind, etc.) of the shared audiovisual content. In this case, mechanisms are implemented for publishing state changes to the content server 10 (or between shared devices 20 and personal devices 30) and for rebroadcasting these changes to other devices "linked" to the same shared content. These other devices can then control their local playback in a manner synchronized with these state changes.

[0093] By way of illustration, a simple implementation of these mechanisms can be based on Apple's Shareplay technology (trade name). Indeed, this technology incorporates a synchronization protocol between Apple devices for playing multimedia content.

[0094] In this example, shared multimedia content is being played on an Apple TV (trade name). Viewers with disabilities are equipped with Apple devices. A FaceTime (trade name) call is initiated between these multiple devices, including the Apple TV. The SharePlay option available on the Apple TV is selected to play the same multimedia content on the different devices. A mode present on these devices and configured with the user's disability then plays only the audio track corresponding to their disability. The SharePlay technology ensures that the playback of audio description on one device, the playback of closed captions for the hearing impaired on another device, and the audio and video playback on the Apple TV are synchronized.

[0095] Returning to [Fig. 1], viewer B is equipped with a smartphone 30B to which they connect an open-back headset. They launch the multimedia application to connect to the content server 10, then select the audio description track only, a track different from the main tracks played on the shared equipment. The smartphone 30B thus plays back the audio commentary only, which is superimposed on the sound and image from the shared equipment 20. Here, the commentary blends with the sound / audio emitted by the speaker 22.

[0096] In an embodiment offering improved comfort, the smartphone 30B can also reproduce the main audio component with a reduced volume, so as to compensate for the attenuation, by the "open" headphones, of the sound / audio emitted by the speaker 22.

[0097] Similarly, viewer C is equipped with a tablet or AR 30C headset. They launch the multimedia application to connect to the content server 10. This application is configured to accommodate viewer C's disability and therefore automatically selects the subtitle track for the hearing impaired (or a sign language track), which is different from the main tracks displayed on the shared equipment. The AR 30C tablet or headset then displays the subtitles alone, which complement the sound and image from the shared equipment 20, or even overlay the image displayed on the shared screen 21 (thus merging the visual outputs of the shared and personal equipment).

[0098] In an embodiment where conventional subtitles are displayed on the common screen 21, the AR headset of viewer C can mask these conventional subtitles and display the subtitles for the hearing impaired over the masking. Alternatively, the subtitles for the hearing impaired are displayed in a separate location from the subtitles. classic titles, with dynamic adaptation of the display on the AR headset when viewer C moves their head.

[0099] Like viewer B, viewer D is equipped with a 30D smartphone to which he connects open-back headphones. He launches the multimedia application to connect to the content server 10, then selects the main audio track for which he wants audio processing, for example, playback of the track at a high volume or louder playback of an audio element, typically dialogue, and / or audio compensation at frequencies where viewer D has hearing loss. The 30D smartphone then plays back the audio track modified by the audio processing.

[0100] Similarly, for a viewer with reduced visual impairment, AR glasses can recover the main video component, perform video processing on it (increase contrast, adapt colorimetry) and restore this processed component to the viewer.

[0101] For a dyslexic viewer, the same approach as for customer C can be used, in which the subtitles displayed are those for dyslexic.

[0102] The present invention, as described above, allows each user to enhance or correct their shared audiovisual content consumption experience using their own consumer electronics, without disturbing other viewers or requiring the installation or purchase of additional equipment. This enhancement, provided by the additional component, allows the user to avoid being confined to a purely individual consumption experience.

[0103] The mechanisms described above ensure synchronization of multiple devices (the TV and individual devices) during a multimedia playback session.

[0104] Of course, the present invention is not limited to the embodiments described above by way of example; it extends to other variants. Other embodiments are possible.

Claims

Demands

1. A method for reproducing audiovisual content consisting of multiple components comprising the following steps: establishing (230) a second continuous transmission or streaming session between a personal device (30B, 30C, 30D) belonging to a viewer and a content server (10) to continuously transmit to the personal device at least one so-called "additional" component of audiovisual content that is also the subject of a first streaming session established between a common device (20) belonging to several viewers (A, B, C, D) and the content server and of which video and audio components transmitted continuously to the common device are reproduced, by the common device, respectively using a common display device (21) and a common sound device (22), and reproducing (250) the additional component of the audiovisual content using a reproduction device specific to the personal device,The proprietary playback device is configured to superimpose the additional component onto the video or audio component, so that the playback of the additional component is augmented, through sound or visual fusion, in a manner synchronized for the viewer of the personal equipment, with the playback of the video or audio component by the common display or sound device.

2. A method according to claim 1, comprising: the establishment (230) of several streaming sessions between several personal devices (30B, 30C, 30D) to different viewers and the content server (10) to obtain data from the audiovisual content, and the rendering (250), using a rendering device specific to each personal device and configured to superimpose an additional component on the video or audio component, of an additional component of the audiovisual content such that the rendering of the additional component augments, by sound fusion or visual fusion, in a synchronized manner for the viewer of the personal device concerned, the rendering of the video or audio component by the common display or sound device.

3. A method according to claim 1, wherein the additional component is a subtitling component for the hearing impaired restored, for the viewer of the personal equipment, in superposition of the video component restored by the common display device.

4. A method according to claim 3, wherein the rendering device specific to the personal equipment is one of: a virtual reality headset or glasses, an augmented reality headset or glasses, a holographic device.

5. A method according to claim 1, wherein the additional component is an audio description component rendered, for the viewer of the personal equipment, in superposition of the audio component rendered by the common sound device.

6. A method according to claim 5, wherein the playback device specific to the personal equipment is one of: an open-back headset, a hearing aid, a bone conduction audio device, an augmented reality headset or glasses equipped with an open-back audio system.

7. A method according to claim 1, wherein the additional component is one of: a subtitle component for the hearing impaired, an audio description component, a sign language component, a subtitle component for dyslexic viewers, the audio component in which case the playback by the playback device specific to the personal equipment applies at least one audio processing to it compared to playback by the common sound device, the video component in which case the playback by the playback device specific to the personal equipment applies at least one video processing to it compared to playback by the common display device.

8. A method according to any one of the preceding claims, wherein establishing a second streaming session comprises one of the following steps: - initiating, by the personal equipment (30), a discovery procedure to discover the common equipment (20) on a local network (50) and obtain from the common equipment identification information for the first streaming session, - entering, by the viewer concerned on their personal equipment (30), identification information for the first streaming session, - connecting the personal equipment to the content server (10) using the same user account as that used on the common equipment to obtain the audiovisual content components, followed by a step (230) of requesting the content server to establish a mirror streaming session of the first session streaming.

9. A method according to claim 1, wherein the video component and the audio component are not rendered by the personal equipment.

10. Personal equipment (30B, 30C, 30D) comprising a proprietary playback device and configured to: establish a second continuous transmission or streaming session with a content server (10) to obtain, by continuous transmission, at least one so-called "additional" component of audiovisual content that is also the subject of a first streaming session established between a common device (20) with several viewers (A, B, C, D) and the content server and of which video and audio components transmitted continuously to the common device are played back, by the common device, respectively by means of a common display device (21) and a common sound device (22), and play back the additional component of the audiovisual content by means of the personal equipment's proprietary playback device,The proprietary playback device is configured to superimpose the additional component onto the video or audio component, so that the playback of the additional component is augmented, through sound or visual fusion, in a manner synchronized for the viewer of the personal equipment, with the playback of the video or audio component by the common display or sound device.

11. Audiovisual system (1) comprising: a content server (10), a common device (20) for several viewers (A, B, C, D) and comprising a common display device (21) and a common sound device (22), the common device being configured to establish a first continuous transmission or streaming session with the content server to obtain, by continuous transmission, components of audiovisual content, and to render a video component of the audiovisual content using the common display device and an audio component of the audiovisual content using the common sound device, and at least one personal device (30B, 30C, 30D) belonging to one of the viewers and comprising a personal rendering device, the personal device being configured to establish a second streaming session with the content server to obtain, by continuous transmission, at least one so-called "additional" component of the same audiovisual content, and to render the additional component of the audiovisual content using the own rendering device configured to superimpose the additional component onto the video or audio component, so that the rendering of the additional component increases by sound fusion or visual fusion, in a synchronized manner for the viewer of the personal equipment, the rendering of the video or audio component by the common display or sound device.

Images