Method for managing an audio stream using a camera and associated decoding equipment

Abstract

The invention relates to a method for managing an audio stream played by a group of audio playback devices, said group being arranged in a given location. According to the invention, the method comprises the steps of: Detecting, on at least one image, the user(s) present in the image and deducing, for each of said users, at least one characteristic piece of information regarding the user's position in said image; Determining, at least from the various characteristic pieces of information, an optimal bearing angle (βopt); Providing mixing means distributing the audio stream among the different audio playback devices in the group, a characteristic value of the optimal bearing angle so that the mixing means distribute the audio stream at least according to said value. FIGURE IN THE ABRIDGED DIAGRAM: Fig. 2

Description

Title of the invention: Method for managing an audio stream using a recording device and associated decoder equipment

[0001] The invention relates to the field of audio reproduction via a group of audio reproduction equipment.

[0002] BACKGROUND OF THE INVENTION

[0003] It is now common practice in modern home multimedia installations to associate a set-top box with a group of audio playback devices comprising different audio playback devices in order to improve the acoustic experience of a user. Indeed, the user is thus much more "enveloped" in the sound broadcast by the group of audio playback devices than if said sound were broadcast by a single audio playback device.

[0004] Usually, the group of audio reproduction equipment is associated with mixing means which make it possible to distribute the channels of a multichannel audio stream received by the decoder equipment between the different audio reproduction equipment.

[0005] The Dolby ATMOS system (registered trademark) thus optimizes the rendering of multichannel sound according to the arrangement of the audio reproduction equipment in relation to a theoretical listening position of the user in the room. For example, if the audio reproduction group comprises two audio reproduction equipment arranged to the right and left of the decoder equipment, the system will consider that the user is located between the two audio reproduction equipment.

[0006] Thus, this type of system does not take into account the actual position of the user. Therefore, the user's acoustic experience is in reality only of good quality if the user is close to the theoretical listening position.

[0007] SUBJECT OF THE INVENTION

[0008] An aim of the invention is to propose a method for managing an audio stream which makes it possible to improve the acoustic experience of the user.

[0009] An aim of the invention is to propose corresponding decoder equipment. Summary of the invention

[0010] In order to achieve this aim, a method is proposed for managing an audio stream read by at least one group of audio playback equipment comprising at least two audio playback equipment, said group being arranged in a given location.

[0011] According to the invention, the method comprises at least the steps of: - Detect on at least one image acquired by at least one camera of the given location, the user(s) present in the image and deduce from this, for each of said users, at least one piece of information characteristic of the position of the user considered in said image, - Determine at least from the various characteristic information, an optimal bearing angle defined by the angle formed between: • An axis of the camera, and • An axis along which a sound reproduced by the group of restitution equipment audio propagates to reach the different users who were present in the image, - Provide mixing means distributing the audio stream between the different audio reproduction equipment in the group with a characteristic magnitude of the optimal bearing angle so that the mixing means distribute the audio stream at least according to said value.

[0012] In this way, the invention makes it possible to adapt to the actual positions of the users present in the given location, including in the case where there are several users. By providing a bearing angle value linked to the actual position of the different users in the given location, the mixing means can adapt the audio stream so as to improve the acoustic experience of the different users.

[0013] The invention therefore makes it possible to obtain a spatialized acoustic rendering adapted to the position of the users present in the given location.

[0014] Advantageously, the invention does not require a calibration step prior to the use of the group of sound reproduction equipment.

[0015] Optionally, the camera generates a new image of the given location at regular intervals, and the optimal bearing angle value is recalculated for each new image so that the mixing means distribute the audio stream between the different audio reproduction equipment of the group on the basis of this new optimal bearing angle value.

[0016] Thus the invention adapts dynamically to the different users. The invention makes it possible in particular to take into account not only the actual position of the users, but also the presence of several users but also the movement of said users.

[0017] This further improves the acoustic experience of users.

[0018] This gives a spatialized acoustic rendering dynamically adapted to the position of the users present in the given location.

[0019] Optionally, the characteristic information is at least one abscissa in the image.

[0020] Optionally, the characteristic information is information characteristic of the position of the user's face in the image.

[0021] Optionally, the optimal bearing angle is linked to an average position of the different users appearing in the image.

[0022] Optionally, the optimal bearing angle is linked to a spatial average of the position of the different users appearing on the image or to an angular average of the position of the different users appearing on the image or to the positions of the two users furthest from each other on the image.

[0023] Optionally, a dispersion angle is also estimated which characterizes the dispersion of the different users present in the image and the mixing means are provided with a magnitude characteristic of the dispersion angle so that the mixing means distribute the flow at least as a function of said value.

[0024] Optionally, the audio stream is a multi-channel audio stream and the group of audio rendering equipment comprises at least one audio rendering equipment less than the number of channels of the multi-channel audio stream.

[0025] Optionally, the optimal bearing angle is also estimated taking into account the attention of the users present on the image.

[0026] Optionally, the orientation of the heads of the users present in the image is taken into account to determine the optimal bearing angle [3opt.

[0027] Optionally, we take into account the potential falling asleep of users present in the image to determine the optimal bearing angle [3opt.

[0028] Optionally, the mobility of users present in the image is taken into account to manage the audio stream.

[0029] Optionally, the mixing means are also provided with the distance of the users from the installation.

[0030] Optionally, the mixing means distribute the audio stream between the different audio reproduction equipment of the group based on one or more sets of at least one precalculated audio parameter.

[0031] Optionally, the audio stream is a multi-channel audio stream and the group of audio rendering equipment comprises at least one audio rendering equipment less than the number of channels of the multi-channel audio stream.

[0032] The invention also relates to an installation making it possible to implement the method as mentioned above, comprising at least two audio reproduction devices, means for receiving at least one audio stream, the mixing means making it possible to distribute the channel(s) of the audio stream between the audio reproduction devices, a camera and means for analyzing at least one image provided by the camera.

[0033] Optionally the installation is a decoder equipment.

[0034] The invention also relates to a computer program comprising instructions which cause an installation such as the aforementioned to execute the method such as the aforementioned.

[0035] The invention also relates to a computer-readable recording medium, on which the computer program as mentioned above is recorded.

[0036] Other characteristics and advantages of the invention will emerge from reading the following description of the particular non-limiting embodiments of the invention. Brief description of the drawings

[0037] The invention will be better understood in light of the description which follows with reference to the appended figures among which:

[0038] [Fig-1] [Fig.l] represents an installation according to a first embodiment of the invention;

[0039] [Fig.2] [Fig.2] is a schematic top view of a given location incorporating the installation as illustrated in [Fig.l], two users being present in the given location;

[0040] [Fig.3] [Fig.3] schematically illustrates the main steps implemented by the installation illustrated in [Fig.l] to manage an audio stream received by said installation;

[0041] [Fig.4] [Fig.4] represents an installation according to a second embodiment of the invention. DETAILED DESCRIPTION OF THE INVENTION

[0042] With reference to Figures 1 and 2, the installation according to a first embodiment is an installation comprising an audio system associated with a group of audio reproduction equipment. The installation is arranged in a given location and for example in a room of a dwelling.

[0043] The installation is therefore intended at least to broadcast sound to users present in the room.

[0044] The audio system is for example a Hi-Fi system or decoder equipment 10.

[0045] The group of audio reproduction equipment is for example integrated into the audio system.

[0046] For example, the decoder equipment 10 is a decoder box. Optionally, the decoder box is a television decoder box. For example, the television decoder box is a VSB television decoder box for “Video Sound Box” (registered trademark) and for example a VSB4 television decoder box.

[0047] The decoder equipment 10 comprises a communication interface, an interface by means of which the decoder equipment acquires in service at least one audio input stream, and for example an audio / video input stream, which can come from one or more broadcasting networks. The broadcasting networks can be of any type. Thus, according to a first variant, the broadcasting network is a television network. via satellite, and the decoder equipment 10 receives the input stream through a satellite dish. According to a second variant, the broadcasting network is an Internet connection and the decoder equipment 10 receives the input stream through said Internet connection. According to a third variant, the broadcasting network is a digital terrestrial television (DTT) network or a cable television network. Overall, the broadcasting network can be of various sources: satellite, cable, IP, DTT (Digital Terrestrial Television network), locally stored audio / video stream, etc.

[0048] The decoder equipment 10 is therefore optionally provided with an output allowing it to be connected to video or audio / video reproduction equipment such as a television which is therefore here external to said decoder equipment 10.

[0049] Furthermore, the decoder equipment 10 is provided with processing means making it possible, among other things, to process the input stream. The processing means comprise, for example, a processor and / or a calculator and / or a microcomputer... In the present case, the processing means comprise a processor.

[0050] The audio reproduction equipment is for example loudspeakers integrated into the decoder equipment 10. According to a particular embodiment, the decoder equipment 10 comprises at least two loudspeakers and for example at least three loudspeakers and for example at least four loudspeakers. Optionally, the decoder equipment 10 is equipped with three loudspeakers 101, 102, 103 arranged on three successive sides of the decoder equipment and a fourth loudspeaker 110 arranged on the bottom of the decoder equipment 10. The three loudspeakers 101, 102, 103 of the sides are wideband while the bottom loudspeaker 110 is dedicated to the reproduction of low frequencies. This configuration with four loudspeakers is commonly called a “3.1 system”.

[0051] Furthermore, the installation comprises mixing means making it possible to distribute at least one channel of the input stream between the different audio reproduction equipment 101, 102, 103, 110. This makes it possible to generate a spatialization effect for the user(s) present in the room.

[0052] Optionally, the audio stream of the input stream is a multi-channel audio stream and the group of audio rendering devices comprises at least one audio rendering device less than the number of channels of the multi-channel audio stream. For example, the audio stream is a five-channel stream.

[0053] Preferably, the mixing means are integrated into the decoder equipment 10. For example, the mixing means are integrated into the processing means.

[0054] In particular, the mixing means comprise a memory on which a market library is stored and / or communicate remotely (via for example the communication interface of the decoder equipment) with a market library. The market library is for example the library Dolby ATMOS (registered trademark).

[0055] Such a library allows the mixing means to ensure the distribution of the flow from input data indicated to the library.

[0056] Such mixing means (and the associated library) are already known from the prior art and usually make it possible to distribute the channels of the audio stream between the different audio reproduction equipment according to input data provided by the user having placed the installation in the room during the initialization of the audio reproduction equipment.

[0057] In the context of the invention, the input data will be provided directly by the installation itself, for example by the decoder equipment 10, to the mixing means. These input data will be described below.

[0058] According to another aspect, the installation comprises a camera 120. The camera 120 is for example integrated into the decoder equipment 10. The camera 120 is for example arranged at the level of the side carrying the audio reproduction equipment 103 and framed by the two other sides also carrying the audio reproduction equipment 101, 102. The camera 120 is thus centered in the decoder equipment 10.

[0059] The decoder equipment 10 is arranged so that its side carrying the camera 120 is turned towards the given location, in this case the interior of the room. The camera 120 is for example a camera.

[0060] The installation also comprises image analysis means. Said means are for example integrated into the decoder equipment 10 and for example integrated into the processing means of the decoder equipment 10.

[0061] The image analysis means are configured to analyze at least one image provided by the camera 120, detect at least one item of information characteristic of the position of each of the users present in the image and transmit this position information (i.e. the aforementioned input data) to the mixing means so that said mixing means can distribute the channels of the audio stream between the audio reproduction equipment in view of this information.

[0062] With reference to [Fig.3], a particular implementation of a method for managing the audio stream by the installation described above will now be detailed.

[0063] During a prior step, the camera 120 acquires at least one image of the room.

[0064] During a first step 310, the image analysis means detect on the image the user(s) present on the image. For example, the analysis means detect the faces present on the image to detect the users.

[0065] The analysis means (or other means of the installation such as for example the processing means) deduce therefrom, for each of said users, at least one in characteristic formation of the position of the user considered in said image and therefore in the present case of the position of the user's face.

[0066] According to a first option, the installation (here the decoder equipment) is based on the Viola-Jones method, which is a known algorithm for detecting faces in an image. This algorithm takes an image as input and produces a list of rectangles encompassing each detected face. For more details, one can refer, for example, to the internet page https: / / fr.wikipedia.org / wiki / M%C3%A9thode_de_Viola_et_Jones or to the article Paul Viola and Michael Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features”, IEEE CVPR, 2001.

[0067] For example, the rectangle encompasses the face and in particular the coordinates of the eyes, nose and mouth.

[0068] According to a second option, the installation (here the decoder equipment) relies on a neural network to detect the position of the faces in the image. The neural network is for example integrated into the decoder equipment and for example into the analysis means and / or the processing means. The neural network is for example the BlazeFace neural network (registered trademark) or MobileNet (registered trademark). For more details concerning BlazeFace, one can refer for example to the article BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs, V. Bazarevsky, Y. Kartynnik, A. Vakunov, K. Raveendran, M. Grundmann - Computing Research Repository - 2019 (https: / / arxiv.org / abs / 1907.05047). For more details about MobileNet, one can refer for example to the article MobileNets: Efficient Convo-lutional Neural Networks for Mobile Vision Applications, AG Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weynand, M. Andreetto, H.Adam -Computing Research Repository - 2017 (https: / / arxiv.org / abs / 1704.04861). .

[0069] The neural network takes an image as input and produces a list of rectangles encompassing each detected face. For example, the rectangle encompasses the face as well as the coordinates of the eyes, nose and mouth.

[0070] Whether one considers the first option, the second option or another option for detecting the faces of users, the installation (and for example the decoder equipment) then constructs a list of positions of the faces in the image, by taking the coordinates of a given point in each detected rectangle. The information characteristic of the position of the users present in the image are therefore coordinates (abscissa, ordinate), coordinates linked to the image reference system.

[0071] For example, the installation constructs a list of positions of the faces in the image, by taking the coordinates of the center of each detected rectangle. Alternatively, the installation constructs a list of positions of the faces in the image, by taking the coordinates of the barycenter of a triangle formed by the positions of the eyes and the mouth in said rectangle.

[0072] The first step described 310 thus makes it possible to establish a list of positions of the faces in the image comprising for each face a set of coordinates linked to the face in the image. More precisely, the list is a list of positions of faces detected in the image, each position being represented by a pair (x, y) of values ​​representing the coordinates in pixels of the point associated with the face in the image.

[0073] Subsequently, to simplify the formulas which follow, we assume that the point with coordinates (0, 0) corresponds to the center of the image. However, this postulate is not limiting and the point with coordinates (0, 0) may correspond to another point of the image than its center, such as for example one of the corners of the image.

[0074] During a second step 320, the installation (and more precisely here the decoder equipment 10) determines from the list of coordinates established in the first step 310, an optimal bearing angle |3opt defined by the angle formed between: - A first axis 11 of the camera 120, and - A second axis 12 along which a sound reproduced by the audio reproduction equipment group propagates to reach the different users who were present in the image.

[0075] It is recalled that in general a bearing angle designates an angle between the first axis 11 of the camera 120 and another second axis 12 belonging to the same horizontal plane as the first axis 11 of the camera 120.

[0076] [Fig.2] represents such a bearing angle [3opt. It is an angle extending between the two aforementioned axes, said axes also belonging to the same horizontal plane.

[0077] The optimal bearing angle [3opt] preferably defines the direction (the second axis 12) in which the sound broadcast by the audio reproduction group is desired to have maximum quality.

[0078] The optimal bearing angle [3opt] is such that the second axis 12 is preferably generally directed towards the center of a group formed by the different users appearing on the image.

[0079] Preferably, the optimal bearing angle [3opt is linked to an average position of the different users appearing on the image.

[0080] According to a first option, the optimal bearing angle [3opt is determined from a spatial average of the position of the different users appearing on the image. An average x of the abscissas linked to each of the faces detected on the image is calculated.

[0081] The optimal bearing angle is then given by the formula: )

[0082] Where:

[0083] popt denotes the optimal bearing angle,

[0084] W denotes the width of the acquired image in pixels (typically W = 1920), and

[0085] a denotes the field angle of the camera 120 in a horizontal plane (typically a = 120°).

[0086] According to a second option, the optimal bearing angle |3opt is determined from an angular average of the position of the different users appearing in the image. We thus calculate a bearing angle [3 for each face detected from the abscissa x linked to said face by:

[0087] fi = tan'1 )

[0088] The optimal bearing angle [3opt is then given by the average of the different bearing angles [3. According to a third option, the optimal bearing angle [3opt is determined from the minimum xmin and maximum xmax abscissas linked to the detected faces.

[0089] For this purpose, the corresponding bearing angles [3min and [3max] are calculated using the formula:

[0090] = tan*1 tan# ) [0091 ] = tan*1 ( tanf )

[0092] The optimal bearing angle [3opt is then defined as the average of [3min and [3 max*

[0093] During a third step 325, which is optional, the installation (and here the decoder equipment 10) determines, from the list of coordinates established in the first step 310, a dispersion angle δ which characterizes the dispersion of the different users present on the image.

[0094] [Fig. 2] represents such a dispersion angle δ. This is an angle δ extending between two axes 13, 14 belonging to the same horizontal plane and each intersecting the axis 11 of the camera 120, the two axes 13, 14 also passing through respectively the first and second of the two users present furthest from each other (on the image) of all the users present on the image.

[0095] The dispersion angle ô thus defines the width of the zone in which we want the sound broadcast by the audio reproduction group to have maximum quality.

[0096] Preferably, the dispersion angle δ is linked to a standard deviation of the different positions of the users appearing on the image.

[0097] According to a first option, the dispersion angle ô is determined from a spatial standard deviation of the position of the different users appearing on the image. thus calculates a standard deviation o of the abscissas linked to the detected faces. Then we calculate the dispersion angle ô with the following formula:

[0098] 5 - 2 x tan1 (~tan|)

[0099] This first option gives good results when the users are all close to the axis 11 of the camera 120, but the inventors were able to observe that the dispersion becomes overestimated when certain users are on the edges of the image.

[0100] Thus, according to a variant of the first option, the dispersion angle δ is preferably calculated with the following formula:

[0101] s = 2x 1-tan'^^^tan^ j|

[0102] According to a second version, the dispersion angle δ is linked to an angular standard deviation of the different users appearing on the image. A bearing angle [3 is thus calculated for each face detected from the abscissa x linked to said face as indicated previously.

[0103] The dispersion angle ô is then defined as being the standard deviation of said bearing angles [3 of the detected faces.

[0104] According to a third option, the dispersion angle ô is determined from the minimum xmin and maximum xmax abscissas linked to the detected faces. The bearing angles [3min and [3max] are calculated as indicated previously and then the dispersion angle ô is defined as being the difference of said bearing angles:

[0105] 6 =

[0106] During a fourth step 330, the installation (and for example the decoder equipment 10 such as for example its processing means) provides the mixing means with a characteristic quantity of the optimal bearing angle [3opt. The characteristic quantity of the optimal bearing angle [3opt is here directly the value of the optimal bearing angle [3opt calculated in the second step 320. Furthermore, the mixing means are also provided with a characteristic quantity of the dispersion angle δ. The characteristic quantity of the dispersion angle δ is here directly the value of the dispersion angle δ calculated in the third step 325.

[0107] The optimal bearing angle [3opt and the dispersion angle ô therefore constitute the input data for the mixing means.

[0108] It is understood that these input data are determined solely by the installation without any of the users or the person who set up the installation needing to intervene.

[0109] In a manner known per se, the mixing means then adapt the distribution of the channels of the audio stream as a function of at least these input data. Preferably, the mixing means adapt the distribution of the channels of the audio stream as a function of the less of this input data and the relative position of the audio rendering equipment in the decoder equipment 10.

[0110] It is therefore noted that it is not necessary to modify the existing mixing means to implement the method of the invention since the mixing means already operated from input data provided by the user. The invention, however, provides them with input data that is different from the existing data and much more interesting. This input data provided by the invention can replace and / or complement the input data usually provided to the mixing means.

[0111] Preferably, the preliminary step of acquiring an image of the part is carried out at regular intervals. For example, this step is carried out at an interval of between 0.1 and 3 seconds, and for example between 0.5 and 2 seconds and for example between 1 and 1.5 seconds and is for example one second.

[0112] The different steps described above 310, 320, 325, 330 are consequently also implemented at regular intervals (for example the same as that of the acquisition of the images). Optionally the different steps 310, 320, 325, 330 described above are implemented so as to determine a new optimal bearing angle [3opt and a new dispersion angle ô for each new image transmitted to the analysis means. In this way, the mixing means adapt the distribution of the channels of the audio stream in a regular manner.

[0113] The method described therefore allows dynamic adaptation of the audio stream retransmitted to users.

[0114] Other options are possible.

[0115] According to a first option, the method which has been described also takes into account the attention of the user(s) present on the image to manage the audio flow.

[0116] According to a first proposal, the method which has been described also takes into account the attention of the user(s) present on the image to manage the audio stream, and for example to calculate the optimal bearing angle [3opt. According to a first possibility, the method takes into account the orientation of the head of the users present on the image to determine the optimal bearing angle [3opt.

[0117] In particular, a user is considered to be very attentive if he is facing the installation, in particular towards the decoder equipment 10. Typically, the user will appear here from the front on the image. On the other hand, if the user is not very attentive, he will be visible in profile on the image. It can also be noted that users who turn their backs to the installation are already implicitly ignored since their face is not visible on the image.

[0118] According to this first possibility, the analysis means are configured to not only detect a user on an image but also locate his eyes and his mouth. For example, the analysis methods rely on the BlazeFace neural network already mentioned or on a variant of the Viola-Jones method such as that described in the article "Face Detection Using Modified Viola Jones Algorithm," A. Gupta, Dr. R. Tiwari - International Journal of Recent Research in Mathematics Computer Science and Information Technology - 2015 (https: / / www.paperpublications.org / upload / book / Face%20Detection%20Using%20Modified%20Viola%20Jones%20Algorithm- 164.pdf)

[0119] We know that for a face from the front, the triangle formed by the eyes and the middle of the mouth is an equilateral triangle. Thus when the face turns, the distance between the eyes on the image decreases until it reaches 0 for a face from the side.

[0120] Consequently, the installation (and for example the decoder equipment 10) measures, for each face detected on the image, the ratio between the interocular distance and the eye-mouth distance and assigns a weight to each face according to this ratio.

[0121] For example:

[0122] w _ ^d(LE.RE) M d(LEM)+d(REM) + H°

[0123] or even:

[0124] .., _ d(LE,RE) W ma^d(LEM), d(REM))

[0125] With for the two formulas:

[0126] w the weight assigned to a face present in the image,

[0127] LE, RE and M the respective positions of the left eye, the right eye and the mouth of said face in the picture,

[0128] d(P,Q) denotes the distance between points P and Q,

[0129] and w0 is a predetermined constant which designates the weight attributed to the profile faces (we can thus choose for example w0 = 0 or w0 = 0.25).

[0130] The optimal bearing angle [3opt (and optionally the dispersion angle ô) is then determined by taking into account the weight of each face present in the image. For example, the optimal bearing angle |3opt is linked to an average weighted by the weights of each face present in the image (instead of being linked to a direct average). For example, the dispersion angle ô is linked to a standard deviation weighted by the weights of each face present in the image (instead of being linked to a direct standard deviation).

[0131] According to a second possibility, the method detects potential drowsiness of users present in the image to determine the optimal bearing angle [3opt. The state of the art includes numerous methods for detecting drowsiness, in particular to detect the level of attention of car drivers. We can therefore refer for example to the article “A Deep Learning Approach To Detect Driver Drowsiness,” M. Tibrewal, A. Srivastava, Dr. R. Kayalvizhi - International Journal of Engineering Research & Technology - 2021 (https: / / www.ijert.org / a-deep-learning-approach-to-detect-driver-drowsiness). Thus, for each face detected in the image, we determine whether the user is drowsy or not and we remove from the list obtained at the end of the first step 310 the coordinates associated with drowsy users. Alternatively, we assign a lower weight to drowsy users than to other users. For example, we assign drowsy users a weight w divided by two compared to the weight calculated according to one of the two formulas indicated previously.

[0132] According to a second possibility (which can be combined with the first), the method which has been described takes into account even more the mobility of the users present on the image to manage the audio flow, and for example to calculate the dispersion angle ô (in order to optionally have a larger dispersion angle ô if one or more users move quickly).

[0133] For example, the installation (and for example the decoder equipment 10) concatenates the lists obtained at the end of the first step 310 for several successive images (for example for the last N images, N being between 5 and 15, N being for example equal to 10). The second step 320 and the third step 325 are then executed using the concatenated list.

[0134] According to another example, the installation (and for example the decoder equipment) creates two lists: a list corresponding to the faces detected in the last captured image and a concatenated list corresponding to the last N images (for example N being between 5 and 15, N being for example equal to 10). Then the optimal bearing angle [3opt is determined from the list linked to the last captured image and the dispersion angle ô is determined from the concatenated list. From then on, the sound reproduction quality is optimal for the current position of the users, but the size of the optimal listening zone is adapted to their mobility in order to maximize the sound reproduction quality if they move.

[0135] According to a second proposal (which can be combined with the first proposal), the method which has been described also takes into account the attention of the user(s) present on the image to manage the audio flow, and for example to calculate another input data than the optimal bearing angle [3opt.

[0136] Thus, according to a third possibility (which can be combined with the first or the second possibility), the method which has been described takes into account, to manage the audio flow, the distance of the users to the installation and more particularly here to the decoder equipment 10.

[0137] According to this third possibility, the image analysis means are configured to not only detect a user in an image but also locate his eyes and mouth. For example, the analysis means rely on the neural network BlazeFace or the variation of the Viola-Jones method.

[0138] Then for each detected user, we estimate the distance from the user by the following formula:

[0139] dist = S!^ “ref

[0140] With:

[0141] dist the distance in meters between the detected user and the camera 120,

[0142] ME is the position of the midpoint between the two eyes,

[0143] M is the position of the mouth,

[0144] d(ME,M) denotes the distance in pixels between the mouth M and the midpoint between the two eyes ME,

[0145] dref is a predetermined constant corresponding to the value of d(ME,M) for a person of average height (for example 1 meter 70) located at a given distance from the camera 120 (for example at a distance of 1 meter). For example, we choose dref = 40 pixels / meters.

[0146] Input data related to this distance is then provided to the mixing means. This input data can, for example, allow the mixing means to adjust the sound level generated by each audio reproduction device and / or to refine the way in which the sound is transcribed with respect to the position of the user(s) in the given location (the further a user is from the installation, the smaller the gap between the audio reproduction devices will seem to him: the mixing means can take this into account to consequently exaggerate the spatialization effect for the same subjective rendering for the user).

[0147] For example, the mixing means are provided with input data characteristic of a first average of the estimated distances. The input data is directly said first average or is data characterizing the geometry of the audio reproduction equipment in the installation and linked to said first average.

[0148] As a replacement or in addition, the distance of each user is estimated, then a second average is calculated, weighted (see the possibilities for calculating the weights w mentioned above) of the distances estimated for all the users present in the image. The input data is directly said second average or is data characterizing the geometry of the audio reproduction equipment in the installation and linked to said second average.

[0149] Of course, the invention is not limited to the embodiments described above and variant embodiments can be made without departing from the scope of the invention as defined by the claims.

[0150] The invention will thus apply to any installation allowing sound reproduction of an audio stream and integrating a camera. Preferably, the installation will allow multi-channel sound reproduction of an audio stream.

[0151] The installation will preferably include: - At least two audio playback devices, - Means for receiving at least one audio stream, - Mixing means for distributing the channel(s) of the audio stream between the audio playback devices, - A camera, - Means for analyzing at least one image provided by the camera, detecting at least one piece of information characteristic of the position of each of the users present in the image and transmitting this position information to the mixing means so that said mixing means can distribute the channel(s) of the audio stream between the audio reproduction equipment in view of this information.

[0152] These different elements may be arranged within the same equipment, and for example within decoder equipment, or may form, in part or in whole, separate entities.

[0153] The means for receiving at least one audio stream and / or the mixing means and / or the means for analyzing at least one image may thus comprise, for example, a processor and / or a calculator and / or a microcomputer... which may or may not be common to some, several, or all of these aforementioned means.

[0154] Thus, although here the decoder equipment is a decoder box, the decoder equipment could be any other equipment capable of performing audio decoding, and for example a game console, a computer, a smart TV, a digital tablet, a mobile phone, a digital television decoder, an STB decoder box, etc.

[0155] Similarly, although here the audio reproduction equipment is integrated into the decoder equipment, at least one of the audio reproduction equipment may be an element separate from the decoder equipment. Thus, although here the audio reproduction equipment is a loudspeaker integrated into the decoder equipment, at least one of the audio reproduction equipment may be an externally connected speaker or other equipment equipped with a loudspeaker, for example a sound bar. At least one of the audio reproduction equipment may thus be remote from the decoder equipment in the place where the users are located and not arranged in or in the immediate vicinity of the decoder equipment, at least one piece of information on the position of the remote audio reproduction equipment will then be provided to the mixing means (for example during the installation of the system or during a prior calibration step).

[0156] A different number of audio playback devices may be available than indicated. For example, with reference to [Fig.4], the installation may include two audio reproduction equipment. For example, the two audio reproduction devices may be integrated into the decoder equipment. Optionally, the decoder equipment is equipped with two audio reproduction devices which are arranged on the same side of said decoder equipment, at each of the longitudinal ends of said side. For example, the camera is arranged on the same side as the one integrating the two audio reproduction devices and is arranged between said two audio reproduction devices.

[0157] Similarly, the number of channels in the input audio stream may be different from what has been indicated. The audio stream may therefore have one channel, two channels, three channels, four channels, seven channels, etc.

[0158] Furthermore, the ratio between the channels of the audio stream and the number of audio playback devices may be different from what has been indicated. For example, there may be more audio playback devices than channels in the input stream. Optionally, the installation may include a virtualization system (integrated for example into the mixing means and / or the decoder equipment) making it possible to generate additional channels from the initial number of channels in order to ensure that each audio playback device can broadcast sound.

[0159] Although here it is the faces of the users which are detected on the images, as a replacement or in addition it could be other parts of the users' bodies which could be detected such as for example the users' trunk.

[0160] Although here the dispersion angle and the optimal bearing angle are determined in successive steps, said angles may be determined simultaneously during the same step.

[0161] Although here the optimal bearing angle is transmitted directly (and optionally the dispersion angle directly), the mixing means may be provided with a quantity characteristic of said optimal bearing angle (in parallel with a quantity characteristic of the dispersion angle). For example, the mixing means may be provided with the same information which is characteristic of both the optimal bearing angle and the dispersion angle.

[0162] Although here the mixing means are based on a library, such as the Dolby ATMOS library, to distribute the channel(s) between the different audio reproduction equipment, the mixing means may be based on another technology. For example, the mixing means may be configured to implement the “ambisonics” method. This method makes it possible to place virtual sound sources around a user by calculating gains Gu for each audio reproduction equipment i and each source j as a function of a theoretical position of the user relative to the audio reproduction equipment. For more details on the “ambisonics” method, one may, for example, refer to the internet page https: / / fr.wikipedia.org / wiki / Ambisonie and / or the thesis Numerical methods for sound spatialization, from simulation to binaural synthesis, M. Aussal - Ecole Polytechnique X - 2014 (https: / / pastel.archives-ouvertes.fr / tel-01095801

[0163] By applying this method to one of the implementations of the invention (for example, and in a non-limiting manner, to that of figure 4) to optionally take into account the dispersion angle ô, we calculate for example several gains Gÿ([3) corresponding to several bearing angles [3 in the range going from |3opt - ô / 2 to |3opt + ô / 2 by applying the ambisonic method. Then we apply for each source and each audio reproduction equipment a gain equal to the average Gy of the calculated Gÿ([3). Thus we will send the following signals:

[0164] G^xC + G^xG+G^xD + G^xR

[0165] for left audio playback equipment

[0166] Grc x C + Grg x G + Grd x D + GRR x R

[0167] for the right audio reproduction equipment.

[0168] The installation (and for example the decoder equipment) may include a memory on which is recorded one or more sets of at least one precalculated parameter (namely the audio parameters to be supplied to the different audio reproduction equipment by the mixing means - for example in the context of the “ambisonics” method, the audio parameters will include the aforementioned gains Gij) corresponding to several bearing angles and / or several dispersion angles.For example, the memory may store one or more sets of at least one precalculated parameter corresponding to several general bearing angles (for example, general optimal bearing angles taken from -60° to +60° in steps of 10° - depending on the actually calculated optimal bearing angle, the closest general optimal bearing angle will be estimated and the corresponding set of parameter(s) will be applied) and / or on which one or more sets of at least one precalculated parameter corresponding to several general dispersion angles (for example, a dispersion angle of 10° corresponding to the case of a single user and a dispersion angle of 45° corresponding to a group of several users - depending on the actually calculated dispersion angle (and / or the number of users detected in the image), the closest general dispersion angle will be estimated and the corresponding set of parameter(s) will be applied).

[0169] The installation (and for example the decoder equipment) may apply filtering, in particular temporal filtering, between several successive values ​​of optimal bearing angles and / or dispersion angles. This will make it possible to avoid undesirable effects linked to sudden changes in audio parameters transmitted by the mixing means to the different audio reproduction equipment and based on the angle of optimal bearing and / or dispersion angle (for example, if a face is poorly detected and constantly appears and disappears from one image to the next). The filtering could, for example, be an average (weighted or not) of the last calculated values ​​of optimal bearing angles and / or dispersion angle, or any other low-pass filtering applied to the sequence of the last calculated values ​​of optimal bearing angles and / or dispersion angle.The filtering may be a hysteresis applied to the optimal bearing angle and / or the dispersion angle (i.e. the mixing means will only modify the audio parameters if the difference between the current optimal bearing angle and the optimal bearing angle that was used to calculate the current audio parameters is greater than a first fixed threshold (for example a threshold between 2 and 10° and for example a threshold of 5°) and / or if the difference between the current dispersion angle and the dispersion angle that was used to calculate the current audio parameters is greater than a second fixed threshold (for example a threshold between 5 and 15° and for example a threshold of 10°).Filtering may be implemented by changing a set of audio parameters only if several successive determinations (for example at least two or at least three successive determinations) of optimal bearing angle and / or dispersion angle designate a set of parameters different from the active set.

[0170] Although here the optimal bearing angle is based on the abscissas linked to the detected faces, the optimal bearing angle may be based on the ordinates of said detected faces or may take into account both the abscissas and the ordinates of the detected faces.

[0171] Although here we always rely on a single image to estimate a value (for example the optimal bearing angle and / or the dispersion and / or the distance from a user to the installation), we can also rely on a larger number of images (for a given number N of images) and determine for example the average of the values ​​calculated for these N images. We can for example provide this average as input data to the mixing means or determine an input data to be provided to the mixing means from this average.

Claims

Claims

1. Method for managing an audio stream played by at least one group of audio playback equipment comprising at least two audio playback equipment, said group being arranged in a given location, the method being characterized in that it comprises at least the steps of: - Detecting on at least one image acquired by at least one camera (120) of the given location, the user(s) present in the image and deducing therefrom, for the user or each of said users, at least one piece of information characteristic of the position of the user considered in said image, - Determining at least from at least one piece of characteristic information, an optimal bearing angle (|3opt) defined by the angle formed between: • An axis (11) of the camera, and • An axis (12) along which a sound reproduced by the group of audio playback equipment propagates to reach the user(s) who were present in the image,- Provide mixing means distributing the audio stream between the different audio reproduction equipment in the group with a characteristic magnitude of the optimal bearing angle so that the mixing means distribute the audio stream at least according to said value.,

2. Method according to claim 1, in which the camera (120) generates at regular intervals a new image of the given location, and the optimal bearing angle value ([3opt) is recalculated for each new image so that the mixing means distribute the audio stream between the different audio reproduction equipment of the group on the basis of this new optimal bearing angle value.

3. Method according to one of claims 1 to 2, in which the characteristic information is at least one abscissa in the image.

4. Method according to one of claims 1 to 3, in which the characteristic information is information characteristic of the position of the user's face in the image.

5. Method according to one of claims 1 to 4, in which the optimal bearing angle (|3opt) is linked to an average position of different users appearing on the image.

6. Method according to one of claims 1 to 5, in which the optimal bearing angle ([3opt) is related to a spatial average of the position of different users appearing on the image or to an angular average of the position of different users appearing on the image or to the positions of two users furthest from each other on the image among the different users appearing on the image.

7. Method according to one of claims 1 to 6, in which a dispersion angle (a) is also estimated which characterizes the dispersion of different users present on the image and the mixing means are provided with a magnitude characteristic of the dispersion angle so that the mixing means distribute the flow at least as a function of said value.

8. Method according to one of claims 1 to 7, in which the optimal bearing angle ([3opt) is estimated also taking into account the attention of users present on the image.

9. Method according to claim 8, in which the orientation of the heads of the users present in the image is taken into account to determine the optimal bearing angle (|3opt).

10. Method according to claim 8 or claim 9, in which potential drowsiness of users present in the image is taken into account to determine the optimal bearing angle (|3opt).

11. Method according to one of claims 1 to 10, in which the mobility of users present in the image is taken into account to manage the audio stream.

12. Method according to one of claims 1 to 11, in which the mixing means are also provided with the distance of users to an installation comprising the group of audio reproduction equipment.

13. Method according to one of claims 1 to 12, in which the mixing means distribute the audio stream between the different audio reproduction equipment of the group based on one or more sets of at least one precalculated audio parameter.

14. Method according to one of claims 1 to 13, in which the audio stream is a multi-channel audio stream and the group of audio rendering equipment comprises at least one audio rendering equipment less than the number of channels of the multi-channel audio stream.

15. Installation for implementing the method according to one of claims 1 to 14, comprising at least two audio reproduction devices, means for receiving at least one audio stream, the mixing means for distributing the channel(s) of the audio stream between the audio reproduction devices, a camera and means for analyzing at least one image provided by the camera.

16. Installation according to claim 15, in which the installation is a decoder equipment (10).

17. A computer program comprising instructions which cause an installation according to claim 15 or claim 16 to execute the method according to one of claims 1 to 14.

18. A computer-readable recording medium on which the computer program according to claim 17 is recorded.

Images