VIDEO CONFERENCE SYSTEM, METHOD FOR TRANSMITTING INFORMATION AND COMPUTER PROGRAM PRODUCT

Description

[0001] The invention relates to a video conferencing system configured for bidirectional transmission of video and audio signals using streaming technology between at least three endpoints, each comprising a transmitting device and / or a receiving device for audio streams and / or for video streams, according to claim 1.

[0002] The invention further relates to a method for transmitting information by means of streaming technology via communication channels between at least three endpoints, each comprising a transmitting device and / or a receiving device for audio streams and / or for video streams, according to claim 9.

[0003] Furthermore, the invention relates to a computer program product with program instructions stored or storable on a physical data carrier according to claim 15.

[0004] Video conferencing systems of the aforementioned type and associated transmission methods are known; they are enjoying increasing popularity, particularly due to the restrictions on everyday and professional life in the wake of the Corona epidemic.

[0005] Previously known videoconferencing systems are disclosed in US 2014 / 132701 A1, CN 108 650 484 A, US 2016 / 170970 A1 and US 2006 / 120307 A1.

[0006] Particularly when such video conferencing systems are used in education, it regularly occurs that individual participants in a video conference cannot easily follow an instructor, speaker, or presenter because they do not understand their language, or not sufficiently. Consequently, there is a need for a video conferencing system or a method for transmitting information that allows for simple and flexible (simultaneous) translations of a transmitted audio stream, thus enabling a larger number of users to participate in video conferences.

[0007] This problem is solved according to the invention by a video conferencing system with the features of claim 1, by a method for transmitting information with the features of claim 9, and by a computer program product with the features of claim 15.

[0008] Advantageous further developments of the invention are defined in the respective subclaims and are hereby explicitly included in the description by reference.

[0009] A videoconferencing system according to the invention is designed for the bidirectional transmission of video and audio signals using streaming technology between at least three endpoints, each of which has a transmitting device and / or a receiving device for audio streams and / or for video streams. The videoconferencing system according to the invention comprises: i) at least one first endpoint with at least one audio transmitter and preferably also with a video transmitter; ii) at least one second endpoint with at least one audio receiver; iii) at least one third endpoint with at least one audio transmitter and one audio receiver; in which a) the first endpoint is connected to the second endpoint and the third endpoint to transmit a first audio stream to the second endpoint and the third endpoint by means of the audio stream transmitter and preferably also to transmit a first video stream in parallel to the first audio stream by means of the video stream transmitter at least to the second endpoint; b) the second endpoint has a request device, preferably a graphical user interface, and is connected to the third endpoint to selectively request a second audio stream from the third endpoint by means of a request signal generated or generable by the request device and to receive from the third endpoint by means of the receiving device which second audio stream is derived from the first audio stream;c) the third endpoint is configured to receive the first audio stream from the first endpoint and the request signal from the second endpoint, and, depending on the request signal, to generate the second audio stream and transmit it to the second endpoint.

[0010] The third endpoint is configured to detect the reception of a further request signal at another third endpoint and, depending on this further request signal, to generate a further second audio stream and transmit it to the second endpoint from which the further request signal originated. Preferably, a given third endpoint is configured to generate only one second audio stream at a given time, and most preferably, the further second audio stream can only be generated when no second audio stream is currently being generated. The second audio stream and / or the further second audio stream is a simultaneous translation of the first audio stream, preferably a human translation.

[0011] An inventive method for transmitting information by means of streaming technology via communication channels between at least three endpoints, each having a transmitting device and / or a receiving device for audio streams and / or for video streams, is characterized in that: a) a first endpoint is connected at least temporarily to a second endpoint and to a third endpoint via a corresponding communication channel and transmits a first audio stream to the second and third endpoints by means of the audio stream transmitter, while the first endpoint preferably also transmits a first video stream in parallel to the first audio stream by means of the video stream transmitter at least to the second endpoint; b) the second endpoint is connected at least temporarily to the third endpoint via a corresponding communication channel in order to selectively request a second audio stream from the third endpoint by means of a request signal and to receive from the third endpoint which second audio stream is derived from the first audio stream;c) the third endpoint receives the first audio stream from the first endpoint and the request signal from the second endpoint and, depending on the request signal, generates the second audio stream from the first audio stream and transmits it to the second endpoint; d) the second endpoint outputs at least the second audio stream.

[0012] The third endpoint detects the receipt of another request signal at a further third endpoint and, depending on this request signal, generates a further second audio stream and transmits it to the second endpoint from which the further request signal originated. Preferably, a given third endpoint generates only one second audio stream at a given time, and most preferably, the further second audio stream is generated only when no second audio stream is currently being generated. The second audio stream and / or the further second audio stream is generated by a simultaneous translation of the first audio stream, preferably by a human translator. Most preferably, a video stream of a human translator is output to a user of the second endpoint together with an audio stream of the human translation.

[0013] A computer program product according to the invention comprises program information stored or storable on a physical data carrier, the execution of which on a computer system has at least three endpoints connected or connectable via communication channels, each of which has a transmitting device and / or a receiving device for audio streams and / or for video streams, a) a video conferencing system according to the invention for the bidirectional transmission of video and audio signals and / or b) a method according to the invention for transmitting information between the at least three endpoints is carried out.

[0014] The computer program product according to the invention need not necessarily be stored on a physical data carrier; it can also be provided as a download or in some other way in a non-physical form. The essential point is that, upon execution of the associated program information, a videoconferencing system according to the invention is generated or a method according to the invention is carried out.

[0015] When the term "endpoints" is used in the description, it generally refers to computers, in particular and without limitation conventional desktop PCs, which function as endpoints of transmission channels in the video conferencing system, a fact that is familiar to those skilled in the art. Such endpoints typically have transmitting and receiving equipment to receive audio and video streams from other participants in the video conferencing system and, in turn, to transmit audio and video streams to other participants in the video conferencing system. This, too, is familiar to those skilled in the art. The endpoints mentioned do not have to be located in the same place, but can be connected to each other (wired and / or wirelessly) via a computer network, such as the global internet or a company intranet. They can also be located at physically distant locations.

[0016] To generate the aforementioned audio and video streams, the endpoints of a video conferencing system typically include appropriate recording devices, such as cameras or microphones, which is also familiar to those skilled in the art. The corresponding output of the audio and video streams is achieved using suitable devices, such as speakers, headphones, or screens (displays). This, too, is generally known to those skilled in the art.

[0017] Typically, each participant in a video conferencing system has their own endpoint, such as a personal computer. This device is configured to receive and transmit audio and video streams from other participants. Furthermore, each participant can also contribute their own audio and video streams to the system, which can then be shared by other participants.

[0018] According to the invention, the videoconferencing system initially comprises at least one first endpoint configured to transmit audio streams. This first endpoint can be the endpoint of an instructor or presenter who provides a corresponding presentation audio stream (first audio stream) in the videoconferencing system. Preferably, a corresponding (first) video stream also exists in parallel and is likewise provided in the videoconferencing system.

[0019] The second endpoint typically belongs to a "normal" participant in the video conferencing system, for example, a student who wants to listen to at least the first audio stream. Accordingly, the second endpoint is equipped with at least an audio receiver to receive the first audio stream and play it back (via suitable output devices).

[0020] In a preferred application of the videoconferencing system according to the invention, the third endpoint is assigned to a translator (interpreter) who is capable of receiving an audio stream in a first language, generating an audio stream in another language from it, and making it available in the videoconferencing system. Accordingly, the third endpoint has both a receiving device for audio streams and a transmitting device for audio streams.

[0021] Furthermore, according to the invention, the first endpoint is connected to the second and third endpoints. The first endpoint generates the first audio stream and makes it available to the second and third endpoints via the transmitting device. A first video stream can also be transmitted in parallel.

[0022] The second endpoint, i.e., the "normal" participant, has a request device, which can be designed as a graphical user interface, with which it can generate a so-called request signal and transmit it to the third endpoint in order to selectively request a second audio stream from the third endpoint. This second audio stream is preferably a (simultaneous) translation of the first audio stream, which the user of the second endpoint then receives from the third endpoint. However, within the scope of the present invention, it is sufficient if the second audio stream is derived from the first audio stream—it does not necessarily have to be a (simultaneous) translation of the first audio stream, although this is a particularly preferred application of the present invention.

[0023] In principle, the invention can also be extended to videoconferencing systems in which, in addition to or as an alternative to the aforementioned first audio stream, a first video stream originating from the first endpoint is converted or translated into a second video stream upon request. Furthermore, it is also possible, in principle, to convert the first audio stream not into another audio stream, but into a video stream, for example, to display a corresponding sign language translation to a hearing-impaired participant. This is also possible independently of the interpreting or translation function described above.

[0024] However, a preferred application of the invention – as already mentioned – provides that the second audio stream is a (simultaneous) translation of the first audio stream, which is transmitted to the second endpoint or to its user upon request.

[0025] According to the invention, the third endpoint is configured to receive the first audio stream (and / or the first video stream) from the first endpoint and the request signal from the second endpoint, and, depending on the request signal, to generate the second audio stream or the second video stream and transmit it to the second endpoint. In this context, it is possible, but not mandatory, for the second audio stream or the second video stream to be generated only after the request signal has been transmitted or received; it can be generated beforehand, but is only made available or output at the second endpoint after the request signal has been transmitted or received.

[0026] The list i) to iii) of equipment features of the individual endpoints reproduced above indicates only those equipment features that are absolutely necessary within the framework of a videoconferencing system according to the invention or for carrying out the method according to the invention. As a rule, each of the endpoints will have identical equipment features, namely transmitting and receiving devices for audio streams as well as transmitting and receiving devices for video streams, as is known from conventional videoconferencing systems in which all participants can see and speak to each other.

[0027] A first further development of the videoconferencing system according to the invention provides that the second endpoint is configured to receive the first audio stream and the second audio stream simultaneously via the receiving device and to output them simultaneously via a playback device. In this way, the second endpoint, or the participant there, is able to reproduce the second audio stream (translation) in parallel with the original audio stream (first audio stream), thereby optimizing the listening experience because it is possible in this way to grasp not only the actual meaning of what is heard but also nuances of the original.

[0028] Another embodiment of the videoconferencing system according to the invention provides that the second endpoint is configured to output the first and second audio streams at different volume levels, preferably the first audio stream at a reduced volume compared to the second audio stream, and the first audio stream can also be completely muted. This allows the participant at the second endpoint to focus primarily or entirely on the second audio stream, for example, to optimally follow a presentation, even if they do not understand the original language. This volume adjustment preferably occurs automatically upon receipt of the second audio stream; however, an adjustment option for the user may also be provided, either additionally or alternatively.If there are multiple first audio streams (multiple speakers), it may be provided that all first audio streams are output at a reduced volume compared to the second audio stream, as long as the second audio stream is being output.

[0029] In a further development, the videoconferencing system according to the invention can also be configured, with respect to the third endpoint, to transmit the second audio stream only to a second endpoint from which the request signal was previously received. In other words, only those participants who actively request the second audio stream receive it; all other participants, on the other hand, follow the first audio stream—especially if they understand the speech of the first audio stream sufficiently well. In this way, the videoconferencing system is not burdened with unnecessary transmissions, which can negatively affect the available bandwidth and the transmission quality.

[0030] Yet another embodiment of the videoconferencing system according to the invention provides that the second endpoint is configured to selectively cancel the second audio stream at the third endpoint by means of a termination signal generated or generable by the requesting device. Correspondingly, the third endpoint is configured to transmit the second audio stream only to a second endpoint from which no termination signal has previously been received. In this way, a participant can cancel a previously requested second audio stream at any time, particularly if it turns out that a translation is not required. This allows bandwidth that has been occupied to be released again, in order to improve the transmission quality in the videoconferencing system if necessary.

[0031] Yet another embodiment of the videoconferencing system according to the invention provides for a plurality of third endpoints, each providing different second audio streams. Referring to the application example mentioned at the beginning, this could involve a plurality of third endpoints, each with a different translator present as a user to provide translations into different languages, corresponding to the aforementioned different second audio streams. Furthermore, it may be possible for interpreters to translate the translation of another interpreter (relay interpreting). It is also provided that a specific second audio stream can be selectively requested by the second endpoint by sending the request signal to a specific third endpoint.For example, if a first third endpoint provides an English translation, while a second third endpoint provides a French translation, the second endpoint or the participant there can specifically request the second audio stream from the relevant third endpoint that corresponds to their linguistic preferences.

[0032] Another embodiment of the videoconferencing system according to the invention provides that at least the second endpoint additionally has a receiving device for video streams, and at least the third endpoint additionally has a transmitting device for video streams. This transmitting device is configured, upon receiving the request signal from the second endpoint, to transmit a corresponding second video stream to the second endpoint in parallel with the second audio stream, preferably a translation. In this way, the participant at the second endpoint can see the participant at the third endpoint, in particular a translator, while receiving the second audio stream (the translation) from there. Preferably, this occurs in parallel with the reception of the first video stream, which can show a presenter from whom the first audio stream originates.Preferably, the transmission of the second video stream ends simultaneously with the termination of the transmission of the second audio stream. If the requested translation is into sign language, only the second video stream may be transmitted (without the second audio stream).

[0033] In a further development of the videoconferencing system according to the invention, the second endpoint can also be configured to display a (still) image, preferably a symbolic image, before the request signal is sent via the video stream receiving device or a corresponding output device. This image indicates a property of the third endpoint and / or the second audio stream. For example, if the third endpoint is configured to provide a translation of the first audio stream into French, the image could depict a French flag or the Eiffel Tower, so that a user can readily recognize at the second endpoint what type of second audio stream they will receive with their request.

[0034] Preferably, the third endpoint (the interpreter) activates itself or is equipped with a corresponding activation device, i.e., a corresponding control element. As soon as it is ready for interpreting, it activates itself for the other participants (second endpoints) via the activation device and then becomes visible as a voice channel (especially in the form of the aforementioned (still) image).

[0035] This can be displayed, for example, in a sidebar at a second endpoint (participant console). The participant can then switch to different language channels or select corresponding audio streams; these are preferentially displayed using typical symbols / flags of the respective nation (e.g., French - Eiffel Tower), as mentioned above.

[0036] In a further embodiment, the videoconferencing system according to the invention can provide that the number of second audio streams that can be received simultaneously at the second endpoint is limited, preferably to one (1). This avoids the unnecessary use of bandwidth, because a user of the second endpoint will regularly be unable to follow more than one second audio stream at the same time. However, the invention is by no means necessarily limited to the number one in this respect.

[0037] Furthermore, in a particular embodiment of the videoconferencing system according to the invention, the third endpoint may be equipped with a display device that shows the number of second endpoints from which the request signal has been received and which are receiving the second audio stream at a given time. In the application described above, the translator, for example, can thus see at the third endpoint how many participants (at second endpoints) are currently receiving the second audio stream (e.g., the requested translation). As soon as the number becomes zero (0), the translator at the third endpoint can, in principle, cease their translation activity to conserve resources. In this context, it is also possible for a translator to...a translator who can translate into multiple languages ​​makes her capabilities available on another language channel, i.e., to generate another required second audio stream.

[0038] The aforementioned display device need not be limited to the third endpoint, but can be available at all endpoints.

[0039] The videoconferencing system according to the invention provides for a third endpoint to detect the receipt of a further request signal at another third endpoint and, depending on this further request signal, to generate a further second audio stream and transmit it to the second endpoint from which the further request signal originated. This relates to the situation already described above, according to which a further second audio stream, i.e., for example, a further translation into another language, is required in a described videoconferencing system. If the third endpoint (the translator) is able to detect the receipt of such a further request signal or, alternatively, the transmission of such a signal, it is fundamentally possible for it to provide a corresponding further second audio stream (an additional translation).

[0040] Further developing this idea, it can be stipulated that the aforementioned second audio stream (the additional translation) can only be generated if no second audio stream (the first translation) is currently being generated. In other words, a third endpoint (translator) that is already generating a second audio stream (a translation) cannot simply terminate it to optionally generate another second audio stream (a different translation). However, it is fundamentally possible to terminate a currently generated second audio stream if there is an increased demand for another second audio stream and insufficient resources are available to generate both.

[0041] Finally, the videoconferencing system according to the invention also provides that the second audio stream and / or the further second audio stream is a simultaneous translation of the first audio stream, preferably a human translation. This has already been repeatedly mentioned above when referring to the preferred application of the videoconferencing system according to the invention. However, the invention is not limited to simultaneous translations and / or human translations, but specifically also includes machine translations.

[0042] With regard to the method according to the invention, a first embodiment thereof provides that the second endpoint receives and outputs the first and second audio streams simultaneously. This was already mentioned above with regard to the videoconferencing system and its possible configurations.

[0043] Furthermore, in a further development of the inventive method, it can be provided that the second endpoint outputs the first audio stream and the second audio stream (automatically) at different volume levels, preferably the first audio stream at a reduced volume compared to the second audio stream. This was already mentioned above with reference to the video conferencing system.

[0044] Another further development of the method according to the invention provides that the third endpoint transmits the second audio stream only to a second endpoint from which the request signal was received. This aspect was also discussed above with regard to the video conferencing system.

[0045] In a further development of the inventive method, it can be provided that the second endpoint selectively terminates the second audio stream by means of a termination signal at the third endpoint, while the third endpoint only transmits the second audio stream to a second endpoint from which no termination signal has previously been received. This aspect has also been discussed in detail above.

[0046] Yet another embodiment of the inventive method provides that a plurality of third endpoints each provide different second audio streams, wherein the second endpoint selectively requests a specific second audio stream by sending the request signal to a specific third endpoint. In this way, a participant or user at the second endpoint can selectively request specific second audio streams from the respective associated third endpoints, preferably different translation languages. This, too, has already been discussed in detail above. Naturally, a given third endpoint can also offer more than one second audio stream, from which the participant can select.However, this option is usually no longer available once a specific second audio stream has been selected by (any) participant, because a human translator cannot generate more than one second audio stream at a time.

[0047] A further development of the inventive method provides that, upon receiving the request signal from the second endpoint, the third endpoint also transmits a second video stream to the second endpoint in parallel with the second audio stream. In this way, it is particularly possible for a participant at the second endpoint to also see the source of the second audio stream (a translation), which can increase acceptance and promote understanding. It has also been noted that the second video stream can be transmitted in parallel with the first video stream, which typically originates from the instructor or presenter. When the second audio stream is terminated (dismissed), the transmission or reception of the second video stream usually also ends.

[0048] Preferably, the second video stream from the third endpoint, or alternatively an image of the participant at the third endpoint (the interpreter), only appears when they begin translating, i.e., speaking. Only during their actual work, i.e., the translation, is the participant perceptible in the virtual space.

[0049] In particular, the third endpoint is preferred to be visible only as a voice channel in the participant console before it is activated by a request signal from a participant.

[0050] To facilitate the selection of a third endpoint for a participant or user at the second endpoint, a further embodiment of the inventive method may provide that the second endpoint displays a (static) image, preferably a symbolic image, for the third endpoint before sending the request signal. This image indicates a property of the third endpoint and / or the second audio stream. It has already been mentioned that this image could, for example, represent a national flag symbolizing a language of the second audio stream.

[0051] Furthermore, in a further development of the method according to the invention, it can be provided that the number of second audio streams received simultaneously at the second endpoint is limited, preferably to one (1). This allows system resources to be saved or used efficiently, as has already been mentioned.

[0052] A further development of the inventive method provides that the third endpoint displays to its user (translator) the number of second endpoints from which the request signal was received and which are receiving the second audio stream at a given time. Thus, for example, if a translator generates the second audio stream at the third endpoint, they can see at any given time how many participants at second endpoints are currently using the second audio stream. This allows them to react if, for example, the aforementioned number drops to zero (0).

[0053] In the inventive method, the third endpoint detects the receipt of the further request signal at another third endpoint or indicates this to the user there. Depending on the further request signal, it generates a further second audio stream and transmits it to the second endpoint from which the further request signal originated. This aspect has already been discussed in detail above and can particularly apply to the case where a translator at the third endpoint detects that participants at a second endpoint have requested a further second audio stream, for example, a translation of the first audio stream into another language. Under certain circumstances, the third endpoint or the translator can react to this by selectively generating and providing a further second audio stream.This aspect has already been discussed in detail above.

[0054] Preferably, it can be provided that a given third endpoint, for example a translator, generates only one second audio stream at a given time, since the translator will generally not be able to provide two different translations simultaneously. Most preferably, in this context, the additional second audio stream (i.e., the other translation) is only generated if no second audio stream (i.e., the original translation) is currently being generated. This applies to the case where the translator determines that no second audio stream is currently being generated, but another second audio stream is needed. If they are able to do so, they can then provide the aforementioned additional second audio stream. This will be the case, in particular, when the number of participants who originally requested the second audio stream has fallen to zero.However, the invention is by no means limited to such a case.

[0055] As described above for the videoconferencing system, the method according to the invention provides that the second audio stream and / or the further second audio stream is generated by a simultaneous translation of the first audio stream, preferably by a human translator. It should be expressly noted here that the invention is by no means limited to providing human translations. In principle, the third endpoint can provide a machine translation in addition to or as an alternative to a human translation, provided its quality is acceptable for a given application. Furthermore, it is possible that machine translation systems will improve in quality in the future, so that machine translations can then be used preferentially within the scope of the present invention.

[0056] Another embodiment of the method according to the invention provides that a user of the second endpoint triggers the request signal and / or the termination signal, preferably via a graphical user interface. The corresponding input can be made using aids such as a mouse or the like, or directly by touching a suitably touch-sensitive display. Alternatively, voice commands or gesture control are possible. The invention is not limited to a specific input mode.

[0057] Advantageously, the third endpoint can be equipped with hearing protection for the participant (interpreter), for example, to prevent feedback. To protect the participant from loud whistling sounds or other loud and sudden acoustic signals, the hearing protection is designed to automatically filter out volume peaks from the audio signal (audio stream) arriving at the third endpoint. This can prevent, for example, sudden hearing loss, acoustic trauma, or other health problems for interpreters.

[0058] Furthermore, the third endpoint (the interpreter) can have or be equipped with the following operating devices, for example in the form of an interpreter's console in a display sidebar, with the following controls / functions, which are not limited to the specific designs listed only as examples: Throat clearing button: If the interpreter needs to clear their throat, they can quickly interrupt their audio stream. Volume slider: The interpreter can adjust the volume of the keynote speaker (participant at the first endpoint; first audio stream). In relay interpreting (interpreter interpreting from another interpreter's interpretation), the volume of the first interpretation can be adjusted accordingly. Private audio channel between two interpreters: With the push of a button, two interpreters (participants at third endpoints) can converse without the other participants in the video conferencing system hearing them. Automatic handover to another interpreter: Before an interpreter leaves the virtual room (i.e., the video conferencing system), they can automatically hand over to another interpreter with the push of a button.The interpreter can adjust the frequency range used – especially for the initial audio stream arriving at him – so that he can understand everything clearly over his own voice.

[0059] Further features and advantages of the invention will become apparent from the following description of exemplary embodiments with reference to the drawing. Figure 1 schematically shows the structure of a videoconferencing system according to the invention; Figure 2 shows a use of the videoconferencing system according to the invention. Figure 1 for carrying out a method according to the invention; Figure 3 also shows a use of the videoconferencing system according to Figure 1 for carrying out a method according to the invention; and Figure 4 shows a possible design of a user interface within the framework of a video conferencing system according to the invention.

[0060] Both masculine and feminine forms of terms like "participant" or "user" are used interchangeably below. Both forms are always intended to be included, even if not explicitly stated.

[0061] Figure 1 Figure 1 shows a videoconferencing system according to the invention with a number of participants arranged at different endpoints of the videoconferencing system. In principle, all endpoints of the videoconferencing system can be technically equivalent; this is not shown in the figures – here, the technical equipment of the individual endpoints shown is limited to the equipment required according to the invention or in the course of further developments.

[0062] The technical design of the individual endpoints of the video conferencing system has already been described above in the general section of the description.

[0063] According to Figure 1The video conferencing system initially includes a first endpoint EP1, which is assigned to an instructor, presenter, or generally a VIP user. This is in Figure 1 Each endpoint is marked with the reference symbol B1 and a crown. Accordingly, a second endpoint EP2 is assigned to a second user B2, and a third endpoint EP3 to a third user B3. Further endpoints EPn can exist, each of which can in turn be assigned a user Bn. In principle, the number of endpoints in the videoconferencing system is unlimited within the limits of the technical possibilities.

[0064] The endpoints EP1-EPn mentioned above are interconnected via a computer network CN. Preferably, the computer network CN is the global internet or a company intranet. However, the invention is not limited to any specific type of computer network CN.

[0065] The individual endpoints EP1-EPn are capable of exchanging data streams over the computer network CN, preferably audio streams (sound) and video streams (image) within the scope of the present invention. To generate such data streams, all endpoints (EP1-EPn) are equipped with a camera CA and a microphone MI. Furthermore, all endpoints EP1-EPn have transmitting and receiving devices for the aforementioned data streams, symbolized by reference numeral SE in Figure 1. These devices may, but need not, be wireless. At least the second endpoint EP2 and the third endpoint EP3 (and regularly also the subsequent endpoints EPn) also have suitable output / playback devices for received data streams, shown here in the form of a display DI for received video streams and a loudspeaker SP for received audio streams.Furthermore, at least the second endpoint EP2 and the third endpoint EP3 have a user interface Ul, which can be designed as a graphical user interface (GUI), as is particularly the case in . Figure 1 symbolizes.

[0066] Reference numeral CP depicts a data carrier and a corresponding reading device, illustrating that the program instructions (program code) required to set up the videoconferencing system and its components, or to execute a method according to the invention using the videoconferencing system shown, can be stored or stored on a corresponding data carrier in order to equip (set up) the videoconferencing system accordingly. However, it is not fundamentally necessary for the aforementioned program instructions to be stored on a physical data carrier; alternatively or additionally, a download in a correspondingly intangible form, e.g., from the internet, is also possible.

[0067] It has already been noted that user B1 may be an instructor or presenter. In a preferred use case of the video conferencing system, user B2 is a "regular" participant, for example, a trainee who wants to follow a presentation by presenter B1. User B3 in this case is a translator, as described in Figure 1 This is symbolically represented by the speech bubble. The distribution of roles and the associated processes will be discussed in more detail below.

[0068] The additional users or participants Bn can be, in particular, other "normal" participants corresponding to participant B2 or other presenters corresponding to participant B1. However, use cases are also described below in which the additional participants or users Bn can also be, at least in part, other translators corresponding to user B3.

[0069] In Figure 2 is shown how the video conferencing system according to Figure 1 can be advantageously used. In this way, a method according to the invention for transmitting information, in particular the aforementioned audio and video streams, by means of streaming technology via communication channels, which communication channels are formed between the at least three endpoints EP1 to EP3, results.

[0070] The first endpoint EP1 is connected, at least temporarily, and preferably permanently, during a presentation by user B1, to the second endpoint EP2 and the third endpoint EP3 via a corresponding communication channel. The first endpoint EP1 sends a first audio stream via the transmitting device SE to the second endpoint EP2 and the third endpoint EP3. This is represented in the figure by the reference symbol AS1. The second endpoint EP2 and the third endpoint EP3 output the audio stream AS1 via their respective playback devices SP. Simultaneously, the first endpoint EP1 also sends a first video stream VS1 via the transmitting device SE to the second endpoint EP2 and the third endpoint EP3. This is represented in Figure 2This is symbolized by the fact that an image B1' of the presenter B1 is displayed on both the DI display of the second endpoint EP2 and the EP3 display of the third endpoint. The transmission of the first audio stream AS1 and the first video stream VS1 takes place along the communication channels between the endpoints via the CN computer network, as shown in Figure 2 This is symbolized by dashed lines. Participant / user B2 and participant / user B3 can therefore see and hear user (presenter) B1.

[0071] On the DI display of user B2 at the second endpoint EP2, a symbol for a second audio stream AS2 is displayed at reference SB. User B2 can request this AS2 by sending a request signal AF to the third endpoint EP3 or the associated user B3 (translator). To do this, user B2 makes a corresponding request or input via their user interface UI, whereupon endpoint EP2 sends the request signal AF to endpoint EP3. Upon receiving the request signal AF, translator B3 either begins generating the requested second audio stream AS2 (via microphone MI), whereupon endpoint EP3 transmits the audio stream AS2 to the second endpoint EP2, after which... Figure 3This will be discussed in more detail later. Alternatively, user B3 generates the second audio stream AS2 from the very beginning, that is, from the start of the presentation by user B1; however, this is only made available to user B2 upon request (request signal AF). Figure 2 The request signal AF and the second audio stream AS2 are indicated by dashed lines. Transmission takes place along corresponding communication channels via the CN computer network.

[0072] Other users Bn can also request a corresponding second audio stream AS2 from their endpoints EPn by means of a request signal AF at the relevant endpoint EP3, which in Figure 2 This is not shown. Of course, further endpoints analogous to the third endpoint EP3 can also be present in the video conferencing system, each outputting (or capable of outputting) different second audio streams AS2. This would then be analogous to... Figure 2The second audio stream, AS2, is displayed on the DI screen of user B2 and other users Bn via various symbol images (SB). Users B2 and Bn can then select and request their desired second audio stream, AS2, via the user interface (UI). Figure 3 is schematically shown what the user B2 will see at the second endpoint EP2 after processing the in Figure 2 The process steps shown can be seen or heard:

[0073] He / she continues to receive the first audio stream AS1 and the first video stream VS1 from presenter B1 and endpoint EP1, respectively. The corresponding output is provided via the speaker SP and the display DI, as previously explained. Additionally, user B2 at endpoint EP2 receives the second audio stream AS2 from the third endpoint EP3. As shown in Figure 3The third endpoint EP3, or rather the user (translator) B3 located there, additionally generates a second video stream VS2 using their camera CA and sends this together with the second audio stream AS2 to the second endpoint EP2. Endpoint EP2 outputs at least both the first video stream VS1 and the second video stream VS2 via the display DI, as symbolically represented (reference symbols B1', B3').

[0074] On the DI display of endpoint EP3, in addition to the first video stream VS1 (reference B1'), information regarding the number of (secondary) endpoints receiving the second audio stream AS2 at the specified time is also displayed at reference Al. In this case, there are several additional (secondary) endpoints EP2' that receive the second audio stream AS2, as shown in Figure 3 based on the flag in question (see below). Figure 2) is symbolized. Furthermore, there are several other (second) endpoints EP2" that receive another second audio stream AS2" originating from a further (not shown) third endpoint analogous to endpoint EP3. The transmission paths of the aforementioned (further) second audio streams AS2, AS2" are shown in Figure 3 Not fully displayed for the sake of clarity.

[0075] The described situation is explained in Figure 3 symbolized by a different flag at the endpoints EP2", which represents a different language and accordingly a different second audio stream AS2".

[0076] Although this is for the sake of clarity in Figure 3As not shown, the second endpoints EP2' and EP2" continue to receive the first audio stream AS1 and the first video stream VS1 from endpoint EP1. Additionally, the second video stream VS2 is transmitted from the third endpoint EP3, as already noted. In the case of endpoints EP2", which receive a further second audio stream AS2", this is accordingly a corresponding further second video stream VS2".

[0077] Those endpoints EPn that have not requested a (further) second audio stream will conventionally receive only the first audio stream AS1 and the first video stream VS1, which in Figure 3 For the sake of clarity, it is not shown further.

[0078] It has already been pointed out that, in principle, all endpoints of the video conferencing system can be technically identical, although this is not explicitly shown in the diagrams for the sake of clarity. However, there is generally nothing preventing the described video conferencing system from being set up with endpoints equipped with different technical specifications, as long as each endpoint possesses the necessary capabilities to participate in the video conferencing system.

[0079] It has already been noted that endpoint EP2 receives both the first audio stream AS1 and the requested second audio stream AS2. Preferably, both audio streams AS1 and AS2 are output simultaneously via the playback device's loudspeaker SP, with the requested second audio stream AS2 preferably being automatically output at a higher volume than the first audio stream AS1. If necessary, user B2 can adjust this via the user interface (UI).

[0080] With renewed reference to Figure 2It should also be noted that, analogous to requesting the second audio stream AS2, user B2 at endpoint EP2 can also send a termination signal BE to the third endpoint EP3 using the request signal AF. Upon receiving such a termination signal BE, the third endpoint EP3 stops transmitting the second audio stream AS2 to endpoint EP2 or any corresponding provision. The video conferencing system then returns to the state described in the following text: Figure 2 The initial state shown is restored (with the symbol SB displayed at endpoint EP2). In such a case, the value at reference mark Al is reduced according to... Figure 3 The value displayed at endpoint EP3 is set to one (1). The third user B3 is therefore always able to see whether participants B2 still depend on the second audio stream AS2 and could, if necessary, stop generating the second audio stream AS2 when Al = 0 (for example, to generate another audio stream AS2) according to Figure 3to produce, if he / she is able to do so).

[0081] The potential function of the user interface (UI) of endpoint EP3 has not yet been discussed in detail. Advantageously, this user interface (UI) allows user (participant) B3, in the described use case the translator, to select a so-called speech channel, that is, at least one type or property of the second audio stream AS2 or AS2' generated by them. In this way, one and the same user B3 can advantageously provide different second audio streams AS2, AS2', preferably not simultaneously.For example, if translator B3 recognizes from the display at reference mark Al (Figure 3) that the second audio stream AS2 is no longer needed, he / she can provide another (additional) second audio stream AS2" if required and display this via the user interface UI in the video conferencing system (so that, for example, the displayed icon images SB adjust accordingly, preferably automatically).

[0082] It has already been pointed out elsewhere that the described videoconferencing system also functions advantageously if the third endpoint EP3 provides only a second video stream VS2 instead of the second audio stream AS2, which, for example, contains a translation of the first audio stream AS1 into sign language. This is not explicitly shown in the figures, but expressly represents an advantageous alternative embodiment of the described invention.

[0083] Figure 4 Finally, it shows a possible display on the DI screen (see below). Figures 1 to 3 ) of a participant in a video conference conducted using the video conferencing system described above. According to the exemplary illustration in Figure 4 Three participants are present again, who, in accordance with the Figures 1 to 3 are designated with the reference symbols B1 to B3. As the expert will readily recognize, this is a simplification, since the described functions can of course also be applied to larger numbers of participants.

[0084] In Figure 4 The device used by the participant in the video conference, in this case without restriction a notebook or laptop computer, is defined as follows: Figures 1 to 3 designated with the reference symbol EP (endpoint).

[0085] As above based on the Figures 1 to 3As already described, each of the three participants B1 to B3 was assigned a specific participant role prior to the video conference, thereby defining the associated functions, which are based on the Figures 1 to 3 The details of the issues already discussed have been explained in detail. The respective role is indicated by a corresponding label and description under reference KB, which overlays the respective video image of participant B1 to B3.

[0086] In the example according to Figure 4 Participant B1 has the role of the presenter, participant B3 has the role of the interpreter, and participant B2 wants to follow the presentation as a listener.

[0087] As soon as the speaker B1 begins to speak, the interpreter B3 preferably begins their work and also starts to speak or translate. Accordingly, the speaker B1 generates the first audio stream AS1 (see...). Figures 1 to 3), while the interpreter B3 interprets the associated second audio stream AS2 (see below). Figures 1 to 3 ) generated. Due to the role assigned to the interpreter, their audio output is initially not audible to user B2, as the aforementioned audio output is not required by every participant.

[0088] To make interpreter B3 audible, the listener B2 activates the interpreter's audio output via a context menu (KM) on their graphical user interface, thus sending the request signal as previously described. The context menu (KM) overlays the... Figure 4The video image of interpreter B3. The context menu KM is triggered by a mouse click (or alternatively by touch, voice command, or similar) and then displays a button SF for activating or deactivating the interpreter function. This corresponds functionally to the request signal and end signal described in detail above. The context menu KM can also be accessed via a user context BK together with a participant list TL in a displayed sidebar or (display) sidebar SL.

[0089] As soon as listener B2 activates the interpreter function, the audio stream AS2 originating from interpreter B3 (see below) will be used. Figures 1 to 3 ) output via the loudspeaker SP, simultaneously with the audio stream AS1 originating from the presenter B1 (see below). Figures 1 to 3) and possibly with additional audio streams that may originate from other participants in the video conference. The volume of the audio output of all participants present, including the presenter B1, is significantly reduced compared to the interpreter B3 and thus overlaid by the audio output of interpreter B3. At the same time, the EP also receives a video stream from interpreter B3 (video stream VS2, see below). Figures 2 and 3 ) and outputs this together with the video image of the presenter B1 (video stream VS1, see below). Figures 2 and 3 ). Furthermore, a video image of user B2 is displayed, as is well known from conventional video conferencing systems.

[0090] As mentioned previously, there may also be other users in category B1. In this case, it is advantageous that user B2 only hears all speakers (user B1) at full volume again when the request signal AF is deactivated. As long as the request signal AF is activated, interpreter B3 is always audible at full volume, regardless of who is speaking.

[0091] It has already been pointed out that several B3 interpreters can participate in a video conference simultaneously, but advantageously only one B3 interpreter can be activated per B2 participant at any given time.

[0092] Instead of an image of interpreter B3, a symbolic image can initially be displayed, i.e., before the interpreter function is activated, as described in detail above; the image of interpreter B3 only appears when the interpreter function is activated or as long as it is active.

[0093] The interpreter B3 can display additional selection options on its DI display, in particular a number of current receivers of the audio stream it generates and / or language selection options, as already mentioned in detail above.

[0094] Figure 4 This primarily concerns the visible display or user interface at an endpoint EP of the described video conferencing system. For the technical design of the video conferencing system and its internal processes, please refer to the preceding description of the Figures 1 to 3 Reference has been made. In particular, interpreter B3 has a hearing protection device, as already mentioned. This may be integrated into the relevant endpoint (not shown).

[0095] Specifically, the interpreter B3 may have additional controls BE1-BE6 available on the display, e.g. in the sidebar, which have already been mentioned in the general part of the description.

[0096] BE1 refers to an activation device that enables the third endpoint (the interpreter) to activate itself as soon as it is ready for interpreting.

[0097] BE2 designates a throat clearing button, which the interpreter can use to quickly interrupt (mute) the transmission of his audio stream if he needs to clear his throat.

[0098] BE3 refers to a control element for automatically opening a private voice channel between two interpreters: With the push of a button, two interpreters (participants at third endpoints) can converse without the other participants in the video conferencing system hearing them. Figure 4There may be a corresponding button or similar for each additional interpreter.

[0099] BE4 refers to a control element for automatic handover to another interpreter: Before an interpreter leaves the virtual room (i.e., the video conferencing system), they can automatically hand over to another interpreter with the push of a button or similar action. According to Figure 4 There may be a corresponding button or similar for each additional interpreter.

[0100] BE5 is a volume slider: The interpreter can adjust the volume of the keynote speaker (presenter B1, participant at the first endpoint; first audio stream). In relay interpreting (one interpreter interpreting from another interpreter's interpretation), the volume of the first interpretation can be adjusted using control BE5.

[0101] Finally, BE6 refers to another slider that allows the interpreter to adjust the frequency range used – especially for the first audio stream arriving at him – so that he can understand everything clearly over his own voice.

[0102] The invention is not limited in particular with regard to the type and number of operating elements BE1-BE6.

Claims

1. A video conferencing system designed for bidirectional transmission of video and audio signals by means of streaming technology between at least three endpoints (EP1-EP3), each of which has a transmitting apparatus and / or a receiving apparatus (SE) for audio streams (AS1, AS2, AS2") and / or for video streams (VS1, VS2, VS2"), having: i) at least one first endpoint (EP1) with at least one transmitting apparatus (SE) for audio streams and preferably also with a transmitting apparatus (SE) for video streams; ii) at least one second endpoint (EP2) with a receiving apparatus (SE) for audio streams; iii) i. at least one third endpoint (EP3) with at least one transmitting apparatus (SE) for audio streams and a receiving apparatus (SE) for audio streams; in which a) the first endpoint (EP1) is connected to the second endpoint (EP2) and the third endpoint (EP3) in order to transmit a first audio stream (AS1) to the second endpoint (EP2) and the third endpoint (EP3) by means of the transmitting apparatus (SE) for audio streams and, preferably in parallel with the first audio stream (AS1), also to transmit a first video stream (VS1) by means of the transmitting apparatus (SE) for video streams at least to the second endpoint (EP2); b) the second endpoint (EP2) has a request apparatus, preferably a graphical user interface (UI), and is connected to the third endpoint (EP3) in order to selectively request a second audio stream (AS2) from the third endpoint (EP3) by means of a request signal (AF) generated or capable of being generated by the request apparatus and to receive it from the third endpoint (EP3) by means of the receiving apparatus (SE), which second audio stream (AS2) is derived from the first audio stream (AS1); c) the third endpoint (EP3) is designed to receive the first audio stream (AS1) from the first endpoint (EP1) and the request signal (AF) from the second endpoint (EP2) and, depending on the request signal (AF), to generate the second audio stream (AS2) and transmit it to the second endpoint (EP2); characterized in that the third endpoint (EP3) is set up to detect the reception of a further request signal (AF) at a further third endpoint (EP3) and, depending on the further request signal (AF), to generate a further second audio stream (AS2") and to transmit it to the second endpoint (EP2") from which second endpoint (EP2") the further request signal (AF) originated, wherein preferably a given third endpoint (EP3) is designed to generate only one second audio stream (AS2, AS2") at a given time, and wherein most preferably the further second audio stream (AS2") can only be generated when no second audio stream (AS2) is currently being generated; and in that the second audio stream (AS2) and / or the further second audio stream (AS2") is a simultaneous translation of the first audio stream (AS1), preferably a human translation.

2. The videoconferencing system according to claim 1, in which the second endpoint (EP2) is designed to receive the first audio stream (AS1) and the second audio stream (AS2) simultaneously by means of the receiving apparatus (SE) and to output them simultaneously by means of a playback device (SP), preferably at different volumes, most preferably outputting the first audio stream (AS1) at a lower volume than the second audio stream (AS2).

3. The videoconferencing system according to claim 1 or 2, in which the third endpoint (EP3) is designed to transmit the second audio stream (AS2) only to a second endpoint (EP2) from which the request signal (AF) was received.

4. The videoconferencing system according to any one of claims 1 to 3, in which the second endpoint (EP2) is designed to selectively cancel the second audio stream (AS2) at the third endpoint (EP3) by means of the request apparatus (UI) by means of a termination signal (BE) generated or capable of being generated by the request apparatus (UI), and the third endpoint (EP3) is designed to transmit the second audio stream (AS2) only to a second endpoint (EP2), from which no termination signal (BE) was received.

5. The videoconferencing system according to any one of claims 1 to 4, in which a plurality of third endpoints (EP3) is present, each of which provides different second audio streams (AS2, AS2"), wherein a specific second audio stream (AS2, AS") can be selectively requested by the second endpoint (EP2) by sending the request signal (AF) to a specific third endpoint (EP3).

6. The videoconferencing system according to any one of claims 1 to 5, in which at least the second endpoint (EP2) additionally has a receiving apparatus (SE) for video streams, and at least the third endpoint (EP3) additionally has a transmitting apparatus (SE) for video streams, which transmitting apparatus (SE) is set up to transmit a second video stream (VS2) to the second endpoint (EP2) in parallel with the second audio stream (AS2) upon receipt of the request signal (AF) from the second endpoint (EP2).

7. The videoconferencing system according to any one of claims 1 to 6, in which the third endpoint (EP3) has a display apparatus (DI) that displays a number of second endpoints (EP2) from which the request signal (AF) was received and which receive the second audio stream (AS2) at a given time.

8. The videoconferencing system according to any one of claims 1 to 7, in which the second audio stream (AS2) is a simultaneous translation of another second audio stream, preferably a human translation; and / or the third endpoint (EP3) has a control element (BE2) with which the transmission of the second audio stream (AS2) can be temporarily interrupted.

9. A method for transmitting information by means of streaming technology via communication channels between at least three endpoints (EP1-EP3), each of which has a transmitting apparatus and / or a receiving apparatus (SE) for audio streams and / or for video streams, in which: a) a first endpoint (EP1) is connected at least temporarily to a second endpoint (EP2) and to a third endpoint (EP3) via a corresponding communication channel and transmits a first audio stream (AS1) to the second endpoint (EP2) and to the third endpoint (EP3) by means of the transmitting apparatus (SE) for audio streams, while the first endpoint (EP1) preferably transmits, in parallel with the first audio stream (AS1), a first video stream (VS1) by means of the transmitting apparatus (SE) for video streams at least to the second endpoint (EP2); b) the second endpoint (EP2) is connected at least temporarily to the third endpoint (EP3) via a corresponding communication channel in order to selectively request a second audio stream (AS2) from the third endpoint (EP3) by means of a request signal (AF) and to receive it from the third endpoint (EP3), which second audio stream (AS2) is derived from the first audio stream (AS1); c) the third endpoint (EP3) receives the first audio stream (AS1) from the first endpoint (EP1) and the request signal (AF) from the second endpoint (EP2) and, depending on the request signal (AF), generates the second audio stream (AS2) from the first audio stream (AS1) and transmits it to the second endpoint (EP2); d) the second endpoint (EP2) outputs at least the second audio stream (AS2), preferably receives the first audio stream (AS1) and the second audio stream (AS2) simultaneously and outputs them simultaneously, most preferably outputs the first audio stream (AS1) and the second audio stream (AS2) at different volumes, in particular the first audio stream (AS1) at a lower volume than the second audio stream (AS2); characterized in that the third endpoint (EP3) detects the reception of a further request signal (AF) at a further third endpoint (EP3) and, depending on the further request signal (AF), generates a further second audio stream (AS2") and transmits it to the second endpoint (EP2") from which second endpoint (EP2") the further request signal (AF) originated, wherein preferably a given third endpoint (EP3) generates only one second audio stream (AS2, AS2") at a given time and wherein most preferably the further second audio stream (AS2") is generated only when no second audio stream (AS2) is currently being generated; and in that the second audio stream (AS2) and / or the further second audio stream (AS2") is generated by a simultaneous translation of the first audio stream (AS1), preferably by a human translation; wherein, most preferably, a video stream (VS2) of a human translator is output to a user (B2) of the second endpoint (EP2) together with an audio stream (AS2) of the human translation.

10. The method according to claim 9, in which the third endpoint (EP3) transmits the second audio stream (AS2) only to a second endpoint (EP2) from which the request signal (AF) was received; and in which, preferably, the second endpoint (EP2) selectively cancels the second audio stream (AS2) at the third endpoint (EP3) by means of a termination signal (BE), while the third endpoint (EP3) transmits the second audio stream (AS2) only to a second endpoint (EP2) from which no termination signal (BE) has been received.

11. The method according to claim 9 or 10, in which a plurality of third endpoints (EP3) each provide different second audio streams (AS2, AS2"), wherein the second endpoint (EP2) selectively requests a specific second audio stream (AS2, AS2") by sending the request signal (AF) to a specific third endpoint (EP3).

12. The method according to any one of claims 9 to 11, in which the third endpoint (EP3) transmits a second video stream (VS2) to the second endpoint (EP2) in parallel with the second audio stream (AS2) upon receipt of the request signal (AF) from the second endpoint (EP2).

13. The method according to any one of claims 9 to 12, in which the third endpoint (EP3) displays a number of second endpoints (EP2) from which the request signal (AF) was received and which receive the second audio stream (AS2) at a given time.

14. The method according to any one of claims 9 to 13, in which the second audio stream (AS2) is generated by a simultaneous translation of another second audio stream, preferably by a human translator.

15. A computer program product (CP) with program information stored or storable on a physical data carrier, in which, when executed on a computer system with at least three endpoints (EP1-EP3) connected or connectable via communication channels, each of which has a transmitting apparatus (SE) and / or a receiving apparatus (SE) for audio streams and / or for video streams, a) a video conferencing system is designed for bidirectional transmission of video and audio signals according to any one of claims 1 to 8, and / or b) a method for transmitting information between the at least three endpoints (EP1-EP3) is carried out in accordance with any one of claims 9 to 14.