Real-time audio and video feedback during teleconference
By providing real-time audio and video feedback during the feedback period during the conference call, the problem of difficulty for participants to confirm whether the audio and video are working properly is solved, and the fluency and efficiency of the meeting is improved.
Patent Information
- Application Number
- CN202280093659.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-14
- Filing Date
- 2022-12-18
- Publication Date
- 2024-11-26
AI Technical Summary
During a conference call, participants have difficulty confirming in real time whether their audio is heard or whether the screen is seen, resulting in frequent inquiries and potentially interrupting the meeting.
By providing real-time audio or video feedback during the feedback period when participants unmute or initiate screen sharing, sending audio in real-time or generating and sending thumbnails of screen content, ensuring that participants can hear or see in real-time when they are unmute or initiate screen sharing. Your own voice and screen content and stop sending after the feedback period is over to avoid distractions.
It enables participants to know immediately whether the audio and video are working properly when unmute or share, and reduces user interface operation and time, improving meeting fluency and efficiency.
Smart Images

Figure CN119032553A_ABST
Abstract
Description
Background Art
[0001] During a conference call, there are a few common questions that are often heard. One question is when a participant asks, "Can you hear me?" This usually happens when a participant has unmuted their microphone and has started talking, and the participant wants to confirm that the other participants in the call can hear the participant. Another question is when a participant asks, "Can you see my screen?" This usually happens when a participant has initiated screen sharing and wants to confirm that the other participants in the conference call can see the participant's shared screen. Another question is when a participant says, "Sorry, I didn't realize my microphone was on." This usually happens when a participant accidentally unmuted their microphone.
[0002] While these questions can be helpful when a participant is trying to confirm that the participant's audio is being heard or that the participant's screen is being seen, they can also disrupt the conference call experience. For example, asking the audience if they can hear a participant takes time, interrupts the conversation, and generally derails the conference call. Summary of the invention
[0003] This summary is intended to introduce a selection of concepts in a simplified form that are further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
[0004] Techniques for providing real-time feedback during a conference call are described. Operations may include receiving an indication from a client device that a conference call participant has unmuted the participant's microphone, wherein the conference call includes the participant and one or more other participants. Operations may also include receiving incoming audio from the participant's microphone. Operations may also include, during a feedback period triggered by the indication that the participant's microphone has been unmuted, sending the received incoming audio to the client device in real-time as it is received for playback to the participant, and to one or more other client devices associated with the one or more other participants for playback. The operations may also include, when the feedback period ends, sending the received incoming audio in real-time as it is received to one or more other client devices associated with the one or more other participants for playback, wherein after the feedback period ends, the incoming audio received from the participant's microphone is not sent to the computing device for playback to the participant.
[0005] As another example, an operation of providing real-time feedback during a conference call may include receiving an indication from a client device indicating that a microphone associated with a participant of the conference call has been unmuted, wherein the conference call includes the participant and one or more other participants. The operation may also include, during a feedback period starting from when the indication is received, and when incoming audio is received from the microphone of the participant, sending the received incoming audio to the client device in real time when received for playback to the participant, and to one or more other client devices associated with one or more other participants for playback. The operation may also include, when the feedback period ends, sending the received incoming audio to one or more other client devices associated with one or more other participants in real time when received for playback, wherein after the feedback period ends, the incoming audio received from the microphone of the participant is not sent to the computing device for playback to the participant. As another example, an operation of providing real-time feedback during a conference call may include receiving an indication from a client device indicating that screen sharing has been initiated by a participant of the conference call, wherein the conference call includes the participant and one or more other participants. The operation may also include: during a feedback period starting from when the indication is received, and when the shared screen content is received from the client device, generating a screen content thumbnail of the received shared screen content in real time. The operation may also include: during the feedback period, sending the screen content thumbnail to the client device in real time when the screen content thumbnail is generated for display to the participant. The operation may also include: during the feedback period, sending the received shared screen content in real time to one or more other client devices associated with one or more other participants for display when the received shared screen content is received. The operation may also include: when the feedback period ends, sending the received shared screen content in real time to one or more other client devices associated with one or more other participants for display when the received shared screen content is received, wherein, after the feedback period ends, the screen content thumbnail is not generated or sent to the computing device for display to the participant. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] Figure 1 is a diagram illustrating an example teleconferencing service that automatically provides real-time audio feedback during a feedback period.
[0007] Figure 2 is a diagram illustrating an example teleconferencing service that automatically provides real-time video during a feedback period.
[0008] Figure 3 is a diagram showing example screen content thumbnails displayed during a feedback period.
[0009] Figure 4is a flow chart of an example method for providing real-time feedback during a conference call when a microphone has been unmuted.
[0010] Figure 5 is a flow chart of another example method for providing real-time feedback during a conference call when a microphone has been unmuted.
[0011] Figure 6 is a flow chart of an example method for providing real-time feedback during a conference call in response to screen sharing.
[0012] Figure 7 is a diagram of an example computing system in which some described embodiments may be implemented.
[0013] Figure 8 is a sample cloud support environment that you can use with the techniques described in this article. DETAILED DESCRIPTION
[0014] Overview
[0015] The following description is directed to techniques for providing real-time audio and / or video feedback during a conference call. During a conference call, in response to a participant unmuting their microphone, audio feedback can be provided. For example, an indication can be received that the participant's microphone has been unmuted. In response, during a feedback period, incoming audio received from the participant's microphone can be sent in real time to the participant's computing device for playback to the participant and other computing devices that are part of the conference call (and other participants, if relevant). Once the feedback period ends, the participant will no longer receive incoming audio.
[0016] During a conference call, in response to receiving an indication that a conference call participant has initiated screen sharing, video feedback may be provided. For example, an indication that the participant has initiated screen sharing may be received. In response, during a feedback period, a screen content thumbnail may be generated from the shared screen content and sent to the participant's client device in real time for display to the participant. In addition, the shared screen content may be sent to other participants of the conference call in real time for display. Once the feedback period ends, the screen content thumbnail is not generated or sent to the participant's client device for display.
[0017] During a conference call, it can be difficult for a participant to know if others on the call can hear them. For example, a participant may often unmute their microphone and ask, “Can you hear me?” Asking others on the call if they can hear each other takes time and can disrupt the call. Similarly, if a participant begins speaking after unmuting themselves, the participant will not know what others hear (e.g., if they hear the participant or the volume of the participant’s voice). If there is a problem with a participant’s microphone, audio setup, or other issue, it can take a while for a participant to realize the problem. Each of these issues can cause a conference call to be disrupted.
[0018] During a conference call, it can also be difficult for a participant to know if others on the call can see their shared screen. For example, it is common for a participant to share their screen and ask, “Can you see my screen?” Having to ask others on the call if they can see the shared screen takes time and can disrupt the call. Similarly, if a participant just starts talking, the participant will not know what others are looking at. If there is a problem with the participant’s screen sharing setup or other issues, it may take some time before the participant is aware of the problem. Each of these issues can cause a conference call to be disrupted.
[0019] The technology described herein provides a solution to the problem of a participant unmuting their microphone and not knowing whether others in the conference call can hear them. In some solutions, when a participant unmutes, the participant's microphone audio is directed back to the participant for playback within a period of time after the unmuting occurs (also known as a feedback period). There are advantages to using this technology. For example, a participant can hear the participant's microphone audio in real time (e.g., when the participant is speaking) and when other participants hear the audio, allowing the participant to confirm that their microphone audio is working and being received by other participants. In addition, since the microphone audio playback to the participant ends after the feedback period ends, the participant will not be distracted after the feedback period ends.
[0020] There are differences and advantages between the technology described herein and previous solutions. One example difference is that a participant's microphone audio is provided to that participant in real time for playback (e.g., the audio is not recorded for a certain interval and then provided for playback after the interval is over). An advantage of this solution is that participants hear their microphone audio in real time as they speak and immediately know if their microphone is working properly and if their microphone audio is being provided to other participants on the conference call after unmuting.
[0021] Another example difference and advantage is that the participant's microphone audio is automatically sent back to the participant for playback without any manual intervention by the participant (e.g., the participant does not manually record microphone audio). An advantage of this technology is that since it is triggered by unmuting the participant's microphone, it requires less time and less user interface action by the participant.
[0022] Another example difference and advantage is that the participant's microphone audio is sent back to the participant by the teleconference service (more specifically, sent back to the participant's computing device for playback by the participant). One advantage of this technology is that the teleconference service sends the same microphone audio to all participants (including sending it back to the participant from whom the microphone audio was received) so that all participants will hear the same audio. Therefore, the participant who is speaking will receive the same audio as other participants (for example, can hear his or her own voice in the same way as other participants). This solution also provides advantages over local playback technology, in which the participant's microphone audio is played locally (rather than being sent to and received by the teleconference service), because with local playback, the participant does not know whether other participants can hear the participant's microphone audio (only that the participant's microphone is working locally, which does not confirm whether the participant's microphone audio is being sent to and received by the teleconference service, or whether it is being sent by the teleconference service to other participants).
[0023] The technology described herein provides a solution to the problem of a participant sharing their screen but not knowing whether others in the conference call can see their shared screen content. In some solutions, when a participant initiates screen sharing, the conference call service generates a screen content thumbnail (generated in real time as the shared screen content is received) and sends the screen content thumbnail back to the participant for display. There are advantages to using this technology. For example, a participant sees a screen content thumbnail representing their shared screen content in real time as the shared screen content is sent to and seen by other participants, allowing the participant to confirm that their screen sharing is working and being received by other participants. In addition, because the screen content thumbnail ends after the feedback period, the participant will not be distracted after the feedback period ends.
[0024] There are differences and advantages between the technology described in this article and previous solutions. One example difference is that the participant's shared screen content is returned to the participant in real time in the form of a screen content thumbnail for display. An advantage of this solution is that the participant sees their shared screen content in real time while presenting and immediately knows whether their screen sharing is working and being sent to other participants in the conference call.
[0025] Another example difference and benefit is that screen content thumbnails are automatically sent back to participants for display without any manual intervention by the participants. An advantage of this technique is that since it is triggered by initiating screen sharing, it requires less time and less user interface manipulation by the participants.
[0026] Another example difference and advantage is that screen content thumbnails are generated by the teleconferencing service and sent back to the participant's computing device for display. One advantage of this technology is that the teleconferencing service generates screen content thumbnails from the same shared screen content that is sent to other participants, so participants will be able to see and confirm what other participants are receiving.
[0027] the term
[0028] The term "conference call" refers to a call or meeting involving two or more computing devices (e.g., desktop computers, laptop computers, tablet computers, smartphones, etc.). Each computing device is associated with a participant (user) of the conference call. The conference call is managed by a conference call service (e.g., software and / or hardware resources that communicate with the participants' client devices, perform audio and / or video mixing, and perform other conference call operations).
[0029] The term "unmute" means that the participant's microphone is unmuted. For example, the participant can perform certain actions to unmute the participant's microphone (e.g., click an unmute button in the graphical user interface of the client application).
[0030] The term "screen sharing" refers to a conference call participant sharing their computer desktop or a portion of their computer desktop. For example, a conference call participant can select a user interface element (e.g., a "share" button or icon) to initiate sharing of their desktop, a window within their desktop, or another portion of their desktop.
[0031] Sample audio and video feedback during a conference call
[0032] In the technology described herein, during a conference call, audio feedback is provided in response to a conference call participant unmuting his or her microphone. During a feedback period after the microphone is unmuted, the participant's microphone output is sent back to the participant for playback (and also to other participants of the conference call). When the feedback period ends, the participant's microphone output is no longer sent back to the participant.
[0033] Figure 11 is a diagram depicting an example environment 100 including a teleconferencing service 110 for automatically providing real-time audio feedback during a feedback period. The teleconferencing service 110 is implemented using software and / or hardware resources (eg, computer servers, cloud computing resources, software resources, etc.).
[0034] Teleconference service 110 manages teleconferences for various clients of the service (e.g., client devices 130, 132, 134, and 136). Each client device (including client devices 130, 132, 134, and 136) is associated with a participant (user). For example, each client device can run client teleconference software (e.g., a client teleconference application or application) that allows the user of the client device to participate in a teleconference managed by teleconference service 110. In general, teleconference service 110 can manage teleconferences for any number of client devices and associated participants. Teleconference service 110 communicates with client devices via a computer network (e.g., including the Internet).
[0035] like Figure 1 As shown, the teleconference service 110 performs a number of operations for providing real-time feedback during a teleconference. In one example scenario, the teleconference service first receives an indication that a participant has unmuted the participant's microphone, as shown at 112. Next, during a feedback period (e.g., a few seconds) triggered by the indication that the participant's microphone has been unmuted, the teleconference service 110 sends the incoming audio from the participant's microphone in real time to each participant of the teleconference (including the participant whose microphone audio is being received), as shown at 114. When the feedback period ends, the teleconference service 110 sends the incoming audio received from the participant's microphone in real time to the other participants of the teleconference, but not to the participant whose microphone audio is being received, as shown at 116. The teleconference service 110 can perform these operations as long as the microphone of one of the participants of the teleconference has been unmuted.
[0036] To illustrate these operations, consider an example scenario where a conference call has four participants, associated with corresponding client devices 130, 132, 134, and 136. The participant associated with client device 130 has unmuted his microphone and has begun speaking. An indication of the unmute action performed on client device 130 is sent to and received by teleconference service 110.
[0037] Continuing with the example scenario, when the teleconference service 110 receives an indication of unmuting performed by a participant of the client device 130, a feedback period will be triggered. The feedback period can be a relatively short period of time (e.g., 5-10 seconds) starting from when the participant's microphone is unmuted, during which the participant will hear their own microphone output audio in the same manner as other participants. Specifically, in this example scenario, the teleconference service 110 will send the same microphone audio received from the client device 130 to each participant of the conference call (to the participants of the client devices 130, 132, 134, and 136). When the microphone audio is received from the client device 130, the teleconference service 110 sends the microphone audio received from the client device 130 to all participants in real time. In this way, while the participant is speaking, the client device 130 will receive its microphone audio from the teleconference service 110 for real-time playback to the participant.
[0038] Continuing with the example scenario, when the feedback period ends (e.g., after the initial 5-10 seconds after the participant's microphone has been unmuted), teleconferencing service 110 stops sending microphone audio received from client device 130 back to client device 130. Teleconferencing service 110 still sends microphone audio received from client device 130 to the other participants of the conference call (to client devices 132, 134, and 136). Thus, after the feedback period ends, the participant at client device 130 will no longer hear his or her own microphone audio when speaking.
[0039] In some embodiments, the teleconference service 110 uses an audience list to manage the participants who are currently receiving microphone audio. The audience list can be any type of data structure that indicates which participants are currently receiving microphone audio. Typically, when one of the participants of the teleconference is speaking (also referred to as a speaker or presenter), the audience list includes all other participants of the teleconference (except the participant who is speaking). Therefore, all other participants can hear what the presenter says, but the audio of the presenter will not be sent back to the presenter for playback. However, according to the technology described herein, when a participant unmutes his microphone, the teleconference service 110 will add the participant to the audience list together with the other participants of the teleconference. When the teleconference service 110 sends the audio received from the participant's microphone to the audience list, the participant will also receive audio for playback. Once the feedback period ends, the teleconference service 110 removes the participant from the audience list.
[0040] In some embodiments, the teleconference service 110 determines the audio level of the incoming audio from the participant's microphone.Then, the teleconference service 110 can send the indication of the determined audio level to the client device, where it can be presented to the participant of the client device.For example, the determined audio level can be displayed to the participant as a graphical user interface element (for example, a graphical slider or bar indicating the instantaneous or average audio level on a scale), or presented to the participant in another way (for example, as an indication of whether the determined audio level is low).In some embodiments, the determined audio level is sent to the client device only during the feedback period.According to the determined audio level presented to the participant, the participant can take some actions (for example, if the determined audio level is low, the participant can increase its microphone output).The technology described herein for automatically providing real-time audio feedback during the feedback period can also provide advantages when the participant accidentally or unintentionally unmutes his microphone.For example, when an unexpected or unintentional unmuting occurs, the participant will hear his own voice and / or other audio picked up by the microphone in real time, and know that they have been unmuted and other participants are hearing their microphone audio.Then, the participant can correct this situation (for example, by muting their microphone).
[0041] Figure 2 2 is a diagram depicting an example environment 200 including a teleconferencing service 110 for automatically providing real-time video feedback during a feedback period. In general, the teleconferencing service 110 may be the above Figure 1 The same teleconference service 110 described in , and is capable of providing real-time audio and / or video feedback during the feedback period. However, depending on the implementation, a given teleconference service may be configured to only provide real-time audio feedback or only provide real-time video feedback.
[0042] like Figure 2As shown, the teleconference service 110 performs multiple operations to provide real-time feedback during a teleconference. In an example scenario, the teleconference service 110 first receives an indication that a participant has initiated screen sharing during a teleconference, as shown in 210. Next, during a feedback period (e.g., several seconds) triggered by the participant's indication that screen sharing has been initiated (e.g., when the participant selects a share screen button to share the participant's computer screen (also known as the desktop) or a portion of the participant's computer screen (e.g., a specific window)), the teleconference service 110 generates and sends a screen content thumbnail to the participant in real time, as shown in 212. In addition, the teleconference service 110 sends the shared screen content from the participant to other participants of the teleconference, as shown in 212. When the feedback period ends, the teleconference service 110 stops generating and stops sending the screen content thumbnail to the participant, but continues to send the shared screen content received from the participant to other participants of the teleconference. The teleconference service 110 can perform these operations whenever a participant initiates screen sharing.
[0043] By sending the screen content thumbnail to the participant for display, the participant will know in real time whether screen sharing is working and whether other participants can see their shared screen content. If the participant sees the screen content thumbnail, and the thumbnail depicts their shared screen content, the participant knows that other participants are receiving their shared screen content because the teleconference service 110 is generating the screen content thumbnail based on the same shared screen content received from the participant's client device and sending it to the other participants for display. In addition, when the feedback period ends, the teleconference service 110 stops generating and stops sending the screen content thumbnail to the participant so that the participant is not distracted after having a chance to confirm correct screen sharing operation.
[0044] To illustrate these operations, consider an example scenario in which a conference call has four participants, associated with corresponding client devices 130, 132, 134, and 136. The participant associated with client device 132 has initiated screen sharing of at least a portion of the screen of client device 132. An indication that the participant of client device 132 has initiated screen sharing is sent to and received by teleconference service 110.
[0045] Continuing with the example scenario, when the teleconference service 110 receives an indication of screen sharing performed by a participant of client device 132, a feedback period is triggered. The feedback period can be a relatively short period of time (e.g., 5-10 seconds) from the time the participant initiates screen sharing, during which the participant will see a screen content thumbnail that depicts its shared screen content as seen by other participants. Specifically, in this example scenario, when the shared screen content is received from client device 132, the teleconference service 110 generates a screen content thumbnail in real time and sends the screen content thumbnail back to client device 132 for display. The teleconference service 110 also sends the shared screen content to the other participants of the teleconference (participants of client devices 130, 134, and 136) in real time. In this way, the participant of client device 132 will receive its shared screen content back from the teleconference service 110 in the form of a screen content thumbnail for display to the participant in real time. Continuing with the example scenario, when the feedback period ends (e.g., after the initial 5-10 seconds after the participant initiates screen sharing), teleconference service 110 stops generating screen content thumbnails and stops sending them to client device 132. However, while screen sharing is active, teleconference service 110 still sends the shared screen content to the other participants of the teleconference (client devices 130, 134, and 136). Thus, after the feedback period ends, the participant of client device 132 will no longer see the screen content thumbnail of its shared screen content.
[0046] Figure 3 300, which depicts an example screen content thumbnail 320 displayed during a feedback period. Specifically, diagram 300 depicts a screen (or a portion of a screen) of a client device (e.g., client device 132 or another client device). A participant associated with the client device has initiated screen sharing during a teleconference, which causes the client device to send shared screen content 310 to the teleconference service (e.g., to teleconference service 110 or to another teleconference service). In this example, shared screen content 310 is an application window containing example text and graphical content. When the teleconference service receives an indication from the client device that a participant of the teleconference has initiated screen sharing, the feedback period begins. During the feedback period, the teleconference service generates a screen content thumbnail 320 of the shared screen content 310 received from the client device, and sends the screen content thumbnail 320 to the client device for display, as shown in the screen content thumbnail 320 displayed on the client device. Screen content thumbnail 320 is a reduced version (e.g., reduced in size, resolution, bandwidth, etc.) of the shared screen content 310 received by the teleconference service.
[0047] When receiving the shared screen content 310 from the client device, the teleconference service generates the screen content thumbnail 320 in real time. In addition, the teleconference service sends the shared screen content 310 to other participants of the teleconference in real time (specifically, to the client devices associated with the other participants). Once the feedback period ends, the teleconference service will stop generating the screen content thumbnail 320 and stop sending the screen content thumbnail 320 to the client device (i.e., after the feedback period ends, the screen content thumbnail 320 will stop being received and displayed at the client device). However, the teleconference service will continue to send the shared screen content 310 to other participants until the participant ends screen sharing.
[0048] Methods for real-time feedback during conference calls
[0049] Figure 4 4 is a flow chart of an example method 400 for providing real-time feedback when a microphone has been unmuted during a conference call. For example, the example method 400 may be performed by a conference call service (e.g., conference call service 110). At 410, an indication is received that a conference call participant has unmuted his or her microphone. The conference call includes the participant and one or more other participants. At 420, incoming audio is received from the participant's microphone.
[0050] At 430, during a feedback period triggered by an indication that the participant's microphone has been unmuted, the received incoming audio, as it is received, is sent in real-time to the client device for playback to the participant. During the feedback period, as the incoming audio is received, it is also sent in real-time to one or more other client devices associated with one or more other participants for playback.
[0051] At 440, when the feedback period ends, the received incoming audio is still sent in real time to one or more other client devices associated with one or more other participants for playback when it is received. However, after the feedback period ends, the received incoming audio is not sent to the client device for playback to the participant.
[0052] Figure 5 1 is a flow chart of another example method 500 for providing real-time feedback when a microphone has been unmuted during a conference call. For example, the example method 500 may be performed by a conference call service (e.g., conference call service 110). At 510, an indication is received from a client device of a participant of a conference call that a microphone associated with the participant has been unmuted. The conference call includes the participant and one or more other participants.
[0053] At 520, during a feedback period beginning when an indication that a participant's microphone has been unmuted is received, and upon receiving incoming audio from the participant's microphone, the received incoming audio, as it is received, is sent in real-time to the client device for playback to the participant. During the feedback period, as the incoming audio is received, it is also sent in real-time to one or more other client devices associated with one or more other participants for playback.
[0054] At 530, when the feedback period ends, the received incoming audio is still sent to one or more other client devices associated with one or more other participants in real time for playback when it is received. However, after the feedback period ends, the received incoming audio will not be sent to the client device for playback to the participant.
[0055] Figure 6 6 is a flow chart of an example method 600 for providing real-time feedback in response to screen sharing during a conference call. For example, the example method 600 may be performed by a conference call service (e.g., conference call service 110). At 610, an indication is received from a client device that a participant of the conference call has initiated screen sharing. The conference call includes the participant and one or more other participants. At 620, during a feedback period starting from the time the indication that screen sharing has been initiated is received, and after receiving shared screen content from a computing device, multiple operations are performed. The operations include: generating a screen content thumbnail in real time based on the received shared screen content. The operations also include: sending the screen content thumbnail in real time to the client device for display to the participant when the screen content thumbnail is generated. The operations also include: sending the shared screen content in real time to one or more other client devices associated with one or more other participants for display when the shared screen content is received.
[0056] At 630, when the feedback period ends, the received shared screen content is still sent in real time to one or more other client devices associated with one or more other participants for display when it is received. However, after the feedback period ends, the screen content thumbnails are not generated or sent to the client devices for display to the participants.
[0057] Additional Examples
[0058] Example 1 is a computing device comprising: a processor; a network interface; and a memory; the computing device is configured to perform operations for providing real-time feedback during a conference call, the operations comprising: receiving an indication from a client device that a participant of the conference call has unmuted the participant's microphone, wherein the conference call includes the participant and one or more other participants; receiving incoming audio from the participant's microphone; during a feedback period triggered by the indication that the participant's microphone has been unmuted: sending the received incoming audio in real time as it is received to the client device for playback to the participant and to one or more other client devices associated with the one or more other participants; and when the feedback period ends: sending the received incoming audio in real time as it is received to the one or more other client devices associated with the one or more other participants for playback; wherein after the feedback period ends, the received incoming audio from the participant's microphone is not sent to the computing device for playback to the participant.
[0059] Example 2 includes the subject matter of Example 1, and further specifies that, based on an indication that the participant has unmuted, incoming audio from the participant's microphone is automatically sent to the client device during the feedback period for playback to the participant without intervention by the participant.
[0060] Example 3 includes the subject matter of any of Examples 1-2, and further specifies that the computing device operates a teleconference service that performs operations to manage the teleconference.
[0061] Example 4 includes the subject matter of any of Examples 1-3, and further specifies that the operations further include: determining an audio level of incoming audio from a microphone of the participant; and sending an indication of the determined audio level to the client device.
[0062] Example 5 includes the subject matter of Example 4, and further specifies that sending the indication of the determined audio level is performed only during the feedback period.
[0063] Example 6 includes the subject matter of any of Examples 4-5, and further specifies that the indication of the determined audio level is for display to the participant in a graphical user interface.
[0064] Example 7 includes the subject matter of any of Examples 1-6, and further specifies that the operations further include: maintaining an audience list indicating which participants of the conference call will receive microphone audio; wherein, during the feedback period, the participants are added to the audience list; and wherein, when the feedback period ends, the participants are removed from the audience list.
[0065] Example 8 includes the subject matter of any one of Examples 1-7, and further specifies that the operation also includes: receiving an indication that screen sharing has been initiated from the client device; during a second feedback period that begins when the indication that screen sharing has been initiated is received and when shared screen content is received from the client device: generating a screen content thumbnail of the received shared screen content in real time; when generating the screen content thumbnail, sending the screen content thumbnail to the client device in real time for display to the participant; and when receiving the shared screen content, sending the received shared screen content to the one or more other client devices associated with the one or more other participants in real time for display; and when the second feedback period ends: when receiving the shared screen content, sending the received shared screen content to the one or more other client devices associated with the one or more other participants in real time for display; wherein, after the second feedback period ends, the screen content thumbnail is not generated or the screen content thumbnail is not sent to the computing device for display to the participant.
[0066] Example 9 is a method implemented by a computing device for providing real-time feedback during a conference call, the method comprising: receiving an indication from a client device indicating that a microphone associated with a participant of the conference call has been unmuted, wherein the conference call includes the participant and one or more other participants; during a feedback period that begins when the indication is received and when incoming audio is received from the participant's microphone: sending the received incoming audio in real-time as it is received to the client device for playback to the participant and to one or more other client devices associated with the one or more other participants; and when the feedback period ends: sending the received incoming audio in real-time as it is received to the one or more other client devices associated with the one or more other participants for playback; wherein after the feedback period ends, the received incoming audio from the participant's microphone is not sent to the computing device for playback to the participant.
[0067] Example 10 includes the subject matter of Example 9, and further specifies that, based on an indication that the participant has unmuted, incoming audio from the participant's microphone is automatically sent to the client device during the feedback period for playback to the participant without intervention by the participant.
[0068] Example 11 includes the subject matter of any of Examples 9-10, and further specifies that the computing device operates a teleconference service that performs operations to manage the teleconference.
[0069] Example 12 includes the subject matter of any of Examples 9-11, and further specifies that the method further comprises: determining an audio level of incoming audio from a participant microphone; and sending an indication of the determined audio level to the client device.
[0070] Example 13 includes the subject matter of Example 12, and further specifies that the indication of the determined audio level is sent only during the feedback period.
[0071] Example 14 includes the subject matter of any of Examples 12-13, and further specifies that the indication of the determined audio level is for display to the participant in a graphical user interface.
[0072] Example 15 includes the subject matter of any of Examples 9-14, and further specifies that the method further includes: maintaining an audience list indicating which participants of the conference call will receive microphone audio; wherein, during the feedback period, the participants are added to the audience list; and wherein, when the feedback period ends, the participants are removed from the audience list.
[0073] Example 16 includes the subject matter of any one of Examples 9-15, and further specifies that the method also includes: receiving an indication that screen sharing has been initiated from the client device; during a second feedback period that begins when the indication that screen sharing has been initiated is received and when shared screen content is received from the client device: generating a screen content thumbnail of the received shared screen content in real time; when generating the screen content thumbnail, sending the screen content thumbnail to the client device in real time for display to the participant; and when receiving the shared screen content, sending the received shared screen content to the one or more other client devices associated with the one or more other participants in real time for display; when the second feedback period ends: when receiving the shared screen content, sending the received shared screen content to the one or more other client devices associated with the one or more other participants in real time for display; wherein, after the second feedback period ends, the screen content thumbnail is not generated or the screen content thumbnail is not sent to the computing device for display to the participant.
[0074] Example 17 includes the subject matter of any of Examples 1-16, and further specifies that, in response to the participant selecting an unmute button in a graphical user interface of a teleconference client running on the client device, receiving an indication that a microphone associated with the participant has been unmuted.
[0075] Example 18 is one or more computer-readable storage media storing computer-executable instructions for execution on a computing device to perform operations for providing real-time feedback during a conference call, the operations comprising: receiving an indication from a client device indicating that a participant of the conference call has initiated screen sharing, wherein the conference call includes the participant and one or more other participants; during a feedback period starting from the time the indication is received and when shared screen content is received from the client device: generating a screen content thumbnail of the received shared screen content in real time; when generating the screen content thumbnail, sending the screen content thumbnail to the client device in real time for display to the participant; and when receiving the shared screen content, sending the received shared screen content to one or more other client devices associated with the one or more other participants in real time for display; and when the feedback period ends: when receiving the shared screen content, sending the received shared screen content to the one or more other client devices associated with the one or more other participants in real time for display; wherein, after the feedback period ends, the screen content thumbnail is not generated or sent to the computing device for display to the participant.
[0076] Example 19 includes the subject matter of any of Examples 8, 16, or 18, and further specifies that, based on an indication that the participant has initiated screen sharing, the screen content thumbnail is automatically generated and sent to the client device for display to the participant without intervention by the participant.
[0077] Example 20 includes the subject matter of any of Examples 18-19, and further specifies that the computing device is operative to perform a teleconferencing service that manages teleconferencing operations.
[0078] Computing System
[0079] Figure 7 A general example of a suitable computing system 700 in which the described techniques may be implemented is depicted. Computing system 700 is not intended to suggest any limitation as to scope of use or functionality, as these techniques may be implemented in a variety of general-purpose or special-purpose computing systems.
[0080] refer to Figure 7 , the computing system 700 includes one or more processing units 710, 715 and memories 720, 725. Figure 7, the basic configuration 730 is contained within the dashed line. Processing units 710, 715 execute computer executable instructions. A processing unit may be a general purpose central processing unit (CPU), a processor in an application specific integrated circuit (ASIC), or any other type of processor. A processing unit may also include multiple processors. In a multi-processing system, multiple processing units execute computer executable instructions to increase processing power. For example, Figure 7 A central processing unit 710 and a graphics processing unit or co-processing unit 715 are shown. Tangible memory 720, 725 may be volatile memory (e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or some combination of both, accessible by the processing unit(s). The memory 720, 725 stores software 780 implementing one or more techniques described herein in the form of computer-executable instructions suitable for execution by the processing unit(s).
[0081] The computing system may have other functions. For example, the computing system 700 includes storage 740, one or more input devices 750, one or more output devices 760, and one or more communication connections 770. An interconnection mechanism (not shown) such as a bus, controller, or network interconnects the components of the computing system 700. Typically, operating system software (not shown) provides an operating environment for other software executed in the computing system 700 and coordinates the activities of the components of the computing system 700.
[0082] Tangible storage 740 may be removable or non-removable and include disks, magnetic tapes or cartridges, CD-ROMs, DVDs, or any other medium that can be used to store information and accessed within computing system 700. Storage 740 stores instructions for software 780 implementing one or more techniques described herein.
[0083] Input device(s) 750 may be a touch input device (e.g., a keyboard, mouse, pen, or trackball), a voice input device, a scanning device, or other device that provides input to the computing system 700. For video encoding, input device(s) 750 may be a camera, video card, TV tuner card, or similar device that accepts video input in analog or digital form, or reads video samples into a CD-ROM or CD-RW of the computing system 700. Output device(s) 760 may be a display, printer, speaker, CD burner, or other device that provides output from the computing system 700.
[0084] Communication connection(s) 770 allow communication with another computing entity over a communication medium. The communication medium transmits information (e.g., computer executable instructions, audio or video input or output, or other data) in the form of a modulated data signal. A modulated data signal is a signal that has one or more of its characteristics set or changed to encode information in the signal. By way of example, and not limitation, the communication medium may use electrical, optical, radio frequency, or other carriers.
[0085] These techniques may be described in the general context of computer-executable instructions, such as instructions contained in program modules, executed in a computing system on a target real or virtual processor. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The functionality of program modules may be combined or split between program modules as desired in various embodiments. The computer-executable instructions of program modules may be executed within a local or distributed computing system.
[0086] The terms "system" and "device" are used interchangeably herein. Unless the context clearly indicates otherwise, neither term is meant to limit the type of computing system or computing device. In general, a computing system or computing device may be local or distributed, and may include any combination of special-purpose hardware and / or general-purpose hardware and software that implements the functionality described herein.
[0087] For ease of presentation, the detailed description uses terms such as "determine" and "use" to describe computer operations in a computing system. These terms are high-level abstractions of operations performed by a computer and should not be confused with actions performed by a human. The actual computer operations corresponding to these terms vary depending on the implementation.
[0088] Cloud Support Environment
[0089] Figure 8 A general example of a suitable cloud support environment 800 is shown in which the described embodiments, techniques, and technologies can be implemented. In the example environment 800, the cloud 810 provides various types of services (e.g., computing services). For example, the cloud 810 may include a set of computing devices, which may be centrally located or distributed, that provide cloud-based services to various types of users and devices connected via a network such as the Internet. The implementation environment 800 may be used in different ways to complete computing tasks. For example, some tasks (e.g., processing user input and presenting a user interface) may be performed on a local computing device (e.g., connected devices 830, 840, 850), while other tasks (e.g., storing data to be used in subsequent processing) may be performed in the cloud 810.
[0090] In the example environment 800, the cloud 810 provides services for connected devices 830, 840, and 850 with various screen functions. The connected device 830 represents a device with a computer screen 835 (e.g., a medium-sized screen). For example, the connected device 830 can be a personal computer, such as a desktop computer, a laptop computer, a notebook, a netbook, etc. The connected device 840 represents a device with a mobile device screen 845 (e.g., a small-sized screen). For example, the connected device 840 can be a mobile phone, a smart phone, a personal digital assistant, a tablet computer, etc. The connected device 850 represents a device with a large screen 855. For example, the connected device 850 can be a TV screen (e.g., a smart TV) or other devices connected to the TV (e.g., a set-top box or a game console), etc. One or more of the connected devices 830, 840, and 850 can include touch screen functionality. The touch screen can accept input in different ways. For example, a capacitive touch screen detects touch input when an object (e.g., a fingertip or a stylus) distorts or interrupts the current flowing through the surface. As another example, a touch screen can use an optical sensor to detect touch input when a light beam from the optical sensor is interrupted. Some touch screens can detect input without physical contact with the screen surface. Devices without screen functionality can also be used in example environment 800. For example, cloud 810 can provide services for one or more computers (e.g., server computers) without a display.
[0091] Cloud 810 can provide services through service provider 820 or other online service providers (not depicted). For example, cloud services can be customized according to the screen size, display capabilities and / or touch screen capabilities of specific connected devices (e.g., connected devices 830, 840, 850).
[0092] In example environment 800, cloud 810 provides the techniques and solutions described herein to various connected devices 830, 840, 850, at least in part using service provider 820. For example, service provider 820 can provide a centralized solution for various cloud-based services. Service provider 820 can manage service subscriptions for users and / or devices (e.g., connected devices 830, 840, 850 and / or their respective users).
[0093] Example Implementation
[0094] Although the operations of some disclosed methods are described in a particular order for ease of presentation, it should be understood that this manner of description includes rearrangement unless specific language described below requires a particular order. For example, in some cases, the operations described in sequence can be rearranged or performed simultaneously. In addition, for simplicity, the accompanying drawings may not show the various ways in which the disclosed methods can be used in conjunction with other methods. Any of the disclosed methods can be implemented as computer executable instructions or computer program products stored on one or more computer-readable storage media and executed on a computing device (i.e., any available computing device, including a smartphone or other mobile device that includes computing hardware). Computer-readable storage media are tangible media that can be accessed within a computing environment (one or more optical disks, such as DVDs or CDs, volatile memory (such as DRAM or SRAM), or non-volatile memory (such as flash memory or hard disk)). As an example and with reference to Figure 7 , computer-readable storage media include memories 720 and 725 and storage device 740. The term computer-readable storage medium does not include signals and carrier waves. In addition, the term computer-readable storage medium does not include communication connections, such as 770.
[0095] Any computer executable instructions for implementing the disclosed technology and any data created and used during the implementation of the disclosed embodiments may be stored on one or more computer readable storage media. The computer executable instructions may be, for example, a dedicated software application or part of a software application accessed or downloaded through a web browser or other software application (e.g., a remote computing application). Such software may be executed, for example, on a single local computer (e.g., any suitable commercial computer) or in a network environment (e.g., via the Internet, a wide area network, a local area network, a client-server network (e.g., a cloud computing network), or other such networks) using one or more network computers.
[0096] For the sake of clarity, only certain selected aspects of the software-based implementation are described. Other details well known in the art are omitted. For example, it should be understood that the disclosed technology is not limited to any particular computer language or program. For example, the disclosed technology can be implemented by software written in C++, Java, Perl, or any other suitable programming language. Likewise, the disclosed technology is not limited to any particular computer or hardware type. Certain details of suitable computers and hardware are well known and need not be elaborated in this disclosure.
[0097] Furthermore, any software-based embodiment (including, for example, computer-executable instructions for causing a computer to perform any disclosed method) can be uploaded, downloaded, or remotely accessed via appropriate communication means, including, for example, the Internet, the World Wide Web, an intranet, a software application, cables (including fiber optic cables), magnetic communications, electromagnetic communications (including radio frequency, microwave, and infrared communications), electronic communications, or other such communications means.
[0098] The disclosed methods, apparatus, and systems should not be construed as limiting in any way. Instead, the present disclosure is directed to all novel and non-obvious features and aspects of the various disclosed embodiments, individually and in various combinations and sub-combinations with each other. The disclosed methods, apparatus, and systems are not limited to any particular aspect or feature or combination thereof, nor do the disclosed embodiments require the presence of any one or more specific advantages or problems to be solved.
[0099] The technology in any example can be combined with the technology described in any one or more other examples. Given that the principles of the disclosed technology can be applied to many possible embodiments, it should be recognized that the illustrated embodiments are examples of the disclosed technology and should not be considered as limiting the scope of the disclosed technology.
Claims
1. A computing device, comprising: processor; Network interface; as well as Memory; The computing device is configured to perform operations for providing real-time feedback during a conference call, the operations comprising: receiving, from a client device, an indication that a participant of the conference call has unmuted the participant's microphone, wherein the conference call includes the participant and one or more other participants; receiving incoming audio from a microphone of the participant; During a feedback period triggered by an indication that the participant's microphone has been unmuted: sending the received incoming audio in real-time as it is received to the client device for playback to the participant and to one or more other client devices associated with the one or more other participants for playback; and When the feedback period ends: sending the received incoming audio in real-time as it is received to the one or more other client devices associated with the one or more other participants for playback; Wherein, after the feedback period ends, the received incoming audio from the participant's microphone is not sent to the computing device for playback to the participant.
2. The computing device of claim 1, wherein: Based on the indication that the participant has unmuted, the incoming audio from the participant's microphone is automatically sent to the client device for playback to the participant during the feedback period without intervention by the participant.
3. The computing device of claim 1, wherein: The computing device runs a teleconference service that performs operations for managing the teleconference.
4. The computing device of claim 1, the operations further comprising: determining an audio level of the incoming audio from the participant's microphone; as well as An indication of the determined audio level is sent to the client device.
5. The computing device of claim 4, wherein: Sending said indication of the determined audio level is performed only during said feedback period.
6. The computing device of claim 4, wherein: The indication of the determined audio level is used for display to the participant in a graphical user interface.
7. The computing device of claim 1, the operations further comprising: maintaining an audience list indicating which participants of the conference call will receive microphone audio; wherein, during the feedback period, the participant is added to the audience list; and When the feedback period ends, the participant is removed from the audience list.
8. The computing device of claim 1, the operations further comprising: receiving an indication from the client device that screen sharing has been initiated; During a second feedback period starting from when the indication that screen sharing has been initiated is received, and after receiving shared screen content from the client device: Generating a screen content thumbnail of the received shared screen content in real time; When generating the screen content thumbnail, sending the screen content thumbnail to the client device in real time for display to the participant; as well as When receiving the shared screen content, sending the received shared screen content to the one or more other client devices associated with the one or more other participants in real time for display; and When the second feedback period ends: When receiving the shared screen content, sending the received shared screen content in real time to the one or more other client devices associated with the one or more other participants for display; After the second feedback period ends, the screen content thumbnail is not generated or sent to the computing device for display to the participant.
9. A method implemented by a computing device for providing real-time feedback during a conference call, the method comprising: receiving, from a client device, an indication that a microphone associated with a participant of the conference call has been unmuted, wherein the conference call includes the participant and one or more other participants; During a feedback period beginning upon receipt of the indication, and upon receipt of incoming audio from the participant's microphone: sending the received incoming audio in real-time as it is received to the client device for playback to the participant and to one or more other client devices associated with the one or more other participants for playback; and When the feedback period ends: sending the received incoming audio in real-time as it is received to the one or more other client devices associated with the one or more other participants for playback; Wherein, after the feedback period ends, the received incoming audio from the participant's microphone is not sent to the computing device for playback to the participant.
10. The method according to claim 9, wherein: Based on the indication that the participant has unmuted, the incoming audio from the participant's microphone is automatically sent to the client device for playback to the participant during the feedback period without intervention by the participant.
11. The method according to claim 9, wherein: The computing device runs a teleconference service that performs operations for managing the teleconference.
12. The method according to claim 9, further comprising: maintaining an audience list indicating which participants of the conference call will receive microphone audio; wherein, during the feedback period, the participant is added to the audience list; and When the feedback period ends, the participant is removed from the audience list.
13. The method according to claim 9, further comprising: receiving an indication from the client device that screen sharing has been initiated; During a second feedback period starting from when the indication that screen sharing has been initiated is received, and after receiving shared screen content from the client device: Generating a screen content thumbnail of the received shared screen content in real time; When generating the screen content thumbnail, sending the screen content thumbnail to the client device in real time for display to the participant; as well as When receiving the shared screen content, sending the received shared screen content to the one or more other client devices associated with the one or more other participants in real time for display; and When the second feedback period ends: When receiving the shared screen content, sending the received shared screen content in real time to the one or more other client devices associated with the one or more other participants for display; After the second feedback period ends, the screen content thumbnail is not generated or sent to the computing device for display to the participant.
14. One or more computer-readable storage media storing computer-executable instructions for execution on a computing device to perform operations for providing real-time feedback during a conference call, the operations comprising: receiving an indication from a client device that a participant of the conference call has initiated screen sharing, wherein the conference call includes the participant and one or more other participants; During a feedback period starting from when the indication is received, and after receiving the shared screen content from the client device: Generating a screen content thumbnail of the received shared screen content in real time; When generating the screen content thumbnail, sending the screen content thumbnail to the client device in real time for display to the participant; and When receiving the shared screen content, sending the received shared screen content in real time to one or more other client devices associated with the one or more other participants for display; and When the feedback period ends: When receiving the shared screen content, sending the received shared screen content in real time to the one or more other client devices associated with the one or more other participants for display; After the feedback period ends, the screen content thumbnail is not generated or sent to the computing device for display to the participant.
15. One or more computer-readable storage media according to claim 14, wherein: Based on the indication that the participant has initiated screen sharing, the screen content thumbnail is automatically generated and sent to the client device for display to the participant during the feedback period without intervention by the participant.
