Identifying virtual meetings for absent users

Abstract

This invention provides a method and system for identifying virtual meetings for absent users. [Solution] The method includes identifying a calendar event corresponding to a virtual meeting using a first user's calendar application and based on one or more calendar event criteria. The first user of the calendar application is invited to attend the virtual meeting. The method also includes making the calendar application UI present to the first user. The calendar application UI includes a UI element that requests the first user to provide virtual meeting information to the first user who is unable to attend the virtual meeting. The method further includes providing the first user's request for virtual meeting information to the virtual meeting application in response to user activation of the UI element.

Description

Technical Field

[0001] Aspects and embodiments of the present disclosure relate to virtual meetings, and more particularly to identifying virtual meetings for the participation of absent users.

Background Art

[0002] A virtual meeting can be held among multiple participants via a virtual meeting platform. The virtual meeting platform can include the connection of multiple client devices through a network, as well as tools that enable the sharing of each other's audio (e.g., the voice of a user recorded using the microphone of a client device) and / or video streams (e.g., video captured by the camera of a client device, or video captured from the screen image of a client device) for efficient communication. For this purpose, the virtual meeting platform can provide a user interface that includes multiple areas for presenting the video streams of each participating client device.

Summary of the Invention

[0003] The following summary is a simplified overview of the present disclosure to provide a basic understanding of some aspects of the present disclosure. This summary is not an extensive overview of the present disclosure. It is not intended to identify key or critical elements of the present disclosure or to delineate any scope of the specific embodiments of the present disclosure or any scope of the claims. Its sole purpose is to present some concepts of the present disclosure in a simplified form as a prelude to the more detailed description that will be presented later.

[0004] One aspect of this disclosure provides a method. The method includes identifying a calendar event corresponding to a virtual meeting by a first user's calendar application and based on one or more calendar event criteria. The first user of the calendar application may be invited to attend the virtual meeting. The method includes presenting the calendar application user interface (UI) to the first user. The calendar application UI may include a UI element that requests the first user to provide virtual meeting information to the first user who is unable to attend the virtual meeting. The method includes providing the first user's virtual meeting information request to the virtual meeting application in response to user activation of the UI element.

[0005] Other aspects of this disclosure provide a system. The system includes memory and a processing device coupled to the memory. The processing device is configured to perform an operation. The operation includes identifying a calendar event corresponding to a virtual meeting by a first user's calendar application and based on one or more calendar event criteria. The first user of the calendar application may be invited to attend the virtual meeting. The operation includes making the calendar application user interface (UI) present to the first user. The calendar application UI may include UI elements that request the first user to provide virtual meeting information to the first user who is unable to attend the virtual meeting. The operation includes providing the first user's virtual meeting information request to the virtual meeting application in response to user activation of the UI element.

[0006] Other aspects of the present disclosure, when executed by a processing device, provide a non-temporary computer-readable storage medium containing instructions that cause the processing device to perform an action. The action includes a first user's calendar application identifying a calendar event corresponding to a virtual meeting based on one or more calendar event criteria. The first user of the calendar application may be invited to attend the virtual meeting. The action includes making the calendar application user interface (UI) present to the first user. The calendar application UI may include UI elements that request the first user to provide virtual meeting information to the first user who is unable to attend the virtual meeting. The action includes providing the first user's virtual meeting information request to the virtual meeting application in response to user activation of the UI element.

[0007] The aspects and embodiments of this disclosure will be better understood from the detailed description below and from the accompanying drawings of various aspects and embodiments of this disclosure, but these drawings should not be construed as limiting this disclosure to any particular aspect or embodiment, and are for illustrative purposes only. [Brief explanation of the drawing]

[0008] [Figure 1] This disclosure provides exemplary system architectures for identifying virtual meetings for the participation of absent users, according to several embodiments of this disclosure. [Figure 2A] The following are schematic block diagrams of the artificial intelligence (AI) training subsystem of a virtual meeting platform according to several embodiments of this disclosure. [Figure 2B] A schematic block diagram of the AI ​​inference subsystem of a virtual meeting platform according to several embodiments of this disclosure is shown. [Figure 3] The flowcharts below illustrate methods for identifying virtual meetings for the participation of absent users, according to several embodiments of this disclosure. [Figure 4] This disclosure illustrates exemplary user interfaces for a calendar application according to several embodiments of this disclosure. [Figure 5] Other exemplary user interfaces for a calendar application, according to some embodiments of this disclosure, are shown. [Figure 6] This disclosure illustrates exemplary user interfaces for virtual meetings according to several embodiments of this disclosure. [Figure 7] This disclosure illustrates exemplary user interfaces for a calendar application according to several embodiments of this disclosure. [Figure 8] A block diagram shows an exemplary computer system according to some embodiments of the present disclosure. [Modes for carrying out the invention]

[0009] Aspects of this disclosure relate to identifying virtual meetings for the participation of absent users. A virtual meeting platform enables video-based conferencing among multiple participants via each client device connected over a network, and can share each other's audio (e.g., user voices recorded using the client device's microphone) and / or video streams (e.g., video captured by the client device's camera) during the virtual meeting. In some examples, a virtual meeting platform may allow a large number of client devices (e.g., up to 100 or more client devices) to be connected through the virtual meeting. Participants in the virtual meeting can speak with other participants in the virtual meeting. Some existing virtual meeting platforms can provide a user interface (UI) to each client device connected to the virtual meeting, and the UI displays visual items corresponding to the video stream shared over the network within a set of areas within the UI.

[0010] For example, some users may be unable to attend a virtual meeting, or even the entire meeting, because they might have multiple meetings scheduled at the same time, or their first meeting might run longer than expected and overlap with a subsequent meeting. In a typical virtual meeting, if a user is absent during the meeting, or absent for part of the meeting, they cannot participate in the virtual meeting or the portion they missed, nor can they offer their opinion on the topics being discussed. This has several drawbacks. For example, a user invited to a virtual meeting but unable to attend cannot offer their opinion on the topics being discussed, resulting in a decrease in the efficiency and effectiveness of the meeting. Furthermore, participants attending the virtual meeting may need to take notes for absent users, which can be distracting for note-takers and may prevent them from fully participating in the meeting. Additionally, note-takers may miss some points of discussion or misinterpret the items being discussed. Next, note-taking users may need to send notes to absent users (e.g., via email) or conduct other virtual meetings with absent users to provide information they missed, which could consume computing system resources. Furthermore, participating in numerous virtual meetings can be exhausting for users.

[0011] Embodiments of the present disclosure address the above and other deficiencies by providing a system and method that enables a user invited to attend a virtual meeting to attend the virtual meeting on behalf of another user. Attendance by a substitute user may mean that the user participates in the virtual meeting without being physically present in the virtual meeting when it takes place. Embodiments of the present disclosure enable a substitute user attending a virtual meeting to provide points of discussion or other data to be discussed during the virtual meeting for presentation during the meeting. Additionally or alternatively, a substitute user attending a virtual meeting may be provided with a summary of the virtual meeting. Thus, it may be possible for a user to participate in a virtual meeting without being physically present.

[0012] Embodiments of this disclosure also provide a system and method for identifying virtual meetings that are suitable for a user to attend on their behalf. In one embodiment, the system identifies a calendar event to which a user has been invited and determines whether the calendar event satisfies one or more calendar event criteria. A calendar event that satisfies one or more of the criteria may be suitable for a user to attend on their behalf. Such criteria may include calendar events that indicate the user's attendance is optional, calendar events that occur during a period when the user is absent (e.g., due to scheduling conflicts, illness, or other constraints such as vacation), or calendar events in a set of calendar events to which the user has previously declined to attend. The system can cause the calendar application UI to present the user with an option to indicate that the user is unable to attend but will send a substitute to attend the virtual meeting associated with the calendar event. In response to the user selecting the option to send a substitute, the user can input discussion points or other data to be presented during the virtual meeting in the user's absence.

[0013] Aspects of this disclosure offer superior technical advantages over previous solutions. Aspects of this disclosure provide an automated process for identifying calendar events associated with virtual meetings that are suitable for a user to attend on their behalf. Thus, the user does not need to spend time deciding whether or not to attend such virtual meetings. Such automation improves the user's virtual meeting experience, allowing the user to perform other tasks instead of deciding which virtual meetings to attend and which to skip. Aspects of this disclosure provide a way for a user who is not present during a virtual meeting to provide discussion points or other materials for use during the virtual meeting. Aspects of this disclosure provide access to one or more AI-generated summaries of the discussion of the provided discussion points and other materials, which improves the efficiency of the virtual meeting and the absent user. Furthermore, aspects of this disclosure reduce the need for note-taking virtual meeting participants to follow up with absent users, thereby reducing the use of computing system resources (for example, by reducing the number of emails sent from note-taking participants to absent users, and by reducing additional virtual meetings between note-taking and absent users).

[0014] Figure 1 shows an exemplary system architecture 100 according to an embodiment of the present disclosure. The system architecture 100 includes one or more client devices 102A-N or 104, a virtual meeting platform 120, a server 130, and a data store 150, each connected to a network 160.

[0015] In some embodiments, the virtual meeting platform 120 enables one or more users of client devices 102A-N, 104 to connect with each other to a virtual meeting (e.g., virtual meeting 122). The virtual meeting 122 refers to a real-time communication session such as a video-based call or video chat, in which participants can connect with multiple additional participants in real time, and audio and video capabilities may be provided. The virtual meeting 122 includes an audio-based call or chat, in which participants can connect with multiple additional participants in real time, and audio capabilities may be provided. Real-time communication refers to the ability of users to communicate instantaneously (e.g., exchange information) with no and / or negligible (e.g., milliseconds or microseconds) latency. The virtual meeting platform 120 may enable users of the virtual meeting platform 120 to join and engage with the virtual meeting 122 with other users of the virtual meeting platform 120 (users sometimes referred to herein as “virtual meeting participants” or simply “participants”). Embodiments of the present disclosure can be implemented with any number of participants (e.g., up to 100 or more) connecting via the virtual meeting 122.

[0016] In embodiments of this disclosure, “User” or “Participant” may represent a single individual. However, other embodiments of this disclosure may include “User” as a group or organization of Users and / or an entity controlled by an automated source such as a system or platform. Where the systems described herein collect or can access personal information about a User, the User may be provided with the opportunity to control whether the virtual meeting platform 120 or the virtual meeting manager 132 collects User information (e.g., information about the User’s social networks, social behavior or activities, occupation, User’s preferences, or User’s current location), or whether and / or how it receives content that may be more relevant to the User from the virtual meeting platform 120 or the virtual meeting manager 132. Furthermore, certain data may be processed in one or more ways before being stored or used so that personally identifiable information is removed. For example, User-identifying information may be processed so that personally identifiable information cannot be determined, or, if location information is obtained, the User’s geographical location may be generalized (to the city, zip code, or state level, etc.) so that the User’s specific location cannot be determined. Therefore, users can control how information about them is collected and used by the virtual meeting platform 120 or the virtual meeting manager 132.

[0017] In some embodiments, the server 130 includes a virtual meeting manager 132. In one or more embodiments, the virtual meeting manager 132 is configured to manage a virtual meeting 122 among multiple users of the virtual meeting platform 120. The virtual meeting manager 132 may provide UIs 108A-N to each client device 102A-N, 104 to enable users to see and hear each other during the virtual meeting 122. The virtual meeting manager 132 can also collect data associated with the virtual meeting 122 and provide it to each participant in the virtual meeting 122. In some embodiments, the virtual meeting manager 132 provides UIs 108A-N for presentation by client applications 105A-N. For example, each UI 108A-N can be displayed on display devices 107A-107N by client applications 105A-N running on the operating systems of client devices 102A-N, 104. In some embodiments, the virtual meeting manager 132 determines visual items to be presented in UI 108A-N during the virtual meeting 122. Visual items may refer to UI elements that occupy specific areas of UI 108A-N and are specialized in presenting video streams from each client device 102A-N, 104. Such video streams may, for example, depict the users of each client device 102A-N, 104. Such situations include while a user is participating in the virtual meeting 122 (e.g., at a specific moment during the virtual meeting 122, speaking, presenting, listening to another participant speak, viewing another participant, etc.), while a user is in a physical conference room or meeting room (e.g., with one or more participants), or while a document or media content (e.g., video content, one or more images, etc.) is being presented during the virtual meeting 122.

[0018] In some embodiments, the virtual meeting manager 132 includes a video stream processor 134 and a UI controller 136. Each of the video stream processor 134 or the UI controller 136 may include a software application (or a subset thereof) that provides the virtual meeting manager 132 with specific virtual meeting functionality. The video stream processor 134 can be configured to receive video streams from one or more client devices 102A-N, 104. The video stream processor 134 can be configured to determine visual items to present to the UI (e.g., UIs 108-108N described below) of such client devices 102A-N, 104 during the virtual meeting 122. Each visual item may correspond to a video stream from the client devices 102A-N, 104 (e.g., a video stream relating to one or more participants in the virtual meeting 122). In some embodiments, the virtual meeting 122 further includes, for each participant of the one or more participants, first audio data associated with an audio stream generated by each participant's client device 102A-N, 104. The video stream processor 134 can receive audio streams associated with video streams from client devices (for example, from the audiovisual components of client devices 102A-N and 104). When the video stream processor 134 determines a visual item to be presented in the UI, it can notify the UI controller 136 of the determined visual item. The visual item to be presented can be determined based on the current speaker, the current presenter, the order of participants joining the virtual meeting 122, a participant list (for example, alphabetically), etc.

[0019] In some embodiments, the UI controller 136 provides a UI to the virtual meeting 122. The UI may include multiple regions. Each region can display a video stream relating to one or more participants of the virtual meeting 122. The UI controller 136 can control which video streams are displayed by providing one or more client devices 102A-104 with commands indicating which video stream should be displayed in which region of the UI (along with the received video and audio streams provided to client devices 102A-N, 104). For example, in response to notification of visual items decided to be presented in UI 108A-N, the UI controller 136 can transmit commands causing each decided visual item to be displayed in a region of the UI and / or rearranged in the UI.

[0020] In one or more embodiments, the server 130 includes an absent user manager 138. The absent user manager 138 may include a software application (or a subset thereof) that implements specific virtual meeting functions for the virtual meeting manager 132 or the calendar application 140 (described below). As described herein, the absent user manager 138 may be configured to present data associated with users who are absent from a virtual meeting 122 and to generate one or more summaries based on the virtual meeting 122 or other virtual meeting functions. The absent user manager 138 may include an AI inference subsystem 139. The AI ​​inference subsystem 139 may include one or more AI models that are configured to generate transcripts of the virtual meeting 122, generate one or more summaries of the virtual meeting 122, and implement other functions described herein. The absent user manager 138 can use the AI ​​inference subsystem 139 to assist the absent user manager 138 in performing one or more operations. Some aspects of the functionality of the absent user manager 138 are described below in reference to Figure 3.

[0021] In one embodiment, server 130 includes a calendar application 140. The calendar application 140 may include a software application (or a subset thereof) that implements calendar functionality. The calendar application 140 may provide a digital calendar to the user, enabling the user to organize calendar events, send calendar invitations associated with the calendar events to one or more other users, or implement other calendar functions. In some embodiments, the calendar application 140 integrates with the virtual meeting platform 120 or the virtual meeting manager 132 to provide certain calendar functionality to the virtual meeting platform 120 or the virtual meeting manager 132, or to enable the virtual meeting platform 120 or the virtual meeting manager 132 to provide certain virtual meeting functionality to the calendar application 140. For example, a link to virtual meeting 122 may be included in a calendar event of the calendar application 140. The calendar application 140 can receive discussion points from users who cannot attend virtual meeting 122, and the calendar application 140 can provide those discussion points for presentation during virtual meeting 122, as further described below. FIG. 1 shows the calendar application 140 running on server 130, but in some embodiments, the calendar application 140 runs on client devices 102A-N, 104.

[0022] In one embodiment, a calendar invitation includes a media type that enables a user to store and exchange schedule information and scheduling information for calendar events. The calendar invitation can be generated by a calendar application 140. The calendar application 140 can access the calendar invitation and be configured to display information based on the calendar invitation (e.g., data indicating the user who organized the corresponding calendar event, start time, end time, location of the calendar event (which may include a physical location or may include data used to access a virtual meeting 122, etc.)).

[0023] In one or more embodiments, the calendar application 140 includes an event identification manager 142. The event identification manager 142 can include a software application (or a subset thereof) that identifies calendar events of the calendar application 140 that a user may be suitable to attend on behalf of, as further described below. As used herein, a user who uses the absentee user manager 138 and / or the calendar application 140 to indicate that the user cannot attend the virtual meeting 122, provide data (e.g., discussion notes) presented during the virtual meeting 122, generate one or more summaries associated with the absent virtual meeting 122, and / or perform other related operations may be referred to herein as a user who attends the virtual meeting 122 on behalf of.

[0024] In some embodiments, each of the virtual meeting platform 120 or server 130 includes one or more computing devices (such as rack-mount servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, and desktop computers), data stores (e.g., hard disks, memory, and databases), networks, software components, and / or hardware components that can be used to enable users to connect with other users through the virtual meeting 122. The virtual meeting platform 120 may also include a website (e.g., one or more web pages) or application backend software that can be used to enable users to connect with other users through the virtual meeting 122.

[0025] In some embodiments, one or more client devices 102A-N each include one or more computing devices such as a personal computer (PC), laptop, mobile phone, smartphone, tablet computer, netbook computer, or network-connected television. One or more client devices 102A-N may also be called “user devices”. Each client device 102A-N may include an audiovisual component capable of generating audio and video data to be streamed to the virtual meeting manager 132. The audiovisual component may include a device (e.g., a microphone) for capturing audio signals representing user speech and generating audio data (e.g., an audio file) based on the captured audio signals. The audiovisual component may include other devices (e.g., speakers) for outputting audio data to a user associated with a particular client device 102A-N. In some embodiments, the audiovisual component includes an image capture device (e.g., a camera) for capturing images and generating video data (e.g., a video stream) of the captured images.

[0026] In some embodiments, the system architecture 100 includes a client device 104. The client device 104 may differ from one or more client devices 102A-N, as it may be associated with a physical conference room or meeting room. Such a client device 104 may include, or be coupled to, a media system 110, which may include one or more display devices 112, one or more speakers 114, and one or more cameras 116. The display device 112 may be, for example, a smart display or a non-smart display (e.g., a display not configured to connect to a network 160). Users physically present in a room may participate in a virtual meeting 122 using the media system 110 rather than their own devices (e.g., one or more of the client devices 102A-N), and such users may include other remote users. For example, a user in a room participating in a virtual meeting 122 may control the display device 112 to show a slide presentation or view another participant's slide presentation. Similarly, they may also control the sound and / or camera. Similar to one or more client devices 102A-N, client device 104 can generate audio and video data to be streamed to the virtual meeting manager 132 (for example, using one or more microphones, speakers 114, and cameras 116).

[0027] As described above, the audiovisual component of each client device 102A-104 can capture images and generate video data (e.g., a video stream) of the captured images. In some embodiments, client devices 102A-N, 104 transmit the generated video stream to the virtual meeting manager 132. The audiovisual component of each client device 102A-N, 104 can also capture audio signals representing user speech and generate audio data (e.g., an audio file or audio stream) based on the captured audio signals. In some embodiments, client devices 102A-N, 104 transmit the generated audio data to the virtual meeting manager 132.

[0028] In some embodiments, each client device 102A-N or 104 includes its respective client application 105A-N, which may be a mobile application, desktop application, web browser, etc. The client applications 105A-N can present one or more features of the applications 105A-N to the display devices 107A-N or UI (e.g., the UI of UI 108A-N) of the client devices 102A-N for the user to access the virtual meeting platform 120. For example, a user of the first client device 102A can join and engage with the virtual meeting 122 via UI 108A presented on the display device 107A by application 105A. The user can present documents to participants of the virtual meeting 122 via UI 108A. Each of UI 108A-N may include multiple areas that present visual items corresponding to the video streams of the client devices 102A-N provided to the server 130 for the virtual meeting 122. In one embodiment, client applications 105A to N can present one or more features of applications 105A to N on the display devices 107A to N or UI of client devices 102A to N so that the user can access the calendar application 140. For example, a user of the first client device 102A can view their calendar via UI 108A presented on display device 107A by application 105A.

[0029] In one or more embodiments, at least some parts of the virtual meeting manager 132, the absent user manager 138, and / or the calendar application 140 are part of client devices 102A-N, 104. For example, applications 105A-N may include the absent user manager 138 and / or the calendar application 140, which can present data associated with absent virtual meeting users, generate summaries based on virtual meetings 122, present calendars, generate calendar events, and implement other functions. In some embodiments, application 105A of the first client device 102A transmits the video stream generated by client device 102A to other client devices 102B~N, 104, receives the video stream from other client devices 102B~N, 104, and applications 105A~105N can generate their respective virtual meeting UIs 108A~108N or terminate their respective UIs 108A~N, which can be partially generated by the UI controller 136.

[0030] In some embodiments, the data store 150 is persistent storage having the ability to store data along with data structures for tagging, organizing, and indexing the data. Data items may include audio data and / or video stream data according to embodiments described herein. The data store 150 can be hosted by one or more storage devices, such as main memory, magnetic or optical disks, tape, hard drives, flash memory, etc. In some embodiments, the data store 150 is a network-attached file server, while in other embodiments, the data store 150 is several other types of persistent storage, such as object-oriented databases, relational databases, etc., which can be hosted by the virtual meeting platform 120, or one or more different machines (e.g., servers 130) coupled to the virtual meeting platform 120 using a network 160. In some embodiments, the data store 150 stores portions of audio and video streams received from one or more client devices 102A-N, 104 for the virtual meeting platform 120. Furthermore, the data store 150 can store various types of documents, such as slide presentations, text documents, spreadsheets, or any suitable electronic documents (e.g., electronic documents containing text, tables, videos, images, graphs, slides, charts, software programming code, blueprints, lists, floor plans, blueprints, maps, etc.). These documents can be shared with and / or edited simultaneously by users of client devices 102A-N, 104. In some embodiments, as described herein, the data store stores data provided by absent users from a virtual meeting 122 (e.g., points of discussion), one or more summaries generated by the absent user manager 138, transcripts of the virtual meeting 122, or other data.In some embodiments, the data store 150 can store calendar events, calendar invitations, attachments to calendar events or calendar invitations, or other calendar-related data.

[0031] In some embodiments, network 160 includes public networks (e.g., the Internet), private networks (e.g., local area networks (LANs) or wide area networks (WANs)), wired networks (e.g., Ethernet® networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long-Term Evolution (LTE) networks), routers, hubs, switches, server computers, and / or combinations thereof.

[0032] It should be noted that in some embodiments, the functionality of the meeting platform 120 or server 130 is provided by a smaller number of machines. For example, in some embodiments, server 130 is integrated into a single machine, while in other embodiments, server 130 is integrated into multiple machines. Furthermore, in one or more embodiments, server 130 is integrated within the virtual meeting platform 120.

[0033] In general, one or more functions described in some embodiments, which are performed by the virtual meeting platform 120 or server 130, can, where appropriate, also be performed in other embodiments by client devices 102A-N, 104. Furthermore, in some embodiments, functions attributable to specific components can be realized by having different components or multiple components work together. The meeting platform 120 or server 130 can also be accessed as a service provided to another system or device via a suitable application programming interface, and is not limited to use in a website.

[0034] While embodiments of this disclosure have been described from the perspective of a virtual meeting platform 120 and users of the virtual meeting platform 122 participating in a virtual meeting 122, several embodiments can generally be applied to any type of teleconferencing, conference call, or other technical communication method between users. Embodiments of this disclosure are not limited to a virtual meeting platform that provides virtual meeting tools to users.

[0035] Figure 2A shows an exemplary AI training subsystem 200 that can be used to train one or more AI models 230A-M according to embodiments of the present disclosure. In Figure 2A, the AI ​​training subsystem 200 may include a training subsystem 210, which may include a training data engine 212, a training engine 214, a validation engine 216, a selection engine 218, or a test engine 220. The AI ​​training subsystem 200 may include one or more AI models 230A-M.

[0036] In one embodiment, AI models 230A-M include one or more of the following types of machine learning models: artificial neural networks (ANNs), decision trees, random forests, support vector machines (SVMs), clustering-based models, Bayesian networks, or other types of machine learning models. An ANN generally includes a feature representation component with a classifier or regression layer that maps features to a target output space. An ANN may include a plurality of nodes ("neurons") arranged in one or more layers, and neurons may be connected to one or more neurons via one or more edges ("synapses"). Synapses can persist signals from one neuron to another, and the values ​​of the signals can be tuned by the weights, biases, or other configurations of the neurons or synapses. Training an ANN may include tuning the weights or other features of the ANN based on the outputs generated by the ANN during training.

[0037] ANNs can include, for example, convolutional neural networks (CNNs), recurrent neural networks (RNNs), or deep neural networks. A specific type of ANN, a CNN, hosts multiple layers of convolutional filters. Pooling is performed, allowing for non-linear processing in the lower layers, and typically a multilayer perceptron is added on top to map the features extracted by the top layers to a decision (e.g., classification output). Deep networks can include ANNs with multiple hidden layers, or shallow networks with zero or a few (e.g., one or two) hidden layers. Deep learning is a type of machine learning algorithm that uses a cascade of multiple layers of non-linear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. An RNN is a type of ANN that includes memory to allow it to capture temporal dependencies. RNNs can learn input-output mappings that depend on both current and past inputs. RNNs deal with past and future measurements and make predictions based on this continuous measurement information. One type of RNN that can be used is the Long Short-Term Memory (LSTM) neural network.

[0038] ANNs can be trained in supervised (e.g., classification) and / or unsupervised (e.g., pattern analysis) ways. Some ANNs (e.g., deep neural networks) have a hierarchy of layers, where different layers learn different levels of representation corresponding to different levels of abstraction. In deep learning, each level learns to transform its input data into a slightly more abstract and synthetic representation.

[0039] In one embodiment, AI models 230A-M include generative AI models. Generative AI models may deviate from machine learning models based on their ability to generate new, original data rather than making predictions based on existing data patterns. Generative AI models may include generative adversarial networks (GANs), variation autoencoders (VAEs), large language models (LLMs), or diffusion models. In some examples, generative AI models may employ different approaches to training or learning the underlying probability distributions of the training data compared to some machine learning models. For example, a GAN may include a generator network and a discriminator network. The generator network attempts to generate synthetic data samples that are indistinguishable from real data, while the discriminator network attempts to correctly classify real samples from fake samples. Through this iterative adversarial process, the generator network can gradually improve its ability to generate increasingly realistic and diverse data.

[0040] Generative AI models also possess the ability to capture and learn the complex, high-dimensional structure of data. One purpose of generative AI models is to model the underlying data distribution, allowing the model to generate new data points with the same characteristics as the training data. Some machine learning models (e.g., models that are not generative AI models) focus on optimizing specific predictions for a task.

[0041] In some embodiments, AI models 230A–M are AI models trained on a corpus of data. For example, AI models 230A–M may be AI models that are initially pre-trained on a corpus of data to create a base model, and then fine-tuned on more data related to a particular set of tasks to create more task-specific or target models. The base model can initially be pre-trained using a corpus of data that may include public domain data, licensed content, and / or proprietary content. Such pre-training can be used by AI models 230A–M to learn a wide range of elements, including image or speech recognition, common sentence structures, common phrases, vocabulary, natural language structures, and other elements. In some embodiments, this first base model is trained using self-supervised or unsupervised training on such datasets.

[0042] In some embodiments, the second part of training, including fine-tuning, includes unsupervised training, supervised training, reinforcement training, or any other type of training. In some embodiments, this second part of training includes several supervision elements, such as learning techniques incorporating human or machine-generated feedback, being trained according to a set of guidelines, or training on a set of pre-labeled data. In non-limiting examples associated with reinforcement learning, the outputs of AI models 230A-M during training can be ranked by the user according to various factors, including accuracy, usefulness, credibility, acceptableness, or any other metrics useful in the fine-tuning part of training. Thus, AI models 230A-M can learn to prioritize these factors and any other factors relevant to the user when generating responses. Further details regarding training are provided below.

[0043] In some embodiments, AI models 230A-M include one or more pre-trained or fine-tuned models. In non-limiting examples, in some embodiments, the purpose of “fine-tuning” can be achieved with a second model, a third model, or any number of additional models. For example, the output of a pre-trained model can be input to a second AI model that has been trained in a similar manner to the “fine-tuned” portion of the training described above. In this way, two more AI models can accomplish the same work as one pre-trained and then fine-tuned model.

[0044] In one embodiment, the training subsystem 210 manages the training and testing of AI models 230A-M. The training data engine 212 can generate training data. For example, in this disclosure, the training data may include text content. The text content may include transcripts of one or more virtual meetings 122 (e.g., transcripts of one or more virtual meetings). The text content may include other types of text data, such as text documents on various subjects. The training engine 214 can use the text content training data to train generative AI models 230A-M configured to generate one or more summaries of virtual meetings 122. In other examples, the training data may include calendar events, data associated with virtual meetings, or other similar data. The training engine 214 can use the training data to train AI models 230A-M configured to identify one or more calendar events on which a user can use the proxy attendance feature.

[0045] In some embodiments, the training data may include audio data. The audio data may include recordings of a person speaking. The audio data may include one or more phonemes, word fragments, words, sentences, or other parts of speech. Each piece of audio training data may include a corresponding target output which includes a textual representation of the audio data of the audio training data. The training engine 214 can use the audio training data to train speech-to-text AI models 230A-M configured to generate transcripts of virtual meetings 122.

[0046] In an example to illustrate, the training data engine 212 can initialize the training set T to null (e.g., {}). The training data engine 212 can add training data to the training set T and determine whether the training set T is sufficient to train the AI ​​models 230A-M. In some embodiments, the training set T may be sufficient to train the AI ​​models 230A-M if the training set T includes a threshold amount of training data. In response to determining that the training set T is not sufficient for training, the training data engine 212 can identify additional data to use as training data. In response to determining that the training set T is sufficient for training, the training data engine 212 can provide the training data T to the training engine 214.

[0047] The training engine 214 can train AI models 230A-M using training data (e.g., training set T). AI models 230A-M can refer to model artifacts created by the training engine 214 using the training data, such training data including training inputs and, in some embodiments, corresponding target outputs. The training engine 214 can input the training data into AI models 230A-M, and as a result, AI models 230A-M can discover patterns in the training data and configure themselves based on those patterns.

[0048] If AI models 230A-M use supervised learning, the training engine 214 can assist AI models 230A-M in determining whether to map training inputs to target outputs. If AI models 230A-M use unsupervised learning, the training engine 214 can input training data into AI models 230A-M. AI models 230A-M can configure themselves based on the input training data, but if the training data does not include target outputs, the training engine 214 may not assist AI models 230A-M in determining whether they provided the correct output during the training process.

[0049] The validation engine 216 may be able to validate the trained AI models 230A-M using the corresponding set of features from the validation set of the training data engine 212. Based on the corresponding set of features from the validation set, the validation engine 216 can determine the accuracy of each of the trained AI models 230A-M. If the training data does not contain the target output, validating the trained AI models 230A-M may include obtaining the output from the AI ​​models 230A-M and providing that output to another entity for evaluation. The other entity may include other AI models configured to evaluate the output of the AI ​​models 230A-M being trained. The other entity may include humans. The validation engine 216 may discard trained AI models 230A-M that have an accuracy that does not meet the threshold accuracy, or otherwise fail the evaluation. In some embodiments, the selection engine 218 may select trained AI models 230A-M that have an accuracy that meets the threshold accuracy. In some embodiments, the selection engine 218 may be able to select the trained AI model 230A-M having the highest accuracy among multiple trained AI models 230A-M. In some implementations, the selection engine 218 can receive input from other AI models or humans and select the trained AI model 230A-M based on that input.

[0050] The test engine 220 may be able to test the trained AI models 230A-M using the corresponding set of features from the test set obtained from the training data engine 212. For example, the first trained AI model 230A, trained using the first set of features from the training set, can be tested using the first set of features from the test set. Based on the test set, the test engine 220 can determine which of the trained AI models 230A-M has the highest accuracy or is the best in other evaluations.

[0051] In one embodiment, the training engine 214 trains the AI ​​model 230A. The AI ​​model 230A may include an AI model that generates summaries of virtual meetings 122. The training data engine 212 can generate training data that includes transcripts of one or more virtual meetings, and the training engine 214 can use the training data to cause the AI ​​model 230A to undergo the AI ​​model training process. The AI ​​model 230A can undergo verification and testing processes using the verification engine 216 and the test engine 220.

[0052] In one embodiment, the training engine 214 trains the AI ​​model 230B. The AI ​​model 230B may include an AI model that identifies calendar events associated with virtual meetings 122 that are suitable for a user to use the proxy attendance function. The training engine 214 can use the training data to allow the AI ​​model 230B to undergo the AI ​​model training process. The AI ​​model 230B can undergo the verification and testing process using the verification engine 216 and the test engine 220.

[0053] In some embodiments, the AI ​​training subsystem 200 is part of the server 130, the virtual meeting manager 132, the absent user manager 138, the calendar application 140, or the event identification manager 142. Alternatively, the AI ​​training subsystem 200 may be part of another server, system, subsystem, or it may be a standalone system. In some embodiments, the AI ​​training subsystem 200 provides one or more trained AI models 230A-M to the absent user manager 138 or the event identification manager 142.

[0054] Figure 2B shows an exemplary AI inference subsystem 139 that can perform one or more operations using the absent user manager 138 according to an embodiment of the present disclosure. The AI ​​inference subsystem 139 may include one or more AI models 230A-M. The one or more AI models 230A-M may include one or more of the AI ​​models 230A-M trained by the AI ​​training subsystem 200.

[0055] In some implementations, the AI ​​inference subsystem 139 includes an AI input / output component 240. The AI ​​input / output component 240 can be configured to feed data, for example, transcripts of virtual meetings 122 provided by the absent user manager 138, as input to the AI ​​models 230A-M. The AI ​​input / output component 240 can be configured to take one or more outputs from one or more AI models 230A-M and provide one or more outputs to the absent user manager 138.

[0056] As shown above, in some embodiments, AI models 230A-M include an LLM. In some embodiments, the LLM includes a generative AI function. AI models 230A-M can generate new content based on provided input data (e.g., a transcript of a virtual meeting 122). Generative AI models 230A-M can be supported by a prompt subsystem (not shown) that can reside on the system architecture 100. The prompt subsystem may allow users or components of the system architecture 100 to provide input to the generative AI models 230A-M. The prompt subsystem can be configured to facilitate the acquisition of timely and relevant contextual information, which is automatically identified for efficient and accurate processing of prompts by AI models 230A-M. Using network 160 (or other networks), the prompt subsystem can communicate with one or more of the following: a virtual meeting manager 132, an absent user manager 138, a calendar application 140, or an event identification manager 142. In some embodiments, communication between the prompt subsystem and the AI ​​input / output component 240 can be facilitated by a generative model application programming interface (API). Communication between the prompt subsystem and the virtual meeting manager 132, absent user manager 138, calendar application 140, or event identification manager 142 is facilitated by the data management API. In additional or alternative embodiments, the generative model API converts prompts generated by the prompt subsystem into unstructured natural language format, and conversely, converts responses received from AI models 230A-M into any suitable format (including, for example, any structured proprietary format usable by the prompt subsystem). Similarly, the data management API can support instructions that can be used to communicate data requests to the virtual meeting manager 132, absent user manager 138, calendar application 140, or event identification manager 142, and the format of data received from such components.

[0057] The prompt subsystem may contain (or be accessible to) instructions that are stored in one or more tangible machine-readable storage media of a computing device (e.g., server 130) and are executable by one or more processing devices of the computing device. In one embodiment, the prompt subsystem can be implemented on a single machine. In some embodiments, the prompt subsystem may be a combination of client and server components. Alternatively, some parts of the prompt subsystem may run on the client computing device, while other parts of the query tool may run on the server machine.

[0058] Figure 3 is a flowchart illustrating one embodiment of Method 300 for identifying a virtual meeting for the participation of absent users, according to several embodiments of the present disclosure. A processing device having one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or memory devices communicably coupled to one or more CPUs and / or GPUs can perform Method 300 and / or one or more of the individual functions, routines, subroutines, or operations of Method 300. In a particular embodiment, a single processing thread can perform Method 300. Alternatively, two or more processing threads can perform Method 300, each thread performing one or more individual functions, routines, subroutines, or operations of the Method. In an example useful for illustration, the processing threads performing Method 300 can be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization mechanisms). Alternatively, the processing threads performing Method 300 can be asynchronous with respect to each other. The various operations of Method 300 can be performed in a different order (e.g., in reverse) compared to the order shown in Figure 3. Some operations of Method 300 can be performed simultaneously with other operations. Some operations may be optional. In some embodiments, the absent user manager 138 and / or the event identification manager 142 perform one or more of the operations of Method 300.

[0059] In block 310, the processing logic identifies a calendar event corresponding to the virtual meeting 122 by the first user's calendar application 140 and based on one or more calendar event criteria. The first user of the calendar application may include users who are invited to attend the virtual meeting 122. In some embodiments, the event identification manager 142 of the calendar application 140 identifies a calendar event based on calendar event criteria. Identifying a calendar event based on one or more calendar criteria may include determining that the calendar event satisfies one or more of the calendar event criteria. A calendar event that satisfies one or more of the calendar event criteria may be suitable for a user to attend on their behalf.

[0060] In one embodiment, the calendar event criteria include a calendar event indicating that attendance by a first user of the calendar event is optional. In some embodiments, the calendar event may include data indicating whether attendance by an invited user is mandatory or optional. Such data may be included in the calendar event invitation used to generate the calendar event within the calendar application 140. In some cases, if attendance by a first user is optional, the first user may be less likely to attend the calendar event.

[0061] In some embodiments, the calendar event criteria include calendar events that occur during periods when the first user has an absent status. The calendar application 140 may allow the first user to enter periods when the first user is unavailable (e.g., periods of absence, periods of inactivity, periods including meetings that the user has already indicated as scheduled to attend).

[0062] In one embodiment, the calendar event criteria include the fact that a first user has not sent a calendar event response to a calendar event. In some embodiments, in response to a user accessing a calendar invitation using the calendar application 140, the calendar application 140 generates a response to the calendar invitation (which may be referred to herein as a “calendar event response”). The response to a calendar event may include response data indicating whether the responding user plans to attend the calendar event corresponding to the calendar invitation. The response data may indicate that the user who received the calendar invitation plans to attend, cannot attend, is able to attend, or plans to send a substitute, as described below. In some cases, a first user who has not responded to a calendar invitation may indicate that the user is unlikely to attend the calendar event corresponding to the calendar invitation. In some embodiments, a calendar event may only satisfy the no-response criteria after a predetermined amount of time has elapsed since the user’s calendar application 140 received the calendar invitation. The predetermined amount of time may include one hour, two hours, three hours, six hours, twelve hours, twenty-four hours, or several other amounts of time. That time allows the user to respond to the calendar invitation before the event identification manager 142 identifies the corresponding calendar event as suitable for proxy attendance.

[0063] In some embodiments, the calendar event criteria include calendar events that occur during a given time interval. The given time interval may include weekly, weekly, weekday, after business hours, or several other given time intervals. Calendar events that occur during a time interval in which a first user does not normally attend calendar events may indicate that the user is unlikely to attend the calendar event.

[0064] In one embodiment, a calendar event includes a series of calendar events. A series of calendar events may include multiple calendar events that repeat regularly. For example, a series of calendar events may include calendar events that occur daily, on all weekdays, once a week, once a month, or at other time intervals. Calendar events within a series of calendar events may occur at the same time, or calendar events may occur at different times. Calendar events within a series of calendar events may occur on the same day in each time interval (for example, every Thursday for a series of calendar events that repeat weekly, or the 15th of each month for a series of calendar events that repeat monthly), or calendar events may occur on different days.

[0065] In some embodiments, the calendar event criteria include the first user rejecting a predetermined number of calendar events from a set of calendar events. The predetermined number of calendar events in the set may include 1, 2, 3, 4, or several other numbers. Because the first user has rejected a predetermined number of calendar events in the set, the likelihood of the first user attending a set of future calendar events may be reduced.

[0066] In one or more embodiments, the calendar application 140 may include a calendar invitation response UI element. The calendar invitation response UI element may include one or more UI elements that provide information about a calendar event or set of calendar events corresponding to a calendar invitation. The calendar invitation response UI element may include one or more UI elements that allow a first user to select an attendance option associated with the corresponding calendar event. The attendance option may include a first option indicating that the first user accepts attending the virtual meeting 122 of the calendar event. The attendance option may include a second option indicating that the first user refuses to attend the virtual meeting 122. The attendance option may include a third option requesting attendance at the virtual meeting 122 by a substitute. Further information regarding the calendar invitation response UI element is described below in relation to Figure 5.

[0067] In some embodiments, the calendar event criteria include a first user activating UI elements for a predetermined number of calendar events in a set of calendar events. The UI elements may include elements that allow the first user to indicate that they are scheduled to attend virtual meeting 122 on behalf of another user. In some cases, if the first user has requested proxy attendance at virtual meeting 122 for a previous calendar event in the set of calendar events, this may indicate that the first user is likely to continue doing so.

[0068] In some embodiments, the event identification manager 142 identifies a calendar event based on calendar event criteria in response to receiving a calendar invitation for a calendar event in the calendar application 140. For example, as part of the calendar invitation reception process of the calendar application 140, the event identification manager 142 may identify a calendar event as satisfying one or more of the calendar event criteria. In one embodiment, the event identification manager 142 identifies a calendar event based on calendar event criteria in response to a predetermined amount of time having elapsed since the event identification manager 142 last identified a calendar event. This time may include 1 minute, 5 minutes, 10 minutes, 15 minutes, 30 minutes, 1 hour, 2 hours, 6 hours, or several other periods of time. In some embodiments, the event identification manager 142 identifies a calendar event based on calendar event criteria once a day (for example, every day at 8 a.m. (local time)). In one or more embodiments, the event identification manager 142 determines whether a calendar event satisfies one or more of the calendar event criteria only if the event identification manager 142 had not previously determined whether the calendar event satisfies one or more of the calendar event criteria (for example, the event identification manager 142 may not recheck a calendar event for which the event identification manager 142 has already determined whether or not one or more of the calendar event criteria are met).

[0069] In block 320, the processing logic presents the calendar application UI to the first user. The calendar application UI may include UI elements that request the first user to provide virtual information. The virtual meeting information may include a summary of the discussion in virtual meeting 122, a summary of part of the discussion, one or more action items in virtual meeting 122, or other information generated or discussed during virtual meeting 122. The first user may not be able to attend virtual meeting 122. Because the first user may not be able to attend virtual meeting 122, the first user may want to obtain the virtual meeting information. The processing logic can cause the calendar application UI to be presented on the UI 108A~N of the first user's client devices 102A~N and 104.

[0070] In one embodiment, the UI elements of the calendar application UI may include a UI element containing one or more lists of calendar event criteria identified by the event identification manager 142 as satisfying one or more of the calendar event criteria. The UI elements may further include a UI element (e.g., a button) that the first user can interact with to indicate to the organizer(s) of one or more identified calendar events that the first user is scheduled to attend one or more virtual meetings 122 corresponding to one or more identified calendar events on behalf of the first user. A UI element requesting virtual meeting information to be provided to the first user may include a UI element indicating that the first user is scheduled to attend the virtual meeting 122 on behalf of the first user.

[0071] In some embodiments, the UI elements of a calendar application may include a calendar invitation response UI element. As described above, the calendar invitation response UI element may include a UI element (e.g., a button) that the first user can interact with to indicate to the organizer(s) of one or more identified calendar events whether the first user will attend, will not attend, or will have a proxy attend the corresponding virtual meeting(s) 122. The UI element requesting virtual meeting information to be provided to the first user may include a UI element indicating that the first user will have a proxy attend the virtual meeting(s) 122. Furthermore, information regarding the UI elements relating to block 320 is described below in relation to Figures 4 and 5.

[0072] In block 330, in response to user activation of a UI element (for example, a UI element requests virtual meeting information for a first user), the processing logic provides the first user's virtual meeting request to the virtual meeting application. The virtual meeting application may include a virtual meeting manager 132, an absent user manager 138, or an application running on the virtual meeting platform 120. The virtual meeting request may include data indicating to the organizer of virtual meeting 122 that the first user is scheduled to attend virtual meeting 122 on behalf of another user.

[0073] In one embodiment, in response to user activation of a UI element, processing logic causes the calendar application UI to present a second UI element. The second UI element can request first data from the first user. The first data may indicate one or more discussion topics. A virtual meeting information request may include the first data. In some embodiments, the first data includes text content. The text content may include text data (e.g., one or more text strings) or a reference to text data (e.g., a URL (uniform resource locator) linking to text data). The first data may include audio content. The audio content may include an audio file or a reference to audio data (e.g., an audio file stored on the internet and accessible using a URL). The first data may include video content. The video content may include a video file or a reference to video data.

[0074] Figure 4 shows an exemplary UI 400 of a calendar application 140 according to several embodiments of the present disclosure. The UI 400 can be displayed on the displays 107A-N of client devices 102A-N, 104. The UI 400 may include UI elements corresponding to one or more days of the week and one or more time periods of the day. For example, as seen in Figure 4, the UI 400 includes five columns of UI elements corresponding to the days of the week: Monday, Tuesday, Wednesday, Thursday, and Friday. Also, as seen in Figure 4, the UI 400 may include time along the left side of the UI 400. Where the days of the week column and time intersect, the UI 400 may include UI elements corresponding to calendar events scheduled for that day and time. For example, as seen in Figure 4, the UI 400 includes a calendar event 402 scheduled for Monday from 8:30 a.m. to 10:00 a.m. and titled “Weekly Planning Meeting”.

[0075] In one embodiment, as described above in relation to block 310 of method 300, the event identification manager 142 identifies one or more calendar events associated with one or more virtual meetings 122. The one or more virtual meetings 122 may be suitable for a user to attend on their behalf. The UI 400 may include one or more UI elements that show the identified calendar events to the user. For example, as seen in Figure 4, the UI 400 includes a UI element 404 that alerts a first user that the event identification manager 142 has identified one or more calendar events, and the UI element 404 lists the identified calendar events. Also as seen in Figure 4, the UI has UI elements 406A-C marked (here, a thick outline is used) that correspond to the identified calendar events on the calendar. UI elements 406A-C may include any type of visual indication to show that they have been identified by the event identification manager 142, and the visual indication may include a thick outline, being marked in a predetermined color, including a predetermined background, including a predetermined symbol, or using some other visual indication.

[0076] In one or more embodiments, UI element 404 may include UI element 408 (e.g., a button) that a first user can interact with to respond to one or more identified calendar events. In response to user activation of UI element 408, UI 400 may provide a virtual meeting information request to the virtual meeting application. As described above, the virtual meeting information request may include data indicating the first user's plan to attend one or more identified calendar events corresponding to UI elements 406A-C on behalf of another user. As shown in Figure 5, the first user can interact with UI elements 406A-C corresponding to one or more identified calendar events to provide first data (e.g., points of discussion or other data) to be used during the corresponding virtual meeting 122. In some embodiments, in response to the first user indicating that they are attending a calendar event on behalf of another user, UI elements 406A-C corresponding to the calendar events may not be displayed on UI 400 (e.g., the calendar may only display calendar events that the user plans to attend and calendar events that the user has not yet responded to).

[0077] Figure 5 shows another example of the UI 400 of the calendar application 140 according to some embodiments of the present disclosure. In some embodiments, in response to a user interacting with the UI elements of a calendar event 406A (for example, by clicking on the calendar event 406A with a mouse or tapping on the calendar event 406A on a touchscreen), the UI 400 displays UI elements of a detail view 502 corresponding to the calendar event 406A. The UI elements of the detail view 502 can display further information associated with the calendar event 406A. For example, as seen in Figure 5, the detail view 502 includes the title of the calendar event 406A, the date and time of the calendar event 406A, and the location of the calendar event. The location may include a virtual meeting 122. The first user can access the virtual meeting 122 by interacting with a specific part of the detail view 502 (for example, a URL included in the detail view 502).

[0078] In one or more embodiments, the detail view 502 includes one or more UI elements that allow a first user to indicate whether they are scheduled to attend a calendar event 402. For example, as seen in Figure 5, the detail view 502 includes a button 504 labeled "Yes" to indicate that the user is scheduled to attend, "No" to indicate that the user is not scheduled to attend, and "Possible to attend" to indicate that the user may attend. Button 504 may include a "Substitute Attend" button, which indicates that the first user is scheduled to send a substitute to the corresponding virtual meeting 122. In response to the user interacting with the "Substitute Attend" button, the detail view 502 may display one or more UI elements in which the user can provide first data, as seen in Figure 5. For example, the detail view 502 may display a text box 506 in which the user can enter text content, or a reference to video content, audio content, or other type of content (e.g., a URL). UI element 506 can display a button 508 that opens a file selector, allowing the user to select an audio file, video file, or other type of file. In response to the user finishing entering the first data (for example, by interacting with the “Submit” button as seen in Figure 5), the first data can be provided to the absent user manager 138.

[0079] In some embodiments, the first user can modify the first data after the absent user manager 138 has retrieved it. For example, the user can interact with the detail view 502 again and use the text box 506 or file selector button 508 to enter different first data, change the first data initially provided, or remove at least a portion of the first data initially provided. In some embodiments, the first user can change the response from “Attending on behalf of” to “Yes,” indicating that the first user is scheduled to attend the corresponding virtual meeting 122.

[0080] In one embodiment, the calendar application 140 can cause a second calendar application UI to be presented to a second user on the second user's client devices 102A-N, 104. The second user may be an organizer of calendar events (e.g., calendar event 406A). The second calendar application UI may be similar to UI 400, but the second calendar application UI can display calendar events that have been organized or accepted by the second user.

[0081] A second user can interact with a calendar event 406A to display a detailed view 502 of the calendar event in a second calendar application UI. The detailed view 502 may include a list of users invited to the calendar event. The list of users may indicate, for each user, the status of their attendance at the scheduled calendar event. The status may include responses such as "Yes," "No," "Possible to attend," or "Attend using proxy attendance." A response such as "Attend using proxy attendance" may indicate that each user is scheduled to attend a virtual meeting 122 that will be handled by a proxy, providing first data (e.g., points of discussion) to be discussed during the virtual meeting 122.

[0082] In one or more embodiments, a second user can configure a calendar event so as not to allow a substitute to attend the corresponding virtual meeting 122. The second user can configure the calendar event to not allow substitute attendance when first generating the calendar event, or when modifying the calendar event after it has been first generated.

[0083] In one embodiment, the virtual meeting manager 132 causes the virtual meeting UI 108A-N to be displayed during a virtual meeting 122 between one or more participants. The virtual meeting UI 108A-N may include UI elements associated with first data provided by a first user who is not present during the virtual meeting 122. The UI elements associated with the first data can be displayed in response to the absent user manager 138 providing the first data to the UI controller 136, thereby allowing the UI elements to be displayed in the UI 108A-N.

[0084] In one embodiment, the UI element includes a sidebar positioned next to the virtual meeting UI108A-N. The sidebar may include areas of UI108A-N on the left, right, top, and bottom, or other areas of UI108A-N that display information. In some embodiments, the UI element includes notifications presented to the virtual meeting UI108A-N. The notifications may include pop-up dialog boxes within UI108A-N, or several other notification elements within UI108A-N. In some other ways, UI108A-N may present content based on first data of a first user.

[0085] UI elements may contain content based on first data provided by the user. For example, if the first data contains text content, the UI element may contain one or more text strings, a URL, or content from a location referenced by the URL (e.g., a web page or embedded content from a web page). If the first data contains audio content, the UI element may contain a media player that allows the user to play the audio content. If the first data contains video content, the UI element may contain a media player that allows the user to view the video content.

[0086] In some embodiments, the virtual meeting manager 132 causes the presentation of a UI element associated with first data in response to a predetermined amount of time having elapsed since the start of the virtual meeting 122. This allows participants in the virtual meeting 122 to adjust the time they have to join the meeting and perform other initial tasks before they begin discussing one or more topics in the virtual meeting 122. The configuration settings for the virtual meeting 122 may include a predetermined time. Calendar invitation responses may include a predetermined time (so that users who are sending a proxy can set a predetermined time). In one embodiment, the processing logic causes the presentation of a UI element associated with first data in response to a predetermined amount of time elapsed since a different virtual meeting 122 event occurred (e.g., a predetermined number of invitees to the virtual meeting 122 join the virtual meeting, a predetermined portion of the invitees to the virtual meeting 122 join the virtual meeting 122, the host of the virtual meeting 122 interacts with UI elements 108A-N indicating the start of discussion on a topic in the virtual meeting 122, or some other event).

[0087] Figure 6 shows exemplary virtual meeting UIs 108A-N in several embodiments. Each virtual meeting UI 108A-N may include one or more areas 602A-C corresponding to visual items of the virtual meeting 122, such as video streams provided by the client devices 102A-N, 104 of the virtual meeting participants 122. The virtual meeting UI 108A-N may include a toolbar 604, which includes one or more UI elements configured to perform the operations of the virtual meeting. For example, as seen in Figure 6, the toolbar 604 includes an audio control button 606 used to mute and unmute a participant's audio stream, a camera control button 608 used to mute and unmute a participant's video stream, and a screen sharing button 610 used to share a participant's client device 102A-N, 104 with another participant in the virtual meeting 122. In some embodiments, the toolbar 604 may include one or more buttons 612 that cause the AI ​​model 230 to perform one or more functions of virtual meetings 122 (for example, generating a summary of at least some of the virtual meetings 122, generating an action item, or implementing another virtual meeting function as described herein).

[0088] In one embodiment, the virtual meeting UI 108A-N includes a sidebar 614 UI element. The UI element of the sidebar 614 can display information based on first data provided by a first user attending on behalf of another user. For example, as seen in Figure 6, the sidebar 614 includes text data containing discussion points provided by the first user. As described above, the sidebar 614 may include text content, video content, audio content, or references to other types of content. The sidebar 514 may include a media player within the sidebar 614 used to play the audio or video content of the first data. In some embodiments, the toolbar 604 includes buttons to show or hide the sidebar 614 in the UI 108A-N.

[0089] In one or more embodiments, the absent user manager 138 generates a summary based on one or more virtual meetings 122 attended by a first user through a proxy. The summary can cover presentations of at least some of the first data during the virtual meeting 122 (e.g., points of discussion provided by the first user). During the virtual meeting 122, participants in the virtual meeting 122 can discuss multiple topics or points of discussion. The first user may want to know what was discussed regarding the first data during the virtual meeting 122, and therefore the summary can cover the discussion of the first data during the virtual meeting 122.

[0090] In one or more embodiments, the virtual meeting manager 132, or several other components of the server 130, can generate transcripts of virtual meetings 122. Generating transcripts of virtual meetings 122 may involve using a speech-to-text AI model 230. The speech-to-text AI model 230 can receive audio data from one or more audio streams corresponding to different participants in the virtual meeting 122 as input, and can generate textual representations of the audio data to generate transcripts of the virtual meetings 122. The virtual meeting manager 132 can store transcripts on, for example, the server 130 or the data store 150. The speech-to-text AI model 230 can generate transcripts in real time during the virtual meetings 122. The absent user manager 138 can access the transcripts and use the transcripts of the virtual meetings 122 to generate summaries of one or more virtual meetings 122.

[0091] In some embodiments, generating a summary involves generating a summary using a generating AI model 230. The generating AI model 230 may be part of the AI ​​inference subsystem 139 of the absent user manager 138. The generating AI model 230 may include an LLM or other type of generating AI model 230, as described above in relation to Figures 2A and 2B. Generating a summary using the generating AI model 230 may involve a generating AI prompt being input to the generating AI model 230. The generating AI prompt may be at least in part based on one or more transcripts of one or more virtual meetings attended by a first user on behalf of another. The generating AI prompt may include a command to generate a summary based on one or more transcripts.

[0092] For example, the absent user manager 138 can generate a generating AI prompt that includes the command "Generate a summary of the discussion transcript for the following meeting," and the generating AI prompt may further include the transcript of a virtual meeting(s) 122 attended by a first user on their behalf. In some embodiments, the absent user manager 138 can generate a generating AI prompt for the transcript of each meeting to be summarized, and after the individual summaries have been generated, the multiple summaries can be combined into a single summary.

[0093] In one embodiment, the summary includes a text summary. The text summary may include one or more text strings. The summary may include data in other formats (e.g., an audio summary including audio data summarizing the virtual meeting 122). In one embodiment, the summary includes video content. The video content may include recordings of the virtual meeting UI 108A~N and associated audio data during a portion of the virtual meeting 122. The portion of the virtual meeting 122 may include a presentation of at least a portion of the first data during the virtual meeting 122.

[0094] In some embodiments, the absent user manager 138 makes the summary accessible to the first user's client devices 102A-N, 104. In some embodiments, the absent user manager 138 stores the summary in a server 130, a data store 150, cloud storage, a content management platform, or some other location. Making the summary accessible to the first user's client devices 102A-N, 104 may include providing a reference to the stored summary on the server 130, in the data store 150, in cloud storage, on the content management platform, etc. The summary accessible to the first user's client devices 102A-N, 104 may include the absent user manager 138 providing the first user's client devices 102A-N, 104 with a reference to the summary (e.g., a URL associated with the stored summary). In one embodiment, the absent user manager 138 provides a summary to client devices 102A-N, 104 (for example, by providing a file containing the summary to client devices 102A-N, 104 via the network 160).

[0095] In some embodiments, making the summary accessible by the first user's client devices 102A-N, 104 includes making the summary accessible from the calendar application 140. Figure 7 shows an exemplary UI 700 of the calendar application 140. The UI 700 of the calendar application 140 may be similar to the calendar application UI 400 in Figure 4, and may also include UI elements corresponding to one or more days of the week, one or more time periods during the day, or one or more calendar events. In some embodiments, the UI 700 includes UI element 702. UI element 702 may include a list of calendar events attended by the first user on behalf of another user. One or more calendar events in UI element 702 may include UI elements 704A-C (e.g., buttons) that allow the first user to access summaries associated with the virtual meetings 122 corresponding to each calendar event. UI element 702 may contain other UI elements 706, which may enable a first user to access weekly (or other time interval) summaries of virtual meetings 122 corresponding to calendar events listed in UI element 702. In some embodiments, a first user may interact with calendar events presented in UI 700, which may present other UI elements corresponding to the calendar events. The UI elements may include UI elements that enable a first user to access summaries associated with virtual meetings 122 corresponding to the calendar events.

[0096] Figure 8 is a block diagram illustrating an exemplary computer system according to an embodiment of the present disclosure. The computer system 800 may include the client devices 102A-N, 104, the virtual meeting platform 120, or the server 130 in Figure 1. The machine may operate at the capacity of a server or endpoint machine in an endpoint server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a television, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a server, a network router, a switch or bridge, or any machine capable of executing (sequentially or otherwise) a set of instructions that specify the actions that the machine takes. Furthermore, although only a single machine is shown, the term “machine” shall also be interpreted as including any collection of machines that individually or collectively execute one or more sets of instructions to perform any one or more methodologies of the methodologies described herein.

[0097] The exemplary computer system 800 includes a processing device (processor) 802, main memory 804 (e.g., read-only memory (ROM), flash memory, or dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM), double data rate (DDR SDRAM), or DRAM (RDRAM)), static memory 806 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage device 816, which communicate with each other via a bus 830.

[0098] The processing device 802 represents one or more general-purpose processing devices, such as a microprocessor or a central processing unit. More specifically, the processing device 802 may be a multiple instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing another instruction set, or a processor implementing a combination of instruction sets. The processing device 802 may also be one or more dedicated processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor. The processing device 802 is configured to execute processing logic 822 to perform the operations described herein (for example, the operations of the absent user manager 138 or the event identification manager 142).

[0099] The computer system 800 may further include a network interface device 808. The computer system 800 may also include a video display unit 810 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an input device 812 (e.g., a keyboard, an alphanumeric keyboard, a motion-sensing input device, a touchscreen), a cursor control device 814 (e.g., a mouse), and a signal generating device 818 (e.g., a speaker).

[0100] The data storage device 816 may include a non-temporary machine-readable storage medium 824 (sometimes referred to as the “computer-readable storage medium”) which stores one or more instruction sets 826 (e.g., instructions to perform one or more actions of the absent user manager 138 or the event identification manager 142) that embody any one or more of the methodologies or functions described herein. The instructions may also reside, fully or at least partially, in the main memory 804 and / or the processor 802 while being executed by the computer system 800, and the main memory 804 and the processor 802 also constitute the machine-readable storage medium. The instructions may further be transmitted or received through the network 160 via the network interface device 808.

[0101] In one embodiment, instruction 826 includes instructions for determining visual items to be presented in the user interface of a virtual meeting. Although the computer-readable storage medium 824 (machine-readable storage medium) is shown in exemplary embodiments as a single medium, the terms “computer-readable medium” and “machine-readable medium” should be interpreted to include a single or multiple mediums that store one or more sets of instructions (e.g., a centralized or distributed database, and / or associated caches and servers). The terms “computer-readable storage medium” and “machine-readable storage medium” should also be interpreted to include any medium capable of storing, encoding, or carrying a set of instructions executed by a machine, causing the machine to perform any one or more of the methodologies of this disclosure. Accordingly, the terms “computer-readable storage medium” and “machine-readable storage medium” should be interpreted to include, but are not limited to, solid-state memory, optical media, and magnetic media.

[0102] Throughout this specification, any reference to “one embodiment” or “a particular embodiment” means that a specific function, structure, or characteristic described in relation to an embodiment is included in at least one embodiment. Therefore, while the phrases “in one embodiment” or “a particular embodiment” appear in various places throughout this specification, they may, in some contexts, refer to the same embodiment, though not necessarily. Furthermore, specific features, structures, or characteristics can be combined in any preferred manner in one or more embodiments.

[0103] To the extent that the terms “includes,” “including,” “possess,” “contain,” their variations, and other similar terms are used in either embodiments or claims to describe the invention, these terms are intended to be inclusive in the same manner as the term “equip,” without excluding any additional or other elements.

[0104] As used in this application, terms such as “component,” “module,” and “system” are intended to generally refer to computer-related entities, either hardware (e.g., circuits) or software, a combination of hardware and software, or entities relating to an operable machine with one or more specific functionalities. For example, a component may be, but is not limited to, a process running on a processor (e.g., a digital signal processor), a processor, an object, an executable file, an execution thread, a program, and / or a computer. As an example, an application running on a controller, and both the application and the controller, may be components. One or more components may reside within an execution process and / or thread, and components may be localized on one computer and / or distributed across two or more computers. Furthermore, a “device” may be provided in the form of specially designed hardware, general-purpose hardware specialized by the execution of software that enables the hardware to perform a specific function (e.g., generating points of interest and / or descriptors), software on a computer-readable medium, or a combination thereof.

[0105] The aforementioned systems, circuits, modules, etc., are described in relation to interactions between multiple components and / or blocks. Such systems, circuits, components, blocks, etc., may include their components or designated subcomponents, parts of designated components or subcomponents, and / or additional components, and it can be recognized that they follow the various permutations and combinations described above. Furthermore, subcomponents may be implemented not as being contained within a parent component (hierarchical), but as components communicatively coupled to other components. It should also be noted that one or more components may be combined into a single component providing an integrated function, or divided into several individual subcomponents, and communicatively coupled to such subcomponents to provide integrated functionality by providing one or more intermediate layers, such as a management layer. Additionally, any component described herein may interact with one or more other components known to those skilled in the art, but not specifically described herein.

[0106] Furthermore, the terms “example” or “exemplary” are used herein in the sense of serving as an example, case, or illustration. No aspect or design described herein as “exemplary” is necessarily construed to be preferable or advantageous to another aspect or design. Rather, the use of the terms “example” or “exemplary” is intended to concretely present a concept. As used in this application, the term “or” is intended to mean inclusive “or” rather than exclusive “or.” That is, unless otherwise specified or it is clear from the context, “X adopts A or B” is intended to mean either of the natural inclusive substitutions. That is, if X adopts A, if X adopts B, or if X adopts both A and B, “X adopts A or B” satisfies the condition of any of the aforementioned examples. Furthermore, the articles “a” and “an” as used in this application should generally be construed to mean “one or plural” unless otherwise specified or it is clear from the context that they refer to a singular form.

[0107] Finally, the embodiments described herein include the collection of data describing the user and / or the user's activities. In one embodiment, such data is collected only if the user consents to this data collection. In some embodiments, the user is explicitly prompted to allow data collection. Furthermore, the user can opt in or opt out of participating in such data collection activities. In one embodiment, the collected data is anonymized before any analysis is performed to obtain any statistical patterns, so that information identifying the user cannot be determined from the collected data.

Claims

1. It is a method, The method includes identifying a calendar event corresponding to a virtual meeting using a first user's calendar application and based on one or more calendar event criteria, wherein the first user of the calendar application is invited to attend the virtual meeting, and the method further includes: The method includes presenting a calendar application user interface (UI) to the first user, wherein the calendar application UI includes a UI element that requests to provide virtual meeting information to the first user who is unable to attend the virtual meeting, and the method further includes In response to user activation of the UI element, the first user's virtual meeting information request is provided to the virtual meeting application. Methods that include...

2. The method according to claim 1, wherein the one or more calendar event criteria include the calendar event indicating that attendance by the first user of the calendar event is optional.

3. The method according to claim 1, wherein the one or more calendar event criteria include the calendar event that occurs during a period in which the first user has an absent status.

4. The method according to claim 1, wherein the one or more calendar event criteria include that the first user has not sent a calendar event response to the calendar event.

5. The method according to claim 1, wherein the one or more calendar event criteria include the calendar events occurring within a predetermined time interval.

6. The method according to claim 1, wherein the calendar event includes a series of calendar events, and the series of calendar events includes a plurality of calendar events that repeat periodically.

7. The method according to claim 6, wherein the one or more calendar event criteria include the first user rejecting a predetermined number of calendar events from the series of calendar events.

8. The method according to claim 6, wherein the one or more calendar event criteria include the first user activating the UI element for a predetermined number of calendar events from the series of calendar events.

9. The UI element includes multiple attendance options, and the multiple attendance options are A first option indicating that the first user accepts to attend the virtual meeting, A second option indicating that the first user refuses to attend the virtual meeting, The method according to claim 1, further comprising a third option of requesting a proxy to attend the virtual meeting, wherein requesting a proxy to attend represents a request for information about the virtual meeting.

10. It is a system, Memory and The system comprises a processing device coupled to the memory, wherein the processing device is configured to perform an operation, and the operation is, The process includes identifying a calendar event corresponding to a virtual meeting using a first user's calendar application and based on one or more calendar event criteria, wherein the first user of the calendar application is invited to attend the virtual meeting, and the process further includes: The operation includes presenting a first calendar application user interface (UI) to the first user, wherein the first calendar application UI includes a UI element that requests to provide virtual meeting information to the first user who is unable to attend the virtual meeting, and the operation further includes In response to user activation of the UI element, the first user's virtual meeting information request is provided to the virtual meeting application. A system that includes this.

11. The aforementioned operation further, The operation further includes, in response to the user activation of the first UI element, causing the first calendar application UI to present a second UI element requesting first data of the first user, wherein the first data includes text content indicating one or more discussion topics, and the operation further includes, The first data is included as part of the virtual meeting information request. The system according to claim 10, including the following:

12. The aforementioned operation further, This includes ensuring that the generated AI prompt is input to the generated AI model, The generated AI prompt is at least in part based on one or more transcripts of one or more virtual meetings attended by the first user on behalf of another user, and the one or more virtual meetings attended by the first user on behalf of another user include the virtual meeting corresponding to the calendar event. The generated AI prompt includes a command for generating a summary based on the one or more transcripts of the one or more virtual meetings, and the operation further includes: To make the summary accessible by the first user's client device, The system according to claim 10, including the following:

13. The operation further includes presenting a second calendar application UI to a second user, the second user including the organizer of the calendar events, The system according to claim 10, wherein the second calendar application UI includes a second UI element indicating that the first user is scheduled to attend the virtual meeting on behalf of another user.

14. The one or more calendar event criteria are: The calendar event indicates that attendance by the first user of the calendar event is optional, or The calendar event occurs during a period when the first user has an absent status. The system according to claim 10, comprising at least one of the following.

15. The one or more calendar event criteria are: The first user has not sent a calendar event response to the calendar event, or The aforementioned calendar event occurs within a predetermined time interval. The system according to claim 10, comprising at least one of the following.

16. The system according to claim 10, wherein the calendar event includes a series of calendar events, and the series of calendar events includes a plurality of calendar events that repeat periodically.

17. A non-temporary computer-readable storage medium containing instructions, wherein, when the instructions are executed by a processing device, the processing device receives, The calendar application of the first user identifies a calendar event corresponding to a virtual meeting based on one or more calendar event criteria, the first user of the calendar application is invited to attend the virtual meeting, and the instruction further instructs the processing device, The calendar application user interface (UI) is presented to the first user, and the calendar application UI includes a UI element that requests the first user, who is unable to attend the virtual meeting, to provide virtual meeting information, and the instruction further instructs the processing device to: A computer-readable storage medium that, in response to user activation of the UI element, causes the virtual meeting application to provide the first user's virtual meeting information request.

18. The one or more calendar event criteria are: The calendar event indicates that the attendance of the first user is optional, or The calendar event occurs during a period when the first user has an absent status. A computer-readable storage medium according to claim 17, comprising at least one of the following.

19. The one or more calendar event criteria are: The first user has not sent a calendar event response to the calendar event, or The aforementioned calendar event occurs within a predetermined time interval. A computer-readable storage medium according to claim 17, comprising at least one of the following.

20. The aforementioned operation further, This includes ensuring that the generated AI prompt is input to the generated AI model, The generated AI prompt is at least in part based on one or more transcripts of one or more virtual meetings attended by the first user on behalf of another user, and the one or more virtual meetings attended by the first user on behalf of another user include the virtual meeting corresponding to the calendar event. The generated AI prompt includes a command for generating a summary based on the one or more transcripts of the one or more virtual meetings, and the operation further includes: To make the summary accessible by the first user's client device, A computer-readable storage medium according to claim 17, including the following:

Images