User interface for communication sessions

Abstract

This paper discloses a method for improving the user interface for communication sessions. [Solution] The method involves obtaining transcript data containing one or more words from the speech transcription within audio data, the transcript data being generated from the audio data by automatic speech recognition technology, the transcript data and criteria being provided to a large language model, the large language model analyzing the transcript data based on the criteria, selecting words from the transcript data, the large language model generating definitions for the selected words, the selected words and their definitions being obtained, the audio data being broadcast by the device, and the selected words and their definitions being presented on the device's display along with the broadcast of the audio data.

Description

Technical Field

[0001] The present disclosure generally relates to a user interface for a communication session.

Background Art

[0002] Discussions often cover a variety of topics, and participants have different backgrounds and experiences regarding the topics being discussed. Due to the diversity of knowledge each participant has, participants may encounter words that are unfamiliar to them during the discussion. Unfamiliar words can potentially create a communication barrier between those discussion participants who understand the word and those who do not. Furthermore, searching for an explanation of the word during the conversation can be distracting and may cause a lack of attention to the discussion.

[0003] The subject matter claimed in the present disclosure is not limited to embodiments that solve any disadvantages or that operate only in the environments as described above. Rather, this background art is provided only to explain a technical area in which some embodiments described in the present disclosure may be implemented.

Summary of the Invention

[0004] According to an aspect of at least one embodiment, one or more operations may include the step of acquiring transcript data. The transcript data may be generated from audio data containing speech by an automatic speech recognition technique. The transcript data may include one or more words from the transcription of speech in the audio data. The transcript data and a first set of criteria may be provided to a large language model. The large language model may be configured to analyze the transcript data based on the first set of criteria and select words from the transcript data. Definitions of the selected words may be generated by the large language model. The selected words and their definitions may be acquired. The audio data may be broadcast by the device. The selected words, their definitions, and the broadcasted audio data may be presented on the device's display.

[0005] The objectives and advantages of the embodiments are realized and achieved at least by the elements, features, and combinations particularly indicated in the claims. It should be understood that both the above general description and the following detailed description are examples and not limitations of the claimed invention. [Brief explanation of the drawing]

[0006] Exemplary embodiments are described and illustrated with further specifics and details through the following attached drawings.

[0007] [Figure 1] An exemplary environment 100 is shown, including a user interface for a communication session.

[0008] [Figure 2] This demonstrates the operational workflow for presenting the user interface.

[0009] [Figure 3] An example user device for presenting a user interface is shown.

[0010] [Figure 4] This is a flowchart illustrating the method of presenting the user interface.

[0011] [Figure 5] This is a flowchart illustrating the method of presenting the user interface.

[0012] [Figure 6] This disclosure illustrates an exemplary computing system according to one or more embodiments. [Modes for carrying out the invention]

[0013] In work, academic, and other environments, individuals and organizations are often separated according to their technical field, area of ​​study, role, or the function they perform. Each individual and organization may have varying levels of expertise, experience, or familiarity with the language used in other fields or organizations, or in their own field. As a result, the language used by individuals and / or organizations may be related to a particular subject and may not be readily understood by other users who are not familiar with that subject. For example, individuals and / or organizations may use advanced technology, jargon, or simply words unfamiliar to other users.

[0014] In some cases, a lack of understanding of words and terminology used by other individuals and / or organizations can create communication barriers between individuals using devices. For example, an individual might use a user interface to participate in a cross-organizational virtual meeting on a teleconferencing platform such as Skype, Zoom, or Microsoft Teams. In such a meeting, an individual may not understand the words used by another individual, and using those words could create a communication gap and prevent effective communication with the other individual.

[0015] This confusion cannot be resolved without diverting individuals from the discussion, as the user interface does not contextualize or define the words. Users must leave the user interface to find definitions or additional context for the words. Furthermore, the user interface does not provide a mechanism to provide additional context for words used beyond those spoken by other participants in the conversation. Thus, users of the device are left to their own means of self-help, attempting to understand the words as the discussion progresses.

[0016] In some situations, individuals may try to improve their understanding of a discussion by minimizing the user interface, opening a web browser, and searching for words. However, minimizing the user interface distracts the individual's attention from the ongoing discussion, and as a result, they may miss the context in which the word is used or other topics being discussed in the meeting while they are focused on searching for a word in their web browser. Furthermore, as the number of unfamiliar words increases, dynamically searching for terms can become more difficult. Therefore, in a setting where an individual does not understand multiple words being used, they may find it difficult to keep up with word searches as the words are being used in the discussion, and they may become increasingly distracted from the discussion.

[0017] In other situations, individuals may attempt to understand a word by determining the context in which it is used in a discussion. While leveraging context may be less distracting than searching for the word during a discussion, individuals may fail to understand a word based on context, especially if they are unfamiliar with other words being used in the discussion.

[0018] According to one or more embodiments of the present disclosure, words spoken in a communication session that an individual may not understand can be selected, defined, and presented via a user interface on the individual's device when the individual encounters the word during the communication session. By selecting, defining, and presenting the word within the user interface when the individual encounters it, the individual can remain focused on the user interface and the discussion while enhancing their understanding of the subject being discussed. Alternatively or additionally, selecting, defining, and presenting words that an individual may not understand when they encounter them within the user interface can improve the user interface and reduce the amount of distraction the individual would otherwise have experienced if they were dynamically searching for terms.

[0019] In some embodiments, transcript data can be generated from audio data of a communication session, including speech, in order to select, define, and present words on the device during the communication session. The transcript data may be obtained via automated speech recognition (ASR) technology and may include a transcription of the speech within the audio data.

[0020] In some embodiments, transcript data and a first set of criteria can be provided to an artificial intelligence system such as a large-scale language model. The large-scale language model can be configured to analyze the transcript data based on the first set of criteria and select from the transcript data words that are highly technical, jargon, acronyms, or other words that are complex and / or not easily understood by most people. The large-scale language model can also generate definitions of the selected words. Since the large-scale language model includes the context of the communication session, the definitions may be specific to the use of the selected words in the communication session.

[0021] In some embodiments, the audio data of a communication session may be broadcast by the device to the users participating in the communication. In these and other embodiments, the selected words and related definitions may be presented to the user via a user interface on the device's display, substantially in real time with the broadcast of the audio data.

[0022] In some embodiments, the large language model may be provided with another set of criteria regarding the user. In these and other embodiments, the large language model can select words having definitions for presentation based on the other set of criteria. In these and other embodiments, the other set of criteria can include the attributes of the user of the device. For example, the user attributes may include one or more of the technical field in which the user is employed, the business organization associated with the user, the user's education level, the user's job function, and the number of years of the user's practical experience. By using the user attributes, the likelihood that the words having the presented definitions are words unknown to the user can be increased. For example, the definition of the medical term "osteoporosis" can be presented to an engineer rather than a medical professional.

[0023] Embodiments of the present disclosure are described with reference to the accompanying drawings.

[0024] FIG. 1 shows an exemplary environment 100 including a user interface for a communication session according to one or more embodiments of the present disclosure. In some embodiments, environment 100 can include a network 110, a user device 120, a device 130, an automatic speech recognition system 140, and a large language model 150.

[0025] In some embodiments, network 110 can be configured to communicatively couple user device 120, device 130, automatic speech recognition system 140, and large language model 150. In some embodiments, network 110 can be any wired or wireless network, or a combination of multiple networks, configured to transmit and receive communications between systems and devices. In some embodiments, network 110 can include a Personal Area Network (PAN), Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), Storage Area Network (SAN), cellular network, the Internet, an optical network, or a combination thereof.

[0026] In some embodiments, user device 120 can be any computer system capable of communicating via network 110 and participating in a communication session. For example, user device 120 can be a desktop computer, laptop computer, smartphone, mobile phone, tablet computer, or any other computing device that can be used for communication between devices via a network.

[0027] The user device 120 may include a display 122 that can present a user interface. As will be provided in more detail below, the environment 100 may be configured to provide information for presentation in the user interface presented on the display. For example, the user interface may be configured to provide information about the communication session in which the user device 120 is participating. In these and other embodiments, the user device 120 may be configured to present words and definitions of words used during the communication session in the user interface.

[0028] In some embodiments, the device 130 may be any computer system that can communicate via the network 110 and participate in a communication session. For example, the device 130 may be a desktop computer, a laptop computer, a smartphone, a mobile phone, a tablet computer, or any other computing device that can be used for communication between devices via the network. The device 130 may include a display similar to the display 122. As will be provided in more detail below, the user device 120 may perform one or more steps to present words and word definitions in audio data.

[0029] In some embodiments, each of the user device 120 and device 130 may include memory and at least one processor, which are configured to perform the operations described herein, among other operations. In some embodiments, each of the user device 120 and device 130 may include computer-readable instructions configured to be executed by each of the user device 120 and device 130 in order to perform the operations described herein.

[0030] In some embodiments, each of the user device 120 and device 130 may be configured to establish a communication session with another device. For example, each of the user device 120 and device 130 may be configured to establish an outgoing communication session, such as a telephone call, video call, video conference, or other communication session, with another device via a network, such as part of network 110.

[0031] In some embodiments, the automatic speech recognition system 140 may be any system that communicates over the network 110 and can convert audio data into transcript data. The automatic speech recognition system 140 may include any configuration of hardware such as processors, servers, and storage servers that are networked together and configured to perform audio transcription. For example, the automatic speech recognition system 140 may include one or more computing systems, such as multiple servers, each having memory and at least one processor. The automatic speech recognition system 140 may be configured to generate transcriptions from audio using automatic speech recognition technology. In these and other embodiments, the automatic speech recognition system 140 may include a speech engine trained to recognize speech in audio and generate speech transcriptions.

[0032] As used in this disclosure, the term “audio” may be used generally to refer to sounds that may include spoken language. Furthermore, the term “audio” may be used generally to include any form of audio, such as digital, analog, or propagated wave format. Furthermore, in digital format, audio may be compressed using different types of compression schemes.

[0033] In some embodiments, the Large Language Model 150 may be an artificial intelligence system such as an artificial neural network. For example, the Large Language Model 150 may be built using a transformer-based architecture with only a decoder. In these and other embodiments, the Large Language Model 150 may be configured as a prompt-based Large Language Model 150, which may operate by receiving input text and one or more prompts and iteratively predicting the next token or word based on the input text. In these and other embodiments, the Large Language Model 150 may acquire knowledge about language syntax, semantics, and ontology from a corpus used to train the Large Language Model 150. For example, the Large Language Model 150 may be part of a Large Language Model 150 system, such as OpenAI's GPT series models, Microsoft Copilot, Google's PaLM GeminI, Meta's lLaMA family of open-source models, Anthropic's Claude model, and Mistral A's open-source model, among other LLM systems. Alternatively or additionally, the above Large Language Model 150 systems may be examples of Large Language Model 150 implemented in environment 100.

[0034] In some embodiments, user device 120 may be in a communication session with device 130. Audio data, including voice, may be generated during the communication session between user device 120 and device 130. For example, device 130 may acquire audio data from its microphone and transmit that audio data to user device 120. User device 120 may be configured to broadcast the audio data for the user of user device 120 to hear.

[0035] In some embodiments, audio data generated during a communication session may be provided to the automatic speech recognition system 140 by the user device 120. For example, in some embodiments, audio data received by the user device 120 may be provided to the automatic speech recognition system 140.

[0036] The automatic speech recognition system 140 may be configured to receive audio data. After receiving the audio data, the automatic speech recognition system 140 may generate transcript data. The transcript data may contain one or more words of the speech transcription in the audio data. After the automatic speech recognition system 140 has generated the transcript data, it may send the transcript data to the user device 120.

[0037] After the transcript data is acquired by the user device 120, the transcript data may be provided to the large language model 150. The user device 120 may be further configured to provide the large language model 150 with instructions to analyze the transcript data to select from the transcript data any words that are technical, specific to a particular field, jargon, acronyms, or other complex words that may not be readily understood or familiar to the majority of people. Words selected by the large language model 150 may be referred to in this disclosure as unfamiliar words. For example, the transcript data may be analyzed to determine any word that is not included in the 3000 most commonly used words or any other number of words. Alternatively or additionally, the transcript data may be analyzed to determine any word that is not in the average vocabulary of a native speaker. Alternatively or additionally, the transcript data may be analyzed using the neural network of the large language model 150 to select words that are considered unfamiliar based on the training of the large language model 150. For example, the large-scale language model 150 may identify the term "osteoporosis" as a technical term and select the term "osteoporosis" using the large-scale language model 150.

[0038] In some embodiments, the user device 120 may provide the large language model 150 with a set of references via one or more prompts. The reference set may instruct the large language model 150 on how to analyze the transcript data and select one or more words.

[0039] In some embodiments, the user device 120 may be further configured to provide the large language model 150 with instructions to generate definitions for one or more selected words. In these and other embodiments, the user device 120 may provide the large language model 150 with another set of criteria via one or more prompts to instruct the large language model 150 to generate definitions. In these and other embodiments, the large language model 150 may provide the user device 120 with one or more selected words and definitions.

[0040] In some embodiments, the user device 120 can retrieve one or more selected words and definitions. In these and other embodiments, the user device 120 can be further configured to instruct the large language model 150 to select one or more words based on the user attributes of the user device 120. For example, the user device 120 may be further configured to provide the large language model 150 with instructions to select one or more words. In these and other embodiments, the user device 120 may provide the large language model 150 with another set of criteria via one or more prompts to instruct the large language model 150 to select one or more words. In these and other embodiments, the large language model 150 can provide instructions that one or more selected words may be unfamiliar to the user of the user device 120. The user device 120 can be configured to present one or more selected words as indicated by the large language model 150. For example, the large language model 150 may identify the term "osteoporosis" as a technical term. However, the large-scale language model 150 can also provide attributes indicating that the user of the user device 120 has a medical background or may work in the medical field. As a result, the large-scale language model 150 can determine that while the term "osteoporosis" may be unfamiliar to the general public, considering the user's specific characteristics, the term "osteoporosis" may not be unfamiliar to the user.

[0041] In these and other embodiments, the user device 120 may be further configured to provide instructions to the large language model 150 to select one or more words based on the user attributes of the user device 120. For example, in some embodiments, the large language model 150 may select one or more words and then screen the selected one or more words before providing them to the user device 120. Alternatively or additionally, the large language model 150 may select one or more words using attributes relating to the user of the user device 120. As a result, the words obtained by the user device 120 may be words that are likely to be unfamiliar to the user.

[0042] The user device 120 can obtain one or more selected words and their definitions from the large-scale language model 150. While audio data is being broadcast, the one or more selected words and their definitions may be presented on the display 122 of the user device 120 via the user interface.

[0043] In some embodiments, the selected word and its definition may be presented in real time via the user interface during a communication session between the user device 120 and the device 130, in connection with the broadcast of a portion of the audio data containing the selected word. For example, the audio data may include speech containing the selected word, and the selected word and its definition may be presented in real time on the display 122 of the user device 120 immediately after the portion of the audio data has been broadcast. For example, once the audio data is provided to the user device 120 for broadcast, transcript data may be generated and provided to the large language model 150 for word selection and definition generation. Thus, there may be a small lag between when the word is spoken and when the definition is presented, but this lag may be due to processing by the automatic speech recognition system 140 and the large language model 150.

[0044] As described in this disclosure, environment 100 can operate to present a continuous flow of definitions of unfamiliar words in audio data via the device's user interface as audio data is generated during a communication session and broadcast by the device. In this way, environment 100 can present real-time definitions of words from audio data that are part of a communication session via the user interface. Conventionally, user interfaces have not been able to provide real-time definitions of communication sessions, such as unscripted communication sessions. The systems and methods described in this disclosure provide a technical solution for user interfaces that are unable to provide additional information to the user of the user interface.

[0045] Modifications, additions, or omissions may be made to the environment 100 without departing from the scope of this disclosure. For example, in some embodiments, the environment 100 may not include the automatic speech recognition system 140. In these and other embodiments, the user device 120 may include automatic speech recognition technology, enabling the user device 120 to perform functions previously performed by the automatic speech recognition system 140. For example, the user device 120 may individually acquire audio data including speech and generate transcript data from the audio. The automatic speech recognition system 140 may also be included in the device 130 and shared between devices.

[0046] In other examples, in some embodiments, the environment 100 may not include the large language model 150. In these and other embodiments, the user device 120 may include artificial intelligence technology to enable the user device to perform functions previously performed by the large language model 150. For example, the user device 120 may individually acquire transcript data generated from audio data, analyze the transcript data to select one or more words from the transcript data, generate definitions for the selected one or more words, and / or present the selected one or more words and their definitions based on instructions. Similarly, the large language model 150 may be included in the device 130, and the device 130 may individually perform any of the steps provided above before sending the selected one or more words and their definitions to the user device 120 for presentation.

[0047] In some embodiments, the user device 120 can perform each of the operations described. In some embodiments, a separate device, such as device 130, can perform the operations described above. In these and other embodiments, a separate device, such as device 130, can transmit selected words and definitions of selected words to the user device 120 for presentation on the display 122.

[0048] In some embodiments, transcript data can be acquired and / or provided to the large language model 150 by the device 130. Additionally or alternatively, the device 130 can provide the large language model 150 with instructions on how to analyze the transcript data.

[0049] In some embodiments, processes performed by the user device 120 can also be performed by the device 130 during a communication session. For example, the device 130 can acquire transcript data of audio data acquired from the user device 120, instruct the automatic speech recognition system 140 to generate transcript data, and provide the transcript data to the large language model 150 for processing. As a result, the device 130 may be configured to present one or more words and definitions applicable to the user of the device 130 via its user interface. It should be noted that the words presented by the device 130 may differ from the words presented by the user device 120. For example, the user of the user device 120 may have different attributes than the user of the device 130. For example, the user of the user device 120 may be a teacher, and the user of the device 130 may be a student. Therefore, definitions of technical terms may be presented to the student via the user interface of the device 130, and definitions of slang used by the student may be presented to the teacher via the user interface of the user device 120.

[0050] In some embodiments, each of user device 120 and device 130 may perform the operations described above. Thus, each of user device 120 and device 130 may perform similar operations. Alternatively or additionally, a separate device or system may be configured to perform operations to acquire audio data of the communication and to acquire one or more words and definitions for each of user device 120 and device 130. In these and other embodiments, the separate device or system may provide each of user device 120 and device 130 with one or more corresponding words and definitions for presentation via the user interface by each of user device 120 and device 130. For example, the separate device or system may be a system or device that helps host or provide infrastructure for a communication session. Alternatively or additionally, the separate device or system may perform one or more of the operations described above. For example, the separate device or system may acquire transcript data and provide the transcript data to user device 120 and device 130. Alternatively or additionally, a separate device or system may acquire transcript data, one or more selected words, and / or definitions of one or more words. In these and other embodiments, the separate device or system may provide one or more words to user devices 120 and 130. In these and other embodiments, user devices 120 and 130 may select the provided one or more words based on user attributes before presenting the one or more words and definitions via the user interface.

[0051] In some embodiments, the definitions of words used in speech from audio data can be obtained by the user device 120, and the audio is obtained by the user device 120 from device 130. Therefore, the definitions of words spoken by the user of the user device 120 during a communication session do not need to be presented to the user. Alternatively or additionally, definitions can be generated for the user device 120 using the definitions of words from all audio data in the communication session, regardless of the origin of the audio data. Alternatively or additionally, the user may choose to use definitions for all audio, all audio not originating from the user device, or only audio originating from some devices in the communication session to determine the definitions for presentation.

[0052] Figure 2 shows an exemplary operational workflow 200 for presenting a user interface. The operational workflow 200 may include an automatic speech recognition system 210, a large-scale language model 225, and a user device 250. The automatic speech recognition system 210 may be the same as the automatic speech recognition system 140 described with reference to Figure 1. The large-scale language model 225 may be the same as the large-scale language model 150 described with reference to Figure 1. The user device 250 may be the same as the user device 120 described with reference to Figure 1.

[0053] In the operational workflow 200, audio data 205 may be provided to the automatic speech recognition system 210. The audio data may be obtained from a communication session including the user device 250, and the audio data 205 may include audio from the communication session. The audio data 205 may be provided by the user device 250 or some other device.

[0054] The automatic speech recognition system 210 may generate transcript data 215 from the audio data. The transcript data 215 may contain one or more words of the speech transcription in the audio data 205. The transcript data 215 may be provided from the automatic speech recognition system 210 to the large language model 225. For example, in some embodiments, the automatic speech recognition system 210 may provide the transcript data 215 to a user device 250, which can then provide the transcript data 215 to the large language model 225. Alternatively or additionally, another device may provide the transcript data 215 to the large language model 225.

[0055] In some embodiments, a first set of criteria 220 can be provided to a large language model 225. The large language model 225 can be configured to analyze transcript data 215 based on the first set of criteria 220 and select unfamiliar words from the transcript data 215. In some embodiments, the large language model 225 can perform the analysis of the transcript data 215 word by word, phrase by phrase, and / or sentence by sentence to enable the large language model 225 to select one or more words from the transcript data 215 and / or enable the large language model 225 to perform any of the subsequent operations in real time as it receives the transcript data 215. In some embodiments, the first set of criteria 220 can be provided to the large language model 225 by a user device 250. The first set of criteria 220 can provide criteria that instruct the large language model 225 on how to select unfamiliar words from the transcript data as selected words 230. For example, the first set of criteria 220 may include input definitions, desired outputs, output formats, and further restrictions on the outputs.

[0056] In some embodiments, one or more prompts can provide the first set of criteria 220 to the large language model 225. In these and other embodiments, one or more prompts may be constructed using a chain of thought technique in which the prompts break down the main objective into intermediate tasks for the large language model 225, and / or the prompts may be constructed using a meta-prompt technique in which the prompts break down the task into subtasks. An example of a prompt that may be provided to the large language model 225 is: "The job is to select words that may be unfamiliar to the audience. The input is an audio transcript in which sentence after sentence. For each sentence, the task is to select words that the audience may not fully understand. The output shall be in the form of a list of words: [{word 1}, {word 2}]. Leave the list blank if all words in the input phrase are common words that do not require further explanation. Words already identified in previous input phrases do not need to be output."

[0057] In these and other embodiments, the first criterion set 220 may include one or more features relating to a communication session. These features can help the large language model 225 select unfamiliar words. For example, some features of a communication session may include information about the meeting, such as whether the communication session is a virtual meeting, and if so, the size of the meeting, the type of meeting, company information of the meeting participants, the organization responsible for the meeting, the organizations participating in the meeting, the individuals responsible for the meeting, the individuals participating in the meeting, and / or the subject of the meeting. For example, the first criterion set 220 may indicate that the company hosting the meeting is a hospital, and that the organizational participants include the legal department, the human resources department, and the oncology department. This information may be used by the large language model 225 to determine words that may be unfamiliar to this group of people.

[0058] The large-scale language model 225 can analyze the transcript data 215 based on a first criterion set 220 in order to select unfamiliar words 230 from the transcript data 215.

[0059] A second set of criteria 240 may be provided to the large language model 225. The large language model 225 may be instructed to output definitions 235 of selected words 230 based on the second set of criteria 240. The second set of criteria 240 can provide criteria that instruct the large language model 225 on how to generate definitions of unfamiliar selected words 230. For example, the second set of criteria 240 may include input definitions, desired outputs, output formats, and further constraints on the outputs.

[0060] In some embodiments, the selected word 230 may be retrieved from the large language model 225 by the user device 250 and then returned to the large language model 225 along with the second criterion set 240.

[0061] In some embodiments, one or more prompts can provide the large language model 225 with a second set of criteria 240. In these and other embodiments, the prompts may be structured similarly to or differently from the previous prompts. An example of a prompt that may be provided to the large language model 225 is: "The job is to define selected words. The input is the selected words of speech, one by one, in sequence. For each sentence, the task is to provide a definition for each selected word. The output shall be in the form of word-definition pairs: [{word1:definition1},{word2:definition2}]. A word-definition pair shall not be output if it has already been identified in the previous input phrase; that is, each word-definition pair appears only once."

[0062] In these and other embodiments, the second criterion set 240 may also include characteristics relating to the communication session, similar to those of the first criterion set 220. In some embodiments, the large language model 225 may provide definitions of selected words 230 that may be appropriate to the context of the communication session. For example, the large language model 225 may acquire transcript data and understand the context of the transcript data. As a result, the large language model 225 may be able to provide definitions that are more appropriate to the context in which the selected words 230 are used.

[0063] In some embodiments, the first criterion set 220 and the second criterion set 240 may be provided as a single prompt. For example, the following prompt may be provided to the large language model 225: "The job is to select and define words that may be unfamiliar to the audience. The input is an audio transcript consisting of sentence after sentence. For each sentence, the task first selects one of the words that the audience may not fully understand, and then provides a definition for each word. The output shall be in the form of a list of word-definition pairs: [{word1:definition1},{word2:definition2}]. A word-definition pair shall not be output if it has already been identified in the previous input phrase; that is, each word-definition pair appears only once."

[0064] Based on the second reference set 240, the large language model 225 can output a definition 235 for the selected word 230. In these and other embodiments, the large language model 225 can instruct the user device 250 to provide the definition 235 for the selected word 230. As a result, the user device 250 can include the selected word 230 and its definition 235.

[0065] In some embodiments, the third criterion set 242 can be provided to the large language model 225 by the user device 250 to select the words 230 and definitions 235 to be presented by the user device 250. In these and other embodiments, the third criterion set 242 can provide criteria for instructing the large language model 225 on how to select the words 230 and definitions 235. For example, the third criterion set 242 may include input definitions, desired outputs, output formats, and further restrictions on the outputs.

[0066] In some embodiments, the user device 250 can provide a third reference set 242 and / or selected words 230 and definitions 235 to a large language model 225.

[0067] In these and other embodiments, the third criterion set 242 may include one or more attributes of the user of the user device 250. For example, one or more attributes of the user of the user device 250 may include the technical field in which the user is employed, the business organization associated with the user, the user's level of education, the user's job, and the number of years of work experience of the user.

[0068] In some embodiments, one or more prompts can provide the third set of criteria 242 to the large language model 225. In these and other embodiments, the prompts may be structured similarly to or differently from the previous prompts. An example of a prompt that may be provided to the large language model 225 is: "The job is to reduce the number of words in a given list based on the fact that the audience already understands some of these words based on their background. The input is a list of word-definition pairs in the following format: {word1:definition1}, {word2:definition2}, one set after another. The audience's background is [...]. The task is to generate a new list containing all the words that this audience may already understand in the form "[understood word1, understood word2]", and then to remove the word-definition pairs in the understood words list to generate a new word-definition list in the original format." In these and other prompts, the prompts may be dynamically filed with appropriate information.

[0069] Based on the third set of criteria 242, the large language model 225 may be instructed to determine which of the selected words 230 and definitions 235 may be presented to the user device 250. The large language model 225 may provide one or more instructions 245 regarding which of the selected words 230 and definitions 235 are presented to the user device 250. For example, in some embodiments, the instruction 245 may be the large language model 225 providing the selected words 230 and definitions 235 to be presented.

[0070] In some embodiments, one or more of the presented instructions 245 may be acquired by the user device 250. In these and other embodiments, the user device 250 may present the indicated selected word 230 and definition 235 on the user interface of the display. The user device 250 may present the indicated selected word 230 and definition 235 on the user interface of the display in real time during a communication session in connection with the broadcast of a portion of the audio data 205 containing the selected word 230.

[0071] The user interface of the user device 250 may obtain feedback data 255 from the user of the user device 250 regarding one or more presented words and definitions. In some embodiments, the feedback data may be negative data and / or positive feedback data based on negative and / or positive feedback provided by the user of the user device 250. In these and other embodiments, the negative data may indicate that the user is familiar with the presented words. Alternatively or additionally, the positive data may indicate that the user is not familiar with the presented words.

[0072] In some embodiments, the user device 250 may update the third criterion set 242 with feedback data 255. The update of negative feedback data may include the third criterion set 242 containing words familiar to the user. As a result, when the third criterion set 242 is provided to the large language model 225, the large language model 225 may decide not to present one or more of the selected words 230. For example, a user of the user device 250 may provide negative feedback about the word "osteoporosis" on the user interface. In these and other embodiments, the user device 250 may update the third criterion set 242 with that negative feedback. As a result, when the word "osteoporosis" is selected by the large language model 225, the large language model 225 may decide not to have the user device 250 present the word "osteoporosis". Alternatively or additionally, the large-scale language model 225 may learn from words the user is familiar with and, based on the user's knowledge of words to which the user provides negative feedback, may not need to select other words for presentation.

[0073] In some embodiments, updating positive feedback data may include a third criterion set 242 containing words unfamiliar to the user. As a result, when the third criterion set 242 is provided to the large language model 225, the large language model 225 may decide to present selected words 230 and definitions 235. In some embodiments, the large language model 225 may decide to present words and definitions corresponding to positive feedback data if one or more words appear in the transcript data 215, regardless of whether the large language model 225 has selected any words. For example, a user of the user device 250 may provide positive feedback about the word "osteoporosis" on the user interface. In these embodiments and other embodiments, the user device 250 can update the third criterion set 242 using feedback data 255, and when the word "osteoporosis" appears in the transcript data 215, the large language model 225 can decide to present the word "osteoporosis" and its definition, even if "osteoporosis" has not been selected by the large language model 225.

[0074] Modifications, additions, or omissions may be made to the operational workflow 200 without departing from the scope of this disclosure. For example, in some embodiments, the operational workflow 200 may not include the automatic speech recognition system 210. In these and other embodiments, the user device 250 may include automatic speech recognition technology, enabling the user device 250 to perform functions previously performed by the automatic speech recognition system 210. For example, the user device 250 may individually acquire audio data 205 containing speech and generate transcript data 215 from the audio data 205. The automatic speech recognition system 210 may similarly be included in a separate device from the user device 250.

[0075] In other examples, in some embodiments, the operational workflow 200 may not include the large language model 225. In these and other embodiments, the user device 250 may include artificial intelligence technology to enable the user device to perform functions previously performed by the large language model 225. For example, the user device 250 may individually acquire transcript data 215 generated from audio data 205, analyze the transcript data 215 based on a first criterion set 220 to select words 230 from the transcript data 215, generate definitions 235 for the selected words 230, output definitions 235 for the selected words 230 based on a second criterion set 240, decide whether to present the selected words 230 and definitions 235 on the user device 250 based on a third criterion set 242, provide one or more instructions 245 to the user device 250, and / or present the selected words and definitions based on the instructions. Similarly, the large language model 225 may be included in a separate device from the user device 250, and the separate device may perform any of the steps described above individually.

[0076] In some embodiments, data from the first criterion set 220, the second criterion set 240, and / or the third criterion set may be prompts provided in a single interaction with the large language model 225. For example, the following prompt may be provided to the large language model 225: "The job is to select and define words that may be unfamiliar to the audience. The input is a transcript of audio, sentence by sentence. For each sentence, the task is to first select any of the words that the audience may not fully understand, then provide a definition for each word, and reduce the number of word-definition pairs in the list based on the audience's background, which may already understand some of these words. The audience's background is [...]. The output shall be in the form of a list of word-definition pairs: [{word1:definition1},{word2:definition2}]. A word-definition pair shall not be output if it has already been identified in the previous input phrase; that is, each word-definition pair appears only once."

[0077] Alternatively or additionally, the first criterion set 220 or the second criterion set 240 may include one or more attributes of the user of the user device 250 and / or other participants in the communication session. As a result, the large language model 225 does not need to select words that may be familiar to the user or other users, and / or generate definitions of words that may be familiar to the user or other users.

[0078] Figure 3 shows an exemplary user device 300 for presenting a user interface. User device 300 may be any computer system capable of participating in a communication session. For example, user device 300 may be a desktop computer, laptop computer, smartphone, mobile phone, tablet computer, or any other computing device that can be used for communication between devices over a network. User device 300 may be similar to user device 120 or user device 250 shown in Figures 1 and 2, respectively.

[0079] The user device 300 may include a speaker 330. The speaker 330 may be any audio component and / or system capable of outputting audio. In some embodiments, the speaker 330 may output audio generated during a communication session. For example, the speaker 330 may output audio generated during a communication session between device 300 and another device.

[0080] The user device 300 may include a display 310 that can present a user interface. In these and other embodiments, the display may include a visual presentation 312, a transcript field 314, a display area 316 for selected words and definitions, feedback control 318, and a stored word field 320.

[0081] The visual presentation 312 may include a video depiction of the communication session. For example, the visual presentation 312 may present a video feed from another device participating in a communication session with the user device 300.

[0082] The transcript can be displayed on the display 310 of the transcript field 314 and may include text written based on transcript data acquired by the user device 300. In some embodiments, the user device 300 can acquire transcript data generated from audio data by voice via automatic speech recognition technology and present the transcript data. The transcript data may be presented in real time during the communication session.

[0083] The selected words and definitions may be presented on the display 310 by the user device 300 within the display area 316. In some embodiments, the selected words may be obtained from the large language model after the large language model has analyzed the transcript data and selected words from the transcript data. Furthermore, the large language model may generate definitions for the selected words, which may be obtained by the user device 300 for presentation on the display 310.

[0084] Presenting the selected words and definitions on the user device 300's display 310 improves the user interface of the display 310 because the display presents the transcript of the communication session in the transcript field 314 and also presents the selected words from the transcript and provides their definitions. Therefore, the user of the user device 300 does not need to exit the display 310 to search for the selected words and definitions.

[0085] The feedback control 318 may enable the user device 300 to obtain feedback data based on feedback from the user of the user device 300 regarding selected words and definitions. In some embodiments, the feedback data may be negative data and / or positive feedback data based on negative and / or positive feedback provided by the user of the user device 300. In these and other embodiments, the feedback control 318 may include one or more simple user interaction mechanisms, such as a single-click button, a touchscreen tap, and / or a touchscreen swipe, to enable the user of the user device 300 to provide negative and / or positive feedback regarding selected words and definitions to the user device 300.

[0086] For example, the user of the user device 300 may provide negative feedback regarding the selected word and definition, which may generate negative feedback data. In some embodiments, the user device 300 may remove the selected word and definition from the display 310 based on the negative feedback data.

[0087] In another example, the user of the user device 300 may provide positive feedback regarding the selected word and definition, which may generate positive feedback data. In some embodiments, the user device 300 may save the selected word and definition in a stored word field 320 on the display 310 based on the positive feedback data.

[0088] In some embodiments, the user device 300 can save the selected words and definitions to a saved word field 320 on the display 310 without obtaining positive feedback data. In these and other embodiments, the user of the user device 300 can recall the selected words and definitions for re-presentation on the display 310.

[0089] Modifications, additions, or omissions may be made to the user device 300 without departing from the scope of this disclosure. For example, in some embodiments, the user device 300 may not include the visual presentation 312. Alternatively or additionally, the user device 300 may not present the transcript field 314, the feedback control 318, and / or the stored word field 320.

[0090] Figure 4 shows a flowchart of an exemplary method 400 for presenting a user interface, according to one or more embodiments of the present disclosure. Method 400 may be performed using any suitable system, device, or apparatus. For example, Method 400 can be performed using the environment 100 in Figure 1 or the computing system 600 in Figure 6. Although shown in separate blocks, steps and operations related to one or more blocks of Method 400 may be separated into additional blocks, combined into fewer blocks, or deleted, depending on the specific implementation. For example, one or more of the operations described above with respect to the operation workflow 200 in Figure 2 may be performed as part of Method 400.

[0091] The user interface presentation method 400 may include blocks 410, 420, 430, 440, and 450. In block 410, transcript data generated from audio data including speech may be acquired via automatic speech recognition technology. The transcript data may include one or more words of the speech transcription in the audio data. The automatic speech recognition technology may be the same as the automatic speech recognition systems 140 and 210 described with reference to Figures 1 and 2, respectively. The transcript data may be acquired by a device similar to the user devices 120, 250, and 300 described with reference to Figures 1, 2, and 3, respectively. The transcript data may be acquired by a device or a system separate from the device.

[0092] In block 420, the transcript data and the first criterion set may be provided to a large language model. In these and other embodiments, the large language model may be configured to analyze the transcript data based on the first criterion set to select words from the transcript data. In some embodiments, the first criterion set may be provided to the large language model 225 via one or more prompts. In these and other embodiments, the first criterion set may instruct the large language model to select one or more words from the transcript data, which are technical, acronyms, jargon, or other words that a typical person would not understand.

[0093] The first criterion set may be similar to the first criterion set described above, such as the first criterion set 220 described with respect to Figure 2. The large-scale language models may be similar to the large-scale language models 150 and 225 described with respect to Figures 1 and 2, respectively. The transcript data may be provided by devices similar to the user devices 120, 250 and 300 described with respect to Figures 1, 2 and 3, respectively, or by other devices or systems.

[0094] In block 430, the selected words and the definitions of the selected words generated by the large language model may be obtained. In some embodiments, the selected words and their definitions may be obtained by devices similar to the user devices 120, 250, and 300 described with respect to Figures 1, 2, and 3, respectively, or by other devices or systems.

[0095] In block 440, the audio data may be broadcast by the device. The audio data may be generated during a communication session between the device and another device. In some embodiments, the method may further include acquiring the audio data in the device before broadcasting it. The devices may be similar to user devices 120, 250, and 300 described with reference to Figures 1, 2, and 3, respectively. Other devices may be similar to device 130 described with reference to Figure 1.

[0096] In block 450, the selected word and its definition may be presented on the device's display along with the broadcasted audio data. In some embodiments, the display includes a user interface, and the selected word and its definition may be presented on the user interface while the audio data is being broadcast. In these and other embodiments, the user of the user interface can view the selected word and its definition while listening to the audio without navigating away from the user interface. In some embodiments, the selected word and its definition may be presented in real time during the communication session in connection with the broadcast of a portion of the audio data containing the selected word.

[0097] Modifications, additions, or omissions may be made to Method 400 without departing from the scope of this disclosure. For example, the designation of different elements as described is intended to help illustrate the concepts described herein, and is not limiting. Furthermore, Method 400 may include any number of other elements or may be implemented in a system or environment other than those described.

[0098] For example, method 400 may further include the step of providing a second criterion set to a large language model, which is configured to output definitions of words selected based on the second criterion set. The second criterion set may be similar to the second criterion set described above, as shown in the second criterion set 240 in Figure 2.

[0099] In another example, Method 400 may further include the step of the device obtaining instructions to present selected words and definitions of selected words on a display, and the selected words and definitions of selected words may be presented in response to obtaining instructions to present selected words and definitions of selected words on a display. In these and other embodiments, Method 400 may further include the step of the device providing a large language model with a third set of criteria, which the large language model is configured to determine, based on the third set of criteria, whether to present selected words and definitions of selected words on a display and to provide instructions to the device. An example of the third set of criteria includes user attributes of the device. For example, user attributes of the device include the technical field in which the user is employed, the business organization associated with the user, the user's education level, the user's job, and the number of years of work experience of the user. The third set of criteria may be similar to the third set of criteria described above, as shown in the third set of criteria 242 in Figure 2.

[0100] Figure 5 shows a flowchart of an exemplary method 500 for improving a user interface according to one or more embodiments of the present disclosure. Method 500 may be performed using any suitable system, equipment, or apparatus. For example, Method 500 can be performed using the environment 100 in Figure 1 or the computing system 600 in Figure 6. Although shown in separate blocks, steps and operations related to one or more blocks of Method 500 may be separated into additional blocks, combined into fewer blocks, or deleted, depending on the specific implementation. For example, one or more of the operations described above with respect to the operation workflow 200 in Figure 2 may be performed as part of Method 500.

[0101] In block 510, text data containing multiple words may be acquired. The text data may be data acquired from written text, images, videos, or audio, or the text data may be similar to the data of the transcript data acquired in block 410 of method 400. For example, the text data may be generated from a communication session between one device and another device.

[0102] In block 520, text data and a first criterion set may be provided to the artificial intelligence system. In these and other embodiments, the artificial intelligence system may be configured to analyze the transcript data based on the first criterion set to select words from the text data and generate definitions for the selected words. In some embodiments, the artificial intelligence system may be similar to the large language models 150 and 225 in Figures 1 and 2, respectively. The first criterion set may be similar to the first criterion set described above, such as the first criterion set 220 in Figure 2.

[0103] In block 530, the selected word and its definition may be obtained from the artificial intelligence system. In some embodiments, a second set of criteria can be provided to the artificial intelligence system, and the artificial intelligence system is configured to output the definition of the selected word based on the second set of criteria. The second set of criteria may be similar to the second set of criteria described above, such as the second set of criteria 240 described with respect to Figure 2.

[0104] In block 540, text data, selected words, and definitions of selected words may be presented on the device's display. The device may be similar to user devices 120, 250, and 300 described with respect to Figures 1, 2, and 3, respectively. In some embodiments, the display may be configured to present a user interface. In these and other embodiments, the user of the user interface can view the selected words and definitions of selected words within the user interface while listening to audio, without having to navigate away from the user interface.

[0105] In some embodiments, instructions can be obtained to present a selected word and its definition within a user interface on a display, and in response to obtaining instructions to present a selected word and its definition within a user interface on a display, the selected word and its definition can be presented.

[0106] In these and other embodiments, a third set of criteria can be provided to the artificial intelligence system, which is configured to determine whether to present selected words and their definitions within the user interface on the display based on the third set of criteria, and to provide instructions to the device regarding which selected words to display. The third set of criteria may be similar to the third set of criteria described above, such as the third set of criteria 342 described with respect to Figure 2.

[0107] Modifications, additions, or omissions may be made to Method 500 without departing from the scope of this disclosure. For example, the designation of different elements as described is intended to help illustrate the concepts described herein, and is not limiting. Furthermore, Method 500 may include any number of other elements or may be implemented in a system or environment other than those described.

[0108] Figure 6 shows a block diagram of an exemplary computing system 600 according to at least one embodiment of the present disclosure. The computing system 600 may be configured to perform or direct one or more appropriate operations described in the present disclosure. For example, the computing system 600 may be part of one or more elements of Figure 1 and / or may be configured to perform one or more processes of Figure 2, the method of Figure 4, or the method of Figure 5. The computing system 600 may include a processor 610, memory 620, and data storage device 630. The processor 610, memory 620, and data storage device 630 may be communicatively coupled.

[0109] Typically, the processor 610 may include any suitable dedicated or general-purpose computer, computing entity, or processing unit, including various computer hardware or software modules, and may be configured to execute instructions stored in any suitable computer-readable storage medium. For example, the processor 610 may include a microprocessor, microcontroller, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or any other digital or analog circuit configured to interpret and / or execute program instructions and / or process data. Although shown as a single processor in Figure 6, the processor 610 may include any number of processors configured to perform or direct any number of operations described herein, individually or collectively. Furthermore, one or more of the processors may reside on one or more different electronic devices, such as different servers.

[0110] In some embodiments, the processor 610 may be configured to interpret and / or execute program instructions and / or process data stored in memory 620, data storage device 630, or memory 620 and data storage device 630. In some embodiments, the processor 610 may fetch program instructions from data storage device 630 and load the program instructions into memory 620. After the program instructions are loaded into memory 620, the processor 610 may execute the program instructions.

[0111] The memory 620 and data storage device 630 may include computer-readable storage media that carry or have stored computer-executable instructions or data structures. For example, but not limited to, such computer-readable storage media may include tangible or non-temporary computer-readable media, including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM), or other optical disk storage devices, magnetic disk storage devices, or other magnetic storage devices, flash memory elements (e.g., solid memory devices), or any other non-temporary storage media that can be used to carry or store specific program code in the form of computer-executable instructions or data structures and can be accessed by a general-purpose or dedicated computer. In these and other embodiments, the term “non-temporary” as described herein should be considered to exclude only types of temporary media that are not included in the scope of patentable subject matter in Federal Circuit decision of In re Nuijten, 500 F.3d 1346 (Fed. Cir. 2007).

[0112] The above combinations may also be included in the scope of computer-readable storage media. Computer-executable instructions may include, for example, instructions and data configured to cause the processor 610 to perform a specific operation or a set of operations.

[0113] Modifications, additions, or omissions may be made to the computing system 600 without departing from the scope of this disclosure. For example, in some embodiments, the computing system 600 may include any number of other components, which may or may not be expressly described.

[0114] The foregoing disclosure is not intended to limit the invention to the disclosed detailed form or a particular art of use. Therefore, various alternative embodiments and / or modifications to the disclosure are considered possible in light of the disclosure, whether or not they are expressly described or shown herein. Accordingly, by describing embodiments of the disclosure, it is understood that modifications may be made formally and in detail without departing from the scope of the disclosure. Accordingly, the disclosure is limited only by the claims.

[0115] In some embodiments, components, modules, engines, and services different from those described herein may be implemented as objects or processes (e.g., separate threads) that run on a computing system. While some of the systems and methods described herein are generally described as being implemented in software (stored in and / or run on general-purpose hardware), dedicated hardware implementations or combinations of software and dedicated hardware implementations are also possible and conceivable.

[0116] In accordance with common practice, various features shown in the drawings may not be drawn to scale. The drawings shown in this disclosure are not intended to be actual diagrams of any particular device (e.g., apparatus, system, etc.) or method, but are merely idealized representations used to illustrate various embodiments of the disclosure. Accordingly, the dimensions of various features may be enlarged or reduced as appropriate for clarity. Furthermore, parts of the drawings may be simplified for clarity. Accordingly, the drawings may not show all components of a given device (e.g., apparatus) or all operations of a particular method.

[0117] The terms used herein and in particular in the appended claims (e.g., the appended claim bodies) are generally intended to be “broad” terms (for example, the term “including” should be interpreted as “including, but not limited to,” and the term “having” should be interpreted as “having, but not limited to,” etc.).

[0118] Furthermore, where a specific number of introduced claims are intended, such intention is explicitly indicated in the claim; where there is no such enumeration, such intention does not exist. For example, for the sake of understanding, the claims attached below may include the use of the introductory phrases “at least one” and “one or more” to introduce an enumeration of claims. However, the use of such phrases should not be interpreted as meaning that the introduction of an enumeration of claims by the indefinite article “a” or “an” means “at least one” or “one or more,” even when the same claim includes the introductory phrase “one or more” or “at least one” and the indefinite article “a” or “an” (for example, “a” and / or “an” should be interpreted as meaning “at least one” or “one or more”). In other words, the same applies to the use of the definite article used to introduce an enumeration of claims.

[0119] Furthermore, when an enumeration of claims introducing a specific number is explicitly stated, it should be understood that such an enumeration should be interpreted as meaning at least the number enumerated (for example, the statement “two enumerations” without other qualifications means at least two enumerations, or two or more enumerations). Furthermore, in examples where a statement similar to “at least one of A, B, and C etc.” or “one or more of A, B, and C etc.” is used, such a configuration is usually intended to include A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together, etc. For example, the use of the term “and / or” is intended to be interpreted in this manner.

[0120] Furthermore, any separate word or phrase representing two or more alternative terms should be understood to presuppose the possibility of including one of the terms, either of the terms, or both of the terms, whether in the description, claims, or drawings. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B" or "A and B".

[0121] Furthermore, the use of terms such as "first," "second," and "third" is not necessarily used herein to imply a particular order or number of elements. Typically, terms such as "first," "second," and "third" are used as general identifiers to distinguish different elements. Unless otherwise indicated, terms such as "first," "second," and "third" should not be understood to mean a particular order. Furthermore, unless otherwise indicated, terms such as "first," "second," and "third" should not be understood to mean a particular number of elements. For example, a first widget may be described as having a first aspect, and a second widget may be described as having a second aspect. The use of the term "second aspect" in relation to a second widget is to distinguish such aspect of the second widget from the "first aspect" of the first widget, and does not mean that the second widget has two aspects.

[0122] All examples and conditional language described herein are intended for teaching purposes to help the reader understand the present invention and the concepts by which the invention contributes to further development of the art, and should be construed as not being limited to such specifically listed examples and conditions. Although embodiments of the disclosure have been described in detail, it should be understood that various modifications, substitutions, and choices can be made thereto without departing from the spirit and scope of the disclosure.

[0123] In addition to the embodiments described above, the following further notes are disclosed. (Note 1) A step of obtaining transcript data generated from audio data including speech via automatic speech recognition technology, wherein the transcript data includes one or more words of the transcription of the speech in the audio data. A step of providing the transcript data and a first set of criteria to a large-scale language model, wherein the large-scale language model is configured to analyze the transcript data based on the first set of criteria and select words from the transcript data. The steps include obtaining the selected word and the definition of the selected word generated by the large-scale language model, The device broadcasts the audio data, Along with broadcasting the audio data, the device displays the selected word and its definition on its display. A method that includes this. (Appendix 2) The method according to Appendix 1, further comprising the step of obtaining an instruction to present the selected word and the definition of the selected word on the display, wherein the selected word and the definition of the selected word are presented in response to obtaining the instruction to present the selected word and the definition of the selected word on the display. (Appendix 3) The method according to Appendix 2, further comprising the step of providing the large language model with a second set of criteria, wherein the large language model is configured to determine, based on the second set of criteria, whether to present the selected words and definitions of the selected words on the display, and to provide the device with the instructions. (Note 4) The method described in Note 3, wherein the second reference set includes the user attributes of the device. (Note 5) The method according to Note 4, wherein the user's attributes include one or more of the following: the technical field in which the user is employed, the business organization associated with the user, the user's level of education, the user's job duties, and the number of years of work experience of the user. (Note 6) Steps to obtain user feedback regarding the selected words and definitions presented, A step of updating the second set of criteria based on the user feedback, wherein the updated second set of criteria is provided to the large-scale language model in a future communication session, The method described in Appendix 5, which further includes the following. (Appendix 7) The method according to Appendix 1, further comprising the step of providing a second set of criteria to the large language model, wherein the large language model is configured to output definitions of the selected words based on the second set of criteria. (Note 8) The method according to Note 1, wherein the audio data is generated during a communication session between the device and another device, and the method further comprises the step of acquiring the audio data in the device before broadcasting the audio data. (Appendix 9) The method according to Appendix 7, wherein the selected word and the definition of the selected word are presented in real time during the communication session in connection with the broadcast of a portion of the audio data containing the selected word. (Appendix 10) One or more non-temporary computer-readable media configured to store instructions, wherein, when executed, the instructions perform the method described in Appendix 1. (Note 11) A device, One or more non-temporary computer-readable media configured to store instructions, A processor coupled to the computer-readable medium and configured to execute the instructions and perform an operation, The operation includes, A step of obtaining transcript data generated from audio data including speech via automatic speech recognition technology, wherein the transcript data includes one or more words of the transcription of the speech in the audio data. A step of providing the transcript data and a first set of criteria to a large-scale language model, wherein the large-scale language model is configured to analyze the transcript data based on the first set of criteria and select words from the transcript data. The steps include obtaining the selected word and the definition of the selected word generated by the large-scale language model, The device broadcasts the audio data, Along with broadcasting the audio data, the device displays the selected word and its definition on its display. A device including a device. (Note 12) The above operation is The apparatus according to Appendix 11, further comprising the step of obtaining an instruction to present the selected word and the definition of the selected word on the display, wherein the selected word and the definition of the selected word are presented in response to obtaining the instruction to present the selected word and the definition of the selected word on the display. (Note 13) The apparatus according to Note 12, wherein the operation further includes the step of providing a second set of criteria to the large language model, the large language model is configured to determine, based on the second set of criteria, whether to present the selected word and the definition of the selected word on the display, and to provide the instructions to the apparatus. (Note 14) The second reference set is the device described in Note 13, including the user attributes of the device. (Note 15) The apparatus as described in Note 14, wherein the user's attributes include one or more of the following: the technical field in which the user is employed, the business organization associated with the user, the user's level of education, the user's job duties, and the number of years of the user's work experience. (Note 16) The apparatus according to Note 11, wherein the operation further includes the step of providing a second set of criteria to the large language model, the large language model being configured to output definitions of the selected words based on the second set of criteria. (Note 17) The apparatus according to Note 11, wherein the audio data is generated during a communication session between the apparatus and another apparatus, and the operation further includes the step of acquiring the audio data before broadcasting the audio data. (Appendix 18) The apparatus described in Appendix 17, wherein the selected word and the definition of the selected word are presented in real time during the communication session in connection with the broadcast of a portion of the audio data containing the selected word. (Note 19) The communication session is a video conference including multiple devices, and the selected word presented by the device is different from a first word and the definition of the first word presented by another device participating in the video conference, as described in Note 17. (Note 20) Steps to obtain text data containing multiple words, A step of providing the text data and a first reference set to an artificial intelligence system, wherein the artificial intelligence system is configured to analyze the text data based on the first reference set, select words from the text data, and generate definitions for the selected words. The steps include obtaining the selected word and the definition of the selected word from the artificial intelligence system, The steps include presenting the text data, the selected word, and the definition of the selected word on the device's display, A method that includes this. [Explanation of Symbols]

[0124] 100 Environment 110 Network 120 User devices 130 Equipment 140 Automatic Speech Recognition Systems 150 Large-Scale Language Models

Claims

1. A step of obtaining transcript data generated from audio data including speech via automatic speech recognition technology, wherein the transcript data includes one or more words of the transcription of the speech in the audio data. A step of providing the transcript data and a first set of criteria to a large-scale language model, wherein the large-scale language model is configured to analyze the transcript data based on the first set of criteria and select words from the transcript data. The steps include obtaining the selected word and the definition of the selected word generated by the large-scale language model, The device broadcasts the audio data, Along with broadcasting the audio data, the device displays the selected word and its definition on its display. A method that includes this.

2. The method according to claim 1, further comprising the step of obtaining an instruction to present the selected word and the definition of the selected word on the display, wherein the selected word and the definition of the selected word are presented in response to obtaining the instruction to present the selected word and the definition of the selected word on the display.

3. The method according to claim 2, further comprising the step of providing the large language model with a second set of criteria, wherein the large language model is configured to determine, based on the second set of criteria, whether to present the selected words and definitions of the selected words on the display, and to provide the device with the instructions.

4. The method according to claim 3, wherein the second reference set includes the user attributes of the device.

5. The method according to claim 4, wherein the user's attributes include one or more of the following: the technical field in which the user is employed, the business organization associated with the user, the user's level of education, the user's job duties, and the number of years of work experience of the user.

6. Steps include obtaining user feedback on the selected words and definitions presented, A step of updating the second set of criteria based on the user feedback, wherein the updated second set of criteria is provided to the large-scale language model in a future communication session. The method according to claim 5, further comprising:

7. The method according to claim 1, further comprising the step of providing a second set of criteria to the large language model, wherein the large language model is configured to output definitions of the selected words based on the second set of criteria.

8. The method according to claim 1, wherein the audio data is generated during a communication session between the device and another device, and the method further comprises the step of acquiring the audio data in the device before broadcasting the audio data.

9. The method according to claim 7, wherein the selected word and the definition of the selected word are presented in real time during a communication session in connection with the broadcast of a portion of the audio data containing the selected word.

10. One or more non-temporary computer-readable media configured to store instructions, wherein, when executed, the instructions perform the method according to claim 1.

11. It is a device, One or more non-temporary computer-readable media configured to store instructions, A processor coupled to the computer-readable medium and configured to execute the instructions and perform an operation, The operation includes, A step of obtaining transcript data generated from audio data including speech via automatic speech recognition technology, wherein the transcript data includes one or more words of the transcription of the speech in the audio data. A step of providing the transcript data and a first set of criteria to a large-scale language model, wherein the large-scale language model is configured to analyze the transcript data based on the first set of criteria and select words from the transcript data. The steps include obtaining the selected word and the definition of the selected word generated by the large-scale language model, The device broadcasts the audio data, Along with broadcasting the audio data, the device displays the selected word and its definition on its display. A device including a device.

12. The aforementioned operation is, The apparatus according to claim 11, further comprising the step of obtaining an instruction to present the selected word and the definition of the selected word on the display, wherein the selected word and the definition of the selected word are presented in response to obtaining the instruction to present the selected word and the definition of the selected word on the display.

13. The apparatus according to claim 12, wherein the operation further includes the step of providing a second set of criteria to the large language model, the large language model is configured to determine, based on the second set of criteria, whether to present the selected word and the definition of the selected word on the display, and to provide the instruction to the apparatus.

14. The apparatus according to claim 13, wherein the second reference set includes the user attributes of the apparatus.

15. The apparatus according to claim 14, wherein the user's attributes include one or more of the following: the technical field in which the user is employed, the business organization associated with the user, the user's level of education, the user's job duties, and the number of years of work experience of the user.

16. The apparatus according to claim 11, wherein the operation further includes the step of providing a second set of criteria to the large language model, the large language model being configured to output definitions of the selected words based on the second set of criteria.

17. The apparatus according to claim 11, wherein the audio data is generated during a communication session between the apparatus and another apparatus, and the operation further includes the step of acquiring the audio data before broadcasting the audio data.

18. The apparatus according to claim 17, wherein the selected word and the definition of the selected word are presented in real time during the communication session in connection with the broadcast of a portion of the audio data containing the selected word.

19. The apparatus according to claim 17, wherein the communication session is a video conference including multiple devices, and the selected word presented by the device is different from a first word and the definition of the first word presented by another device participating in the video conference.

20. Steps to obtain text data containing multiple words, A step of providing the text data and a first reference set to an artificial intelligence system, wherein the artificial intelligence system is configured to analyze the text data based on the first reference set, select words from the text data, and generate definitions for the selected words. The steps include obtaining the selected word and the definition of the selected word from the artificial intelligence system, The steps include presenting the text data, the selected word, and the definition of the selected word on the device's display, A method that includes this.

Images