Generated content for communication support

Abstract

Methods and systems for using generated content to improve an individual's ability to communicate using electronically assisted communication.

Description

Technical Field

[0001] Cross - reference to related applications This application is a non - provisional application of U.S. Provisional Application No. 63 / 506,154, filed on Jun. 5, 2023, the entire content of which is incorporated herein by reference.

[0002] Methods and systems for using generated content to improve the ability of individuals to communicate using electronically - assisted communication.

Background Art

[0003] With the availability of brain - computer interfaces (BCIs), users can provide instructions to the BCI using endogenous signals, exogenous signals, or combinations of signals. The use of such BCIs can improve an individual's ability to regain lost independence, including the ability to interact with their environment with increased autonomy. Typically, endogenous signals are signals that an individual generates internally. Such endogenous signals can include neural signals detected by sensors that measure electrical pulses generated when an individual generates thoughts or moves their muscles (either through actual movement or imagined movement in the case of a paralyzed individual). Exogenous signals can include any signal that measures an action external to an individual or an action taken outside of the individual. For example, exogenous signals can include signals generated when an individual triggers an external mechanism or electronic device (e.g., mouse click, screen contact / tap, keyboard click, voice command, etc.), signals received by inertial sensors that use inertia to detect physical movement of an individual's body parts. Signals are received using camera - type devices (e.g., eye movement detectors, body movement detectors, etc.) that detect movement of an individual's body parts, breath - operated controls (typically used in wheelchairs), etc.

[0004] Individuals with complete or partial paralysis, disability, or other muscular disorders tend to have limitations on the amount of information they can communicate per unit of time. While assistive technologies such as eye trackers and BCIs can facilitate communication, the speed of communication is typically slow with conventional systems. For example, when an individual with a disability tries to spell a sentence, they typically do so by spelling each letter in the sentence at a slow pace, which can be a laborious process if the number of inputs they can generate to interact with a BCI is limited.

[0005] Many conventional BCI systems are limited because signal generation is typically restricted to a few commands or even a single command to control the BCI system, which can require considerable effort from the user. Therefore, conventional BCI systems often require users to interact extensively with the system to communicate with others. Such increased interaction can lead to user fatigue and ultimately limit user autonomy. There remains a need to improve the ability of BCI users to communicate using BCI systems in a way that reduces user effort while enhancing the personalization of the system for the user. [Overview of the Initiative] [Means for solving the problem]

[0006] This disclosure includes several systems and methods for assisting individuals when using electronically assisted communication. Such assistance can improve the speed of communication, reduce user fatigue, and / or provide personalization so that electronically assisted communication represents the user more closely than conventional electronic speech generators.

[0007] Modifications of the present disclosure include methods for assisting individuals engaged in electronically assisted communication, one modification of the method comprising the steps of: obtaining an input prompt (or input communication / message) using an electronically assistive device; evaluating the input prompt using a computer processor to identify a topic category associated with the input prompt; generating multiple content responses using the topic category such that multiple content responses are associated / responded as conversational with the topic category, each content response from the multiple content responses being further associated with a sentiment identifier selected from multiple sentiment identifiers; electronically displaying the multiple content responses on a user interface display, where a first sentiment identifier associated with a first content response is visually distinguishable on the display from a second sentiment identifier associated with a second content response, and the user interface is configured to allow the individual to select at least one of the multiple content responses as a selected content response; and electronically communicating the selected content response using an electronically assistive device. In alternative modifications, the method steps described above do not require the step of evaluating the input prompt to identify a topic category associated with the input prompt. In such cases, the generation of multiple content responses is carried out so that the multiple content responses are associated with / responded to the input prompt as a conversation.

[0008] Modifications of this disclosure include brain-computer interfaces and electronic assistive devices, which are portable digital devices (such as electronic tablets, smartphones, personal devices, smartwatches, virtual reality headsets / eyewear, and / or computers).

[0009] Generating multiple content responses may involve using a large, network-enabled language model that can operate on electronic assistive devices. Network connectivity may include access via electronic storage, local area networks, virtual networks, cloud-based networks, or any network used for electronic communications.

[0010] Modifications of this disclosure may include accessing a database containing multiple pieces of informational data unique to an individual, and generating multiple content responses may include generating at least one content response containing at least one piece of informational data unique to an individual.

[0011] Individual-specific data includes data selected from groups consisting of demographic information, personal preferences, biographical information, family information, health information, calendar information, treatment information, and historical information.

[0012] Modifications of this disclosure include actively obtaining input prompts by inputting a recording into an electronically assistive device.

[0013] Modifications of this disclosure include methods in which input prompts are acquired passively, such as when a microphone detects a conversation and the system identifies a portion of the conversation as an input prompt.

[0014] Modifications of this disclosure include methods in which a selected content response is communicated electronically using an electronically assistive device, and the steps include displaying the selected content response electronically.

[0015] This disclosure also includes the step of electronically communicating selected content responses using an electronically assistive device. This may include the step of generating an audible voice that reads the selected content aloud. In some cases, the individual has the ability to customize the selected content responses using a user interface before electronically communicating the selected content.

[0016] In another variation, a method for assisting an individual engaged in electronically assisted communication includes the steps of: obtaining an input prompt / communication using an electronically assistive device; generating a plurality of emotion identifiers using the input prompt; electronically displaying the plurality of emotion identifiers on a user interface display, the user interface being configured to allow the individual to select at least one of the plurality of emotion identifiers as a selected emotion identifier; electronically displaying a selected content response associated with the selected emotion identifier on the user interface; and electronically communicating the selected content response using an electronically assistive device. A variation of the method includes the step of initially displaying the plurality of emotion identifiers without any text of the content response. An additional variation includes the step of displaying a single visual identifier (e.g., emoticon, text, color, shading, etc.) associated with each emotion identifier. As stated above, the method is not limited to displaying emotion identifiers; any identifier category may be displayed.

[0017] In another variation, electronically displaying selected content responses associated with a selected sentiment identifier on a user interface includes generating multiple content responses associated with a selected sentiment identifier and displaying the multiple content responses on the user interface, the user interface being configured to allow the user to select a content response from the multiple content responses.

[0018] In an additional variation, evaluating the input prompt to identify the topic category associated with it involves using a computer processor so that multiple content responses are associated with topic categories as a conversation, and each content response from the multiple content responses is further associated with a selected sentiment identifier.

[0019] A variation of the method includes electronically displaying the multiple emotion identifiers on a user interface display such that each of the multiple emotion identifiers is visually distinguishable from the rest of the multiple emotion identifiers.

[0020] Modifications of the present disclosure include an electronic system for assisting an individual engaged in electronically assisted communication, the electronic system including a memory unit; a recording device configured to transmit input prompts to the memory unit; a processor configured to evaluate the input prompts, access the memory unit to identify topic categories associated with the input prompts, and generate a plurality of content responses using the topic categories such that a plurality of content responses are associated with the topic categories as a conversation; and a user interface having an electronic display configured to display the plurality of content responses, the processor further configured to provide the electronic display with a first sentiment identifier associated with a first content response that is visually distinguishable on the electronic display from a second sentiment identifier associated with a second content response, the user interface further configured to enable the individual to select / electronically select at least one of the plurality of content responses as a selected content response, the processor further configured to electronically communicate the selected content response.

[0021] Some variations of this disclosure include an electronic system in which a processor is operably networked to a large language model to generate multiple content responses.

[0022] A modification of the present disclosure includes an electronic system, wherein a processor is operably network-connected to a database containing a plurality of information data unique to an individual, and generating a plurality of content responses includes generating at least one content response including at least one information data unique to the individual.

[0023] A modification of the present disclosure includes an electronic system, wherein a recording device is configured to actively have a recording state for actively recording an input prompt.

[0024] A modification of the present disclosure includes an electronic system, wherein a recording device is configured to actively have a recording state for passively recording an input prompt.

[0025] A modification of the present disclosure includes an electronic system, wherein a processor is configured to electronically communicate a selected content response by sending the selected content response to an external display device.

[0026] A modification of the present disclosure includes an electronic system, wherein a processor is configured to electronically communicate a selected content response by displaying the selected content response on an electronic display.

[0027] A modification of the present disclosure includes an electronic system, further including a speaker such that a processor causes an audible voice of a selected content to be generated.

[0028] A modification of the present disclosure includes a method for assisting an individual engaged in electronic-assisted communication, the method including prompting the individual to enter an input at an input prompt in an electronic assistance device, accessing a database containing personal information data about the individual, generating at least one content phrase using the personal information data, and electronically displaying at least one content phrase on a display.

[0029] A variation of the present disclosure includes a method, and the step of electronically displaying at least one content phrase on a display of a user interface requires a personal action.

[0030] A variation of the present disclosure includes a method, and the step of electronically displaying at least one content phrase on a display of a user interface is performed when an individual begins to enter an input.

[0031] A variation of the present disclosure includes a method, and the step of electronically communicating a selected content response using an electronic support device includes the step of electronically displaying the selected content response.

[0032] A variation of the present disclosure includes a method, and electronically communicating a selected content response using an electronic support device includes the step of generating an audible voice that reads the selected content.

[0033] A variation of the present disclosure includes a method, and includes the step of providing an individual with the ability to customize a selected content response using a user interface before electronically communicating the selected content.

[0034] Modifications of the present disclosure include an electronic system for assisting an individual engaged in electronically assisted communication, the system comprising: a processor unit coupled to a memory unit; a program module operably networked with an image generator, the program module configured to receive a communication command prompt and generate a custom pictogram tile having a unique image associated with the communication command prompt, the program module configured to store the communication command prompt and the custom pictogram tile in the memory unit; and a user interface configured to allow an individual to visually observe a plurality of custom pictogram tiles, the user interface further configured to allow an individual to select at least one of the plurality of custom pictogram tiles as the selected pictogram tile in order to electronically communicate a communication command prompt associated with the selected pictogram tile.

[0035] Modifications of this disclosure include an electronic system in which a program module is operably networked to a database containing personal information data relating to an individual, and generates custom pictogram tiles from the database.

[0036] Modifications of the present disclosure include a method for assisting an individual engaged in electronically assisted communication, the method comprising the steps of receiving a communication command prompt, generating a custom pictogram tile having a unique image associated with the communication command prompt, and a user interface configured to allow the individual to visually observe a plurality of custom pictogram tiles, the user interface further configured to allow the individual to select at least one of the plurality of custom pictogram tiles as the selected pictogram tile in order to electronically communicate a communication command prompt associated with the selected pictogram tile.

[0037] Modifications of this disclosure include methods, in which the step of generating a custom pictogram tile includes the step of using information from a database that includes personal data relating to an individual. [Brief explanation of the drawing]

[0038] [Figure 1A] This diagram shows how an individual accesses a user interface via BCI (Business Interface). [Figure 1B] This diagram shows how an individual directly accesses the user interface. [Figure 2] This is a schematic representation of an example of an electronic assistance device that utilizes generated content. [Figure 3A] This figure shows an example of a user interface that supports individuals engaged in electronically assisted communication. [Figure 3B] This figure shows a user interface similar to that shown in Figure 3A, with additional visual icons or text to distinguish identifiers associated with each content response. [Figure 3C] This figure shows another variation of the user interface, where multiple identifiers are displayed first without any accompanying content message. [Figure 3D] This figure shows another variation similar to Figure 3C, where the selection of emotion identifiers generates a wide range of content responses associated with the selected emotion identifiers. [Figure 4] This figure shows a sample prompt structure supplied to the LLM for user interface generation. [Figure 5A] This figure shows another example of methods and systems for supporting individuals engaging in electronically assisted communication using minimal text input. [Figure 5B] This figure shows a sample prompt structure supplied to the LLM for user interface generation. [Figure 6] This diagram shows a user interface that allows individuals to customize their selected content responses. [Figure 7A] This figure shows an example of what happens when the term "foot pillow" is entered into the search field on the ARASAAC website. [Figure 7B] This figure shows some AI-generated pictograms that can be used with the AAC interface. [Figure 7C] This figure shows some AI-generated pictograms that can be used with the AAC interface. [Figure 8A] This figure shows a modified user interface that displays several content responses generated in response to an input command. [Figure 8B] This figure shows a modified user interface that displays several content responses generated in response to an input command. [Modes for carrying out the invention]

[0039] The following concerns the use of generated content to improve electronic communication support for individuals suffering from complete or partial paralysis, disability, or other muscular disorders that impair traditional oral communication.

[0040] Generative artificial intelligence (AI), specifically language models (LLMs), continues to improve its capabilities. These models demonstrate a powerful understanding of language in context, structure, and semantics.

[0041] The systems and methods described herein leverage the use of LLM to generate faster communication and typing for people using electronically assistive communication devices or systems than would otherwise be possible. While the examples discussed herein include users of BCI systems, the generated content can be applied to any electronically assistive devices, such as portable digital devices (e.g., electronic tablets, smartphones, etc.) and computers, or other similar electronic human-computer interface devices that support individuals using electronically assistive communication. Electronically Assistive Device 100

[0042] Figure 1A shows an example of an individual 10 using a BCI system that includes one or more electrodes 20 for detecting nerve signals from the individual 10 transmitted at 26 by one or more components 22, 24, for ultimately connecting at 28 to an electronic assistive device 100 having a user interface. Meanwhile, Figure 1B shows the electrodes 20 implanted within the individual 10. The concept of generated content can be applied to any type of BCI, including surgically implanted electrodes located outside the body. In addition, as shown in Figure 1B, the concepts disclosed herein can be accessed directly by the individual 10 on the user interface, either in combination with the BCI or separately from the BCI.

[0043] Figure 2 is an exemplary representation of the schematic diagram 50 of an electronic assistive device 100 employing the content generation methods and systems described herein. The electronic assistive device 100 can function as a communication device or as a control device interacting with alternative electronic devices. For example, a user can have the electronic assistive device 100 generate communication as an audible voice message. Alternatively, or in combination, the electronic assistive device 100 can be coupled to another electronic device (e.g., the device described above) to electronically deliver communication by either text or audible voice message.

[0044] Figure 2 shows an exemplary system 50 having a processor 52, hardware 54, memory 56, and an optional network interface 60. The hardware 54 may include a microphone, speaker, display unit, and other items common to electronic human interface devices or electronic assistive devices, as described above. The system may include one or more generating content applications 58 that generate messages in response to input, as described below. The system may also optionally include databases 60, 62 stored locally on the system. Alternatively, or in combination, the system may rely on a network connection 70 to access a remote server 72 to access additional databases 74, 76.

[0045] Figure 3A shows an example of a user interface 102 on an electronic device 100 that assists an individual engaging in electronically assisted communication. As shown, the display may include identifier data 120. As stated above, the methods and systems of this disclosure are intended to assist an individual user in engaging in a conversation that mimics a conversation that is natural to the user. Therefore, it is desirable to optionally display identifier data 120 to confirm that the user's session is being conducted using data or other preferences associated with that user.

[0046] The user interface 102 can also display an input prompt 122 containing a message used to instruct the generation of a conversational response. The input prompt 122 may be a message received by an individual, a topic of conversation entered by an individual, or a topic generated by the system in anticipation of the user's needs, taking into account the time, date, place, medication needs, etc. In some variations, the input prompt message may be acquired when the system records oral communication (e.g., via a microphone) to capture a message as an input prompt from the user or another individual. Alternatively, the system may include an active state that constantly records audio to recognize a phrase or word as an input prompt message. Meanwhile, Figure 3A shows the input prompt 122 displayed on the user interface 102. In alternative variations, it is not necessary for such an input prompt 122 to be visually displayed.

[0047] When a message is received as an input prompt, the user interface 102 displays any number of content responses 140–148 generated by the system. Each content response is the message to be communicated, placed on each of the tiles 140–148. While nine tiles are shown in the illustration, the system can display more or fewer tiles as needed. Further discussion regarding the generation of content responses follows. In this variation, the user interface 102 displays the content responses 140–148, as well as other operational responses 130–132, in a grid arrangement of tiles. As described below, the entities of the content responses 140–148 are associated with the input prompt message 122 as a conversation. This means that the content responses 140–148 present a wide range of options for the individual to provide responses related to / associated with the topic of the input message. It should be noted that the methods and additional variations of the system described herein do not require a grid arrangement configuration. However, variations of the grid arrangement can be an efficient mode of presenting information to the user.

[0048] As described below, the system / method not only generates content responses 140-148 as relating to the topic of the input message and as a conversation, but also classifies the tone of the generated content responses and assigns an identifier to each content message based on the associated category. For example, an ordinary conversation typically involves dialogue and responses with a variety of emotions. In this invention, the user is provided with a visual identification of the content response messages, along with the identifier associated with the response. As shown in Figure 3A, each content response 140-148 includes a visually distinguishable pattern, mark, color, or other visually identifiable feature so that the user can quickly observe the identifier associated with the content message. It should be noted that some content responses may have duplicate identifiers. In an alternative modification, the system may provide content responses in which each response has a unique identifier.

[0049] Typically, an identifier is an adjective that describes the generated content response. Such adjectives may include the tone of the response and the emotions associated with or conveyed by the response. Alternatively, an identifier may include the degree of familiarity, redundancy, technical terminology, formality, or any other description that is typically associated with a wide range of responses on the same topic.

[0050] As an example, Figure 3A shows an input message at 122, which asks about someone visiting an individual. For example, a potential topic category related to this input message is whether the individual wants a visitor soon. The system then generates several content responses that respond to or are associated with this topic category. The system can generate responses using various sentiment identifiers (or other identifiers as described above). Alternatively, the system can generate a response and then determine the sentiment (or other) identifier. The system then displays these content messages 140-148 so that, apart from the message text, each content response with its respective identifier is visually distinguishable from content responses with different identifiers. In the illustration in Figure 3A, each content response tile 140-148 contains a message that is associated / responds as a conversation to whether the individual wants a visitor.

[0051] Figure 3A also shows a user interface 102 that includes a cursor 106 that allows the user to interact with the user interface to select one or more tiles from tiles 130-132 and 140-148. For example, the cursor 106 can be controlled using an eye tracker, BCI signals, a mouse, or other input device. Alternatively, or in combination, a user who can physically interact with the user interface can simply select any tile. In the illustrated example, the user selects tile 141 with the cursor 106, so that the system recognizes the user's desired selection as the selected content response 141. The selected content response message 124 may optionally be displayed on the user interface 102. In some variations, the system then communicates the selected content response electronically using device 100 or the system. Electronic communication may include providing the message using generated voice, recorded voice, sending data representing the message to another device, or a combination thereof. In additional variations, the user may choose to modify the message before it is communicated electronically.

[0052] Figure 3B shows a user interface 102 similar to that shown in Figure 3A, but with additional visual icons or text to distinguish the identifiers associated with each content response. In the modified version shown, each tile contains a symbolic representation of identifiers 150–158. The drawing shows a set of emoticons, but any symbolic representation of an identifier can be used for visual distinction. The drawing also shows text descriptors for identifiers 160–168. Modified versions of user interface 102 may include any combination of symbolic identifiers, text identifiers, and / or visually distinguishable patterns, marks, colors, or other visually identifiable features for the tiles. As shown, tile 141 represents the content response "Let's definitely plan!" with its associated symbolic identifier 151 and text identifier 161, along with cross-hatching to associate the content response with the emotion identifier "Excited".

[0053] Figure 3C shows another variation of the user interface 102 described above. However, in this variation, the interface generates only a set of identifiers. The identifiers may be emoticons 150-158 and / or text identifiers 160-168 without any accompanying messages. For example, if the identifiers are emotion identifiers, the interface 102 provides the user with a variety of emotion identifiers 150-158 and / or 160-168 before any content response is displayed. This approach can mimic a true conversation to an input prompt by allowing an individual to express an emotion in response to an input prompt 122 and then determine the text of the content response. In one variation, the content response for a tile may be communicated electronically once the individual selects each tile.

[0054] Figure 3D shows yet another variation similar to Figure 3C, where the selection of a sentiment identifier generates a wide range of content responses associated with the selected sentiment identifier. This provides the user with a more diverse range of responses associated with the selected sentiment identifier. The user then selects a tile to electronically communicate the content responses associated with that tile. The variations discussed with respect to Figures 3C and 3D are not limited to sentiment identifiers. Instead, they could be any identifier as discussed herein.

[0055] In one variation of the present invention, an LLM model can be used to generate content responses. In a previous example, OpenAI's GPT4 Turbo was used to generate information to be provided on a user interface. However, any LLM model can be used to generate the user interface. Figure 4 shows a sample prompt structure supplied to LLM170 for generating a user interface. LLM170 is given instructions along with one or more facts that are relevant to and / or specific to the user. As mentioned above, this helps the LLM output to provide content responses that are natural to the user in order to ensure more personalized communication. In one example, background 171 is provided as follows: "You are acting as a middle-aged man named John. You have early-stage ALS, you can speak, and you live in New York with your wife and dog. You love golf and are passionate about politics." The LLM then generates a response to the input message based on the instructions to achieve the desired tone and format. It should be noted that LLM can also be used to generate visual distinguishing features for tiles (e.g., color, shading, icons, etc.).

[0056] The action 172 of generating a response to an input message may include generating possible responses for X9 to the input message by using a mixture of the most likely affirmative, positive, neutral, hopeful, dissatisfied, and negative responses. This requires the LLM to give the user choices by providing a list of adjectives that describe the tone of the responses in order to ensure a variety of responses.

[0057] Next, you can instruct the LLM to make the message more human in 173 by revising each answer to ensure the tone is conversational, 100% simple, avoids corporate jargon, mimics natural human dialogue and utterances, ensures sentence lengths are uneven, and that no single answer exceeds 10 words. Finally, shorten each answer. This part of the prompt is a series of steps to make the answers more realistic in relation to what a user might say.

[0058] Once content responses are generated in 172 and made human-like in 173, the information is formatted for display in 174. For example, one could select a hexadecimal color value for each response and instruct the LLM to ensure that the color is one of the following: #3146F5, #7437F9, #DE1C66, #00777E, #C94921, #438550, #C32CC0. Formatting display 174 can also assign emoticons and / or textual descriptions based on the LLM's inference of the emotional state of the generated content responses.

[0059] Next, the generated content responses and related information are supplied to the user interface, which can selectively display the responses selected from the LLM.

[0060] Figure 5A shows another example of a method and system for assisting individuals engaged in electronically assisted communication. As shown, the user interface 202 includes a QWERTY keyboard 204. The keyboard 204 may be a full electronic keyboard or, as shown, a lean keyboard with a minimal number of keys. As described above, the user interface 202 may include a cursor 206 driven by a BCI or eye-tracking device for selecting keys to generate a message. Alternatively, the user can directly interact with the user interface 202 to type a message. The user types the message input into an input prompt 210. The system allows the user to make a best attempt to type the intended message into the input prompt 210. The LLM can then retrieve the string in the input prompt and be prompted to guess what the user was trying to say based on a number of personal data factors related to the user. Such personal data may include the user's writing style, proper nouns the user might use, calendar information, demographic information, personal profile, likes and dislikes, health status, calendar, and, in a patient context, recent appointment schedules, treatment schedules, and current primary hospital information.

[0061] One advantage of this approach is that users have the option to skip vowels and punctuation, leave everything in lowercase, and use backspace very little. This replaces the traditional autocorrect word sequences, eliminating the need for users to click additional buttons while typing and improving the speed of communication. This reduces the number of keys needed on the keyboard (exclamation marks, caps lock, etc.), and as a result, the time required to type a message is reduced.

[0062] Next, the user interface provides an output 210 for electronically displaying at least one content phrase, separate from the input prompt. This separation allows the user to focus on the input message within the input prompt 210 without having to accept suggested or predicted words, as is common in conventional predictive typing models. The user interface is then configured to allow the individual to select at least one content phrase as the selected content response phrase, which can then be electronically communicated using an electronically assistive device, as described above.

[0063] Figure 5B shows a sample prompt structure supplied to the LLM 170 for user interface generation. As shown, the LLM uses background information 231 about the individual to generate inferences 232 about the meaning of the input message. The resulting inferences are supplied to the user interface 233. In one example, the prompt supplied to the LLM included: "Act as a copywriter and correct the grammar and capitalization of a phrase typed by a disabled person using an AAC device. If it is already a correct and complete sentence, simply repeat the phrase I have given you without any additional text. If it is not a correct English sentence, make your best guess as to what the person is trying to say and add words to ensure it is a grammatically correct and complete sentence. If you cannot guess, simply repeat the phrase I have given you. Do not add any additional text. The phrase is the input phrase."

[0064] Figure 6 shows a screen on the user interface 240 that allows an individual 120 to customize a selected content response 124. This gives the user the option to use various parameters 241 to match the message 124 to their own way of speaking or writing to the recipient. In this example, they can increase or decrease its length, make it more formal or casual, and edit the message to type instead. This customization can be used for any of the generated content responses discussed herein.

[0065] Another example of generated content concerns the use of pictograms for use in Assistive Alternative Communication System (AAC) software and interfaces. Typically, pictograms in AAC-based interfaces are provided from vast libraries of graphic art, such as ARASAAC (https: / / arasaac.org / aac / en). The number of symbols in these collections often exceeds 10,000 images. However, while they are considered comprehensive, they do not cover all conceivable needs. For example, Figure 7A shows an example from the ARASAAC site when the term "foot pillow" is entered into the search field. The various pictograms proposed as a result show different foot symbols and pillow symbols separately, but not within the same image.

[0066] Another problem with current AAC interfaces is that they require proper setup to effectively assist the individual using the system. Typically, the caregiver setting up the system must spend considerable time matching images with items / commands. Often, manually selected pictograms do not accurately represent the items they represent. This can often prevent the AAC interface from becoming a functional communication assistant for the user. Generated content systems address these challenges by using an AI image generator (e.g., DALL-E) in the user interface to create custom pictogram images that more accurately represent the items at hand. For example, Figure 7B shows a series of AI-generated foot pillows created by the system. An example of a template prompt is shown below in parentheses, where the phrase between the quotes is the word or phrase that appears on the tile.

[0067] Prompt: {Create a color icon "symbol" in the pictogram style used by AAC software. Ensure the background is white. Return only the symbol and do not add any additional outlines.}

[0068] To ensure that the created icons are consistent with one another, the image generator can be instructed to conform to the style of a reference image. As shown in Figure 7C, the generated pictogram 252 includes a "head pillow" symbol that uses a previously generated "foot pillow" symbol as a reference.

[0069] As referred to herein, the LLM used in the modifications of the systems and methods described herein is configured to incorporate the needs and circumstances of each user. In one embodiment, the LLM uses demographic information, personal profiles, likes and dislikes, health status, and calendar, which in the user's context may be the schedule of the most recent medical appointments, treatment schedules, and the current primary hospital. This personal information is integrated into the LLM by prompting and / or priming via plugging in, creating multiple streams of personal context for grooming the LLM in real time and asynchronously as the information arises, taking into account text generation in response to live input.

[0070] The integration of user personal information with LLM allows for the creation of more relevant content during communication. For example, a user can select a word or set of words as input commands for elements of the sentence they want to create (subject, object, etc.). These words can be suggested by LLM considering the context of the current sentence and the entire chat history, or they can be manually entered by the user. For example, in Figures 8A and 8B, the input commands shown in user interface 102 in Figures 8A and 8B are "Melbourne Football Club" and "personal hygiene," respectively.

[0071] Next, the LLM generates possible generated content responses 181-185 and 191-195 from the input command 122. The user interface 102 is configured to allow the user to select one of the possible generated content responses 181-185 and 191-195 generated from the user's personal information. These basic examples make it possible for a user to have a wide range of content responses 181-185 and 191-195 tailored to that particular user. As shown in the figure, each response may be visually distinguishable so that the user can quickly select the desired content response for electronic communication. As shown in the figure, the user interface 102 may include any number of action tiles 186 for navigation through the user interface.

[0072] Further details of the present invention may involve materials and manufacturing techniques that are within the scope of the skill of the art. The same may apply to method-based embodiments of the present invention relating to additional actions that are generally or logically adopted. In addition, although the present invention has been described with reference to several examples incorporating various features as options, the present invention is not limited to those described or intended with respect to each modification of the present invention.

[0073] Various modifications may be made to the invention as described herein, and equivalents (whether described herein or not for brevity) may be substituted without departing from the true spirit and scope of the invention. Furthermore, any optional features of modifications of the invention may be described and claimed independently or in combination with any one or more of the features described herein. Thus, the invention intends, where possible, to be combinations of various aspects of embodiments or combinations of embodiments themselves. References to singular items include the possibility that there may be multiple instances of the same item. More specifically, as used herein and in the appended claims, unless the context clearly indicates otherwise, singular “a,” “an,” “said,” and “the” include multiple references.

[0074] It is important to note that, where possible, various aspects of the described embodiments, or the embodiments themselves, can be combined. Such combinations are intended to be within the scope of this disclosure. [Explanation of Symbols]

[0075] 10 individuals 20 electrodes 22 components 24 components 50 Schematic Diagram 50 Systems 52 processors 54 Hardware 56 memory 58. Content Generation Application 60 network interfaces 60 databases 62 Databases 70 Network Connections 72 Remote Servers 74 Databases 76 Databases 100 Electronic Assistive Devices 102 User Interface 106 Cursor 120 Identifier Data 122 Input Prompt 124 Content Response Messages 130-132 tiles 140-148 Content Answers 150-158 Emoticons 160-168 Text Identifiers 170 LLM 171 Background 174 Format Display 186 Action Tiles 202 User Interface 204 QWERTY keyboard 206 Cursor 210 Input Prompt 231 Background information 232 Speculation 240 User Interfaces 241 parameters 250 AI-generated foot pillows 252 pictograms

Claims

1. A method for supporting individuals engaged in electronically supported communication, wherein the method is Steps include obtaining input prompts using an electronic assistive device, A step of using a computer processor to evaluate the input prompt in order to identify the topic category associated with the input prompt, A step of generating a plurality of content responses using the topic category such that the plurality of content responses are associated with the topic category as a conversation, wherein each content response from the plurality of content responses is further associated with a sentiment identifier selected from a plurality of sentiment identifiers, The step is to electronically display the plurality of content responses on the user interface display, wherein the first sentiment identifier associated with the first content response is visually distinguishable on the display from the second sentiment identifier associated with the second content response. The user interface is configured to allow the individual to select at least one of the multiple content responses as the selected content response, comprising the steps of displaying, The steps include: electronically communicating the selected content response using the electronic assistance device; A method that includes this.

2. The method according to claim 1, wherein the electronic support device comprises a brain-computer interface.

3. The method according to claim 1, wherein the electronic support device comprises a portable digital device.

4. The method according to claim 1, wherein the electronic support device comprises a computer system.

5. The method according to claim 1, wherein the step of generating the plurality of content responses includes the step of using a large language model that is operably networked to the electronic assistive device.

6. The method according to claim 1, further comprising the step of accessing a database containing a plurality of informational data specific to the individual, wherein the step of generating the plurality of content responses includes the step of generating at least one content response containing at least one informational data specific to the individual.

7. The method according to claim 6, wherein the plurality of information data unique to the individual includes data selected from the group consisting of demographic information, personal preferences, biographical information, family information, health information, calendar information, treatment information, and historical information.

8. The method according to claim 1, wherein the step of obtaining the input prompt is performed actively by inputting a recording to the electronic assistive device.

9. The method according to claim 1, wherein the step of obtaining the input prompt is performed passively.

10. The method according to claim 1, wherein the step of electronically communicating the selected content response using the electronic support device includes the step of electronically displaying the selected content response.

11. The method according to claim 1, wherein the step of electronically communicating the selected content response using the electronic assistance device includes the step of generating an audible voice that reads the selected content aloud.

12. The method according to claim 1, further comprising the step of providing the individual with the ability to customize the selected content response using the user interface before electronically communicating the selected content.

13. A method for supporting individuals engaged in electronically supported communication, wherein the method is Steps include obtaining input prompts using an electronic assistive device, The steps include generating multiple emotion identifiers using the aforementioned input prompts, A step of electronically displaying the plurality of emotion identifiers on the display of a user interface, wherein the user interface is configured to allow the individual to select at least one of the plurality of emotion identifiers as a selected emotion identifier; The steps include: electronically displaying the selected content response associated with the selected emotion identifier on the user interface; The steps include: electronically communicating the selected content response using the electronic assistance device; A method that includes this.

14. The method according to claim 13, wherein the step of electronically displaying the selected content response associated with the selected emotion identifier on the user interface includes the steps of generating a plurality of content responses associated with the selected emotion identifier and displaying the plurality of content responses on the user interface, the user interface being configured to enable the user to select the selected content response from the plurality of content responses.

15. The method of claim 13, further comprising the step of using a computer processor to evaluate the input prompt to identify a topic category associated with the input prompt, such that the plurality of content responses are associated with the topic category as a conversation, and each content response from the plurality of content responses is further associated with the selected sentiment identifier.

16. The method according to claim 13, further comprising the step of electronically displaying the plurality of emotion identifiers on the display of the user interface such that each of the plurality of emotion identifiers is visually distinguishable from the rest of the plurality of emotion identifiers.

17. An electronic system for supporting individuals engaged in electronically supported communication, wherein the electronic system is Memory unit and A recording device configured to transmit input prompts to the memory unit, A processor configured to evaluate the input prompt, access the memory unit to identify the topic category associated with the input prompt, and generate the multiple content responses using the topic category such that the multiple content responses are associated with the topic category as a conversation, wherein each content response from the multiple content responses is further associated with a sentiment identifier selected from a plurality of sentiment identifiers, A user interface having an electronic display configured to display the plurality of content responses, wherein the processor is further configured to provide the electronic display with a first emotion identifier associated with a first content response that is visually distinguishable on the electronic display from a second emotion identifier associated with a second content response. The user interface is further configured to allow the individual to select at least one of the multiple content responses as the selected content response. The processor is further configured to electronically communicate the selected content response. Electronic systems.

18. The electronic system according to claim 17, wherein the processor is operably networked to a large language model for generating the plurality of content responses.

19. The electronic system according to claim 17, wherein the processor is operably networked to a database containing a plurality of information data unique to the individual, and generating the plurality of content responses includes generating at least one content response containing at least one information data unique to the individual.

20. The electronic system according to claim 17, wherein the recording device is configured to actively have a recording state for actively recording the input prompt.

21. The electronic system according to claim 17, wherein the recording device is configured to actively have a recording state in which it passively records the input prompt.

22. The electronic system according to claim 17, wherein the processor is configured to electronically communicate the selected content response by causing an external display device to transmit the selected content response.

23. The electronic system according to claim 17, wherein the processor is configured to electronically communicate the selected content response by displaying the selected content response on the electronic display.

24. The electronic system according to claim 17, further comprising a speaker to cause the processor to generate audible sound of the selected content.

25. A method for supporting individuals engaged in electronically supported communication, wherein the method is A step of prompting the individual to enter input at an input prompt in an electronic assistive device, The steps include: accessing a database containing personal information data relating to the aforementioned individual; A step of generating at least one content phrase using the aforementioned personal information data, The step is to electronically display at least one content phrase on the user interface display, separate from the input prompt, The user interface is configured to enable the individual to select the at least one content phrase as the selected content response phrase, comprising the steps of displaying, The steps include: electronically communicating the selected content response phrase using the electronic assistance device; A method that includes this.

26. The method of claim 25, further comprising the step of evaluating the input prompt to identify a topic category associated with the input prompt.

27. The method according to claim 25, wherein the step of generating at least one content phrase includes the step of using the topic category.

28. The method according to claim 25, wherein the electronic support device includes a brain-computer interface.

29. The method according to claim 25, wherein the electronic support device comprises a portable digital device.

30. The method according to claim 25, wherein the electronic support device comprises a computer system.

31. The method according to claim 25, wherein the step of generating the at least one content phrase includes the step of using a large language model operably networked to the electronic assistive device.

32. The method according to claim 25, wherein the database containing personal information data relating to the said individual includes data selected from the group consisting of demographic information, personal preferences, brief history information, family information, health information, calendar information, treatment information, and historical information.

33. The method according to claim 25, wherein the step of electronically displaying the at least one content phrase on the display of the user interface requires an action by the individual.

34. The method according to claim 25, wherein the step of electronically displaying the at least one content phrase on the display of the user interface is performed when the individual begins to input the input.

35. The method according to claim 25, wherein the step of electronically communicating the selected content response using the electronic assistance device includes the step of electronically displaying the selected content response.

36. The method according to claim 25, wherein the step of electronically communicating the selected content response using the electronic assistance device includes the step of generating an audible voice that reads the selected content aloud.

37. The method according to claim 11, further comprising providing the individual with the ability to customize the selected content response using the user interface before electronically communicating the selected content.

38. An electronic system for supporting individuals engaged in electronically supported communication, wherein the system is A processor unit coupled to a memory unit, A program module operably network-connected to an image generator, the program module being configured to receive a communication command prompt and generate a custom pictogram tile having a unique image associated with the communication command prompt, and the program module being configured to store the communication command prompt and the custom pictogram tile in the memory unit, A user interface configured to enable the individual to visually observe a plurality of custom pictogram tiles, the user interface further configured to enable the individual to select at least one of the plurality of custom pictogram tiles as the selected pictogram tile in order to electronically communicate the communication command prompt associated with the selected pictogram tile, and An electronic system equipped with the following features.

39. The electronic system according to claim 38, wherein the program module is operably network-connected to a database containing personal information data relating to the individual, and generates the custom pictogram tile from the database.

40. A method for supporting individuals engaged in electronically supported communication, wherein the method is Steps to receive a communication command prompt, The steps include generating a custom pictogram tile having a unique image associated with the communication command prompt, A user interface configured to enable the individual to visually observe a plurality of custom pictogram tiles, the user interface further configured to enable the individual to select at least one of the plurality of custom pictogram tiles as the selected pictogram tile in order to electronically communicate the communication command prompt associated with the selected pictogram tile, and A method that includes this.

41. The method according to claim 40, wherein the step of generating the custom pictogram tile includes using information from a database containing personal information data relating to the individual.

Images