Programs, methods, and devices for message management and document generation on a device.

Abstract

To make it possible to know a relationship between a first message and a second message.SOLUTION: Systems and methods described herein are directed to facilitate individuals to collect, store, and automatically extract procedural knowledge from their messaging interactions with collaborative groups. Example implementations involve chat interfaces to communicate and add capability to tag, a text and media to organize a content. Example implementations also add a new thread-like structure to a previous only linear time-line of a chat. Knowledge from the chat can be extracted automatically and can be made into a high-quality multimedia document.SELECTED DRAWING: Figure 2(d)

Description

[Technical Field]

[0001] The present disclosure generally relates to communication platforms and devices, and more specifically relates to message document generation and message management on devices. [Background Art]

[0002] In the related art, messaging applications such as chat enable people to share various types of multimedia by utilizing a simple and well-understood interaction metaphor of conversation timeline. Such related art applications can support small task-oriented user groups to mutually coordinate through group chat in both workplace and personal daily scenarios. However, although existing applications such as chat can provide functions for communication and content sharing, it may be difficult to find content generated and shared over a long period of time in a concise and useful manner. Users of related art implementations can only search for individual words, and cannot find a series of connected comments or answers such as responses to a question, which may be spread throughout the entire chat in conventional chat interfaces.

[0003] In implementations of the related art, content may not be organized for later use or to facilitate access. Being able to do so can be useful for small task-oriented groups having a common goal or project.

[0004] Furthermore, many of the devices implementing such messaging applications like chat use displays with limited display space due to the size of the devices (e.g., smart phones, tablets, etc.), therefore there is a need for an interface for providing chat messages that takes such limited display space into consideration. [Prior Art Documents] [License]

[0005] [License 1] U.S. Patent No. 8558693 [License 2] U.S. Patent No. 8219115 [License 3] U.S. Patent No. 7583972 [Non-licensed literature]

[0006] [Non-licensed Document 1] ALLAN, J., ET AL., "TAKING TOPIC DETECTION FROM EVALUATION TO PRACTICE", PROCEEDINGS OF THE 38TH HAWAII INTERNATIONAL CONFERENCE ON SYSTEM SCIENCES, 4(04), 2005, 10 PGS. [Non-licensed Document 2] CSELLE G., ET AL., "BUZZ TRACK: TOPIC DETECTION AND TRACKING IN EMAIL", IUI'07, JANUARY 28-31, 2007, HONOLULU, HAWAII, 8 PGS. [Non-licensed Document 3] CHUNG, J., ET AL., "WITHOUT THE CLUTTER OF UNIMPORTANT WORDS": DESCRIPTIVE KEY PHRASES FOR TEXT VISUALIZATION", ACM TRANSACTIONS ON COMPUTER-HUMAN INTERACTION, 19(3), ARTICLE 19, OCTOBER 2012, 29 PGS. [Non-licensed Document 4] FONG, V., "AN EDITABLE MULTI-MEDIA AUTHORING EBOOK SYSTEM FOR MOBILE Learning", ICHL 2014, LNCS 8595, pp. 184-195, 2014 [Non-Patent Document 5] KIM J., "TOOL SCAPE: ENHANCING THE LEARNING EXPERIENCE OF HOW-TO VIDEOS", CHI 2013 Extended Abstracts, April 27-May 2, 2013, Paris, France, pp. 2707-2712 [Non-Patent Document 6] "THE 2004 TOPIC DETECTION AND TRACKING (TDT2004) TASK DEFINITION AND EVALUATION PLAN", Version 1.0, October 19, 2004, 16 pgs. [Non-Patent Document 7] "PATENT MESS FOR LOCATION-BASED REMINDERS FOR AMAZON, APPLE, MICROSOFT AND GOOGLE", November 22, 2011, 1 pg. [Non-Patent Document 8] "OUR HOME" (retrieved on May 23, 2016) 9 pgs. URL:HTTP: / / OURHOMEAPP.COM / [Non-Patent Document 9] "SLACK TECHNOLOGIES" (retrieved on May 17, 2016) 2 pgs. URL:HTTP: / / SLACK.COM / [Non-Patent Document 10] "WIKI HOW" (retrieved on May 23, 2016) 3 pgs. URL:HTTP: / / WIKIHOW.COM / MAIN-PAGE [Summary of the Invention] [Problem to be Solved by the Invention]

[0007] The technology disclosed herein provides a message management program based on the relationship between a first message and a second message, a method for managing messages on a mobile device, and a mobile device. [Means for solving the problem]

[0008] A part of the present disclosure is a program that causes a computer to function as: means for managing messages for each of several groups of a chat; means for displaying a plurality of first messages posted on a first display screen that displays the chat timeline of one of the several groups; means for receiving a second message which is a response to any of the first messages; means for managing a plurality of messages consisting of the second message and the first message that received the second message as a thread when the second message is received; and means for displaying a list of the threads on a second display screen different from the first display screen.

[0009] A part of the present disclosure is a computer-operated method for managing messages, which includes managing messages for each of several groups of a chat, displaying a plurality of first messages posted on a first display screen that displays the chat timeline of one of the groups, receiving a second message that is a response to any of the first messages, and upon receiving the second message, managing the plurality of messages consisting of the second message and the first message that received the second message as a thread, and displaying a list of the threads on a second display screen different from the first display screen.

[0010] Aspects of the present disclosure further provide a device comprising a processor, wherein the processor manages messages for each of a plurality of chat groups, displays a plurality of posted first messages on a first display screen that displays a timeline of chats of one group among the plurality of groups, receives a second message that is a response to any one of the first messages, manages, upon receiving the second message, a plurality of messages consisting of the second message and the first message to which the second message responds as a thread, and displays a list of the threads on a second display screen different from the first display screen. [BRIEF DESCRIPTION OF THE DRAWINGS]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an outline of a system according to an implementation example. [Figure 2(a)] FIG. 2 is a diagram illustrating an extended chat application (management program) according to an implementation example. [Figure 2(b)] FIG. 3 is a diagram illustrating an extended chat application (management program) according to an implementation example. [Figure 2(c)] FIG. 4 is a diagram illustrating an extended chat application (management program) according to an implementation example. [Figure 2(d)] FIG. 5 is a diagram illustrating an extended chat application (management program) according to an implementation example. [Figure 2(e)] FIG. 6 is a diagram illustrating an extended chat application (management program) according to an implementation example. [Figure 3] FIG. 7 is an overall flow diagram according to an implementation example. [Figure 4] FIG. 8 is a data flow diagram according to an implementation example. [Figure 5] FIG. 9 is a diagram illustrating a flow of content collection according to an implementation example. [Figure 6] FIG. 10 is a flow diagram of document generation according to an implementation example. [Figure 7(a)] FIG. 11 is a diagram illustrating an example of a document generation interface for a chat application according to an implementation example. [Figure 7(b)] This figure shows an example of a document generation interface for a chat application, based on an implementation example. [Figure 7(c)] This figure shows an example of a document generation interface for a chat application, based on an implementation example. [Figure 8] This is a flowchart illustrating the documentation preparation, assembly, and presentation process using implementation examples. [Figure 9] This figure shows an exemplary computing environment with exemplary computer devices suitable for use in implementation examples. [Modes for carrying out the invention]

[0012] The following detailed description provides further details of the drawings and implementation examples of this application. Reference numbers and descriptions of elements that overlap between drawings are omitted for clarity. Terms used throughout the specification are provided illustratively and are not intended to be limiting. For example, the use of the term “automatic” may include fully automatic or semi-automatic implementations in which a user or administrator controls specific aspects of the implementation, depending on the implementation desired by a person skilled in the art in performing the implementation of this application.

[0013] The implementation examples described herein relate to a message management and document generation system for devices such as mobile devices. On such devices, messages, such as chat messages, may appear ad-hoc, unstructured or unrelated to past messages except for the time they appear upon receipt. Such ad-hoc message reception can be particularly problematic on devices with limited screen space, such as mobile devices, where users must repeatedly scroll through messages to determine their context. The implementation examples provide a message management interface that groups messages by context (e.g., topic or group) and assigns tasks and reminders to people associated with specific message contexts. The implementation examples further provide a system that converts such grouped messages into documents that can be shared among participants in a message exchange.

[0014] Groups that may benefit from the implementation examples described herein include, but are not limited to, family (or elderly) caregivers, animal co-caregivers, groups assisting others or coordinating common tasks or projects, or members of a delivery team. These groups share the following common characteristics: they are problem-solving, especially when multiple parts of the problem are repetitive tasks that may recur in the future; they have several problems that the group may have to deal with repeatedly; they may discuss the topic and arrive at practical solutions for the future; a task list that the group must complete, along with deadlines that the group members are aware of, may help them reach a common goal; and because not all group members can participate simultaneously, communication may be on different topics or occur asynchronously. Therefore, the implementation examples facilitate the extraction of generated knowledge for future use.

[0015] The implementation of related technologies does not provide a messaging application interface that has a knowledge extraction system that allows users to create multimedia documents and save and edit content such as how-to instructions, frequently asked questions (FAQs), and summary reports for future use.

[0016] An example of an application interface implementation for a message management program is an extended chat interface, which allows for the creation of chats as messages, providing summaries of topics and questions that can be marked by priority. These elements are highlighted within the chat and displayed as starting points on the start screen. In this implementation example, instructions, FAQs, and reports are also created based on the knowledge accumulated in the chat messages. This can be user-guided, semi-automated, or fully automated. Existing related technologies cannot be implemented to create chats. Furthermore, related technologies do not have the functionality to convert chat content into a readable document format.

[0017] Figure 1 shows an overview of the system based on an implementation example. System 100 can include an extended chat application 101, a document generator 102, and a document viewer 103. The extended chat application 101 consists of functions capable of supporting task-oriented small groups and is available on mobile devices and desktop computers. The document generator 102 enables the creation of multimedia documents such as how-to instructions, FAQs, and reports. The document viewer 103 is used to filter, view, and play documents, which can also be generated as printable reports 104.

[0018] Figure 2(a) shows an example implementation of an enhanced chat application with a management program for managing messages such as chats. Chat applications of related technologies can include features such as sending text, image, video, and audio messages, editing images and videos, sending animated GIFs and emoticons, sending contact cards, and group communication. The example implementation of the enhanced chat application provides additional elements designed for task-oriented small groups. Such elements can be created during a conversation in the chat view. Depending on the chat entry, a dialog opens, allowing the user to enter additional information (e.g., due date, selected group members, valid duration, etc.). The created element is then marked within the chat and / or displayed in an enhanced start / overview screen as shown in Figure 2. This not only shows the group and past chats, but also indicates unanswered questions or other elements requiring action, as well as newly added elements or messages within the chat, as follows:

[0019] Subthreads: In an example implementation of subthreads, questions or discussions are highlighted during a chat and displayed on the application's start page until they are marked as finished (see, for example, "Where can I buy flowers?" in Figure 2(a)). An example implementation could also divide the chat into a strictly timeline-based view to initiate a threaded structure of elements. Furthermore, a subthread may contain other subthreads, depending on the desired implementation. For example, a subthread might branch into other subthreads based on the group participating in the discussion. Threads / subthreads can have subthreads, making it easy to create thread chains of arbitrary depth, depending on the desired implementation.

[0020] Note: In this note implementation example, notes can be highlighted within the chat, and recipients can mark them to hide them from the start page. Such notes follow a "sticky note" metaphor and can remain on the start screen until the application is closed. In contrast, other discussion elements that are not "sticky note" can be placed as appropriate within the timeline and removed from the screen as new messages are added.

[0021] To-Do / Tasks: In this implementation example, tasks can be highlighted in the chat and displayed in a to-do list in an additional tab within the application. Tasks can be marked as completed by at least one of the designated group members. Depending on the desired implementation, there may also be deadlines or requests for feedback (e.g., a photo of the result).

[0022] Message to the Future: In this implementation example, a message can be planned to be sent at a specific future time and delivered only at that specific future time.

[0023] Time-based reminders: In the example implementation, time-based reminders can be created in the same way as regular messages, except that the sender specifies the date / time of the reminder. This reminder can be configured to appear (for example, immediately) in the recipient's message timeline list, or it can be configured to reappear as the latest message to that recipient at a specific date / time.

[0024] Location-Based Reminders: In an example implementation for location-based reminders, the sender sets a reminder location instead of a date / time to ensure the message is rescheduled. For example, when a message is sent with a location-based reminder, the message can be held in the receiver's queue until the receiver reaches that specific location. In another example implementation, the message may be sent once the receiver indicates it is at that specific location. Other implementations can be used depending on the desired implementation.

[0025] Time and location-based reminders: In this implementation example, time and location reminders can be combined to allow the sender to set a reminder location in addition to the date and time the message should be displayed.

[0026] Group Calendar: In this implementation example, the group calendar provides an overview of individual and group activities. The group calendar may be synchronized with other calendars. Events and appointments can be sent directly to the calendar from the chat.

[0027] Marking messages: In this implementation example, messages posted to the chat timeline can be marked as important or "liked."

[0028] Search: In the implementation example, an interface is provided to facilitate searching for specific words, images, audio, and / or videos within messages such as chats. Media search requires advanced search methods such as date search and media type search.

[0029] Each of the above functions can be sent to a group of members. • All Groups: All group members receive the element. • Part of a group: A defined subset of group members receives elements. • Single member: Only one member of the group receives the element.

[0030] Adding subthreads as questions to start new discussions within a chat subsection is a concept not found in chat applications of related technologies. Using subthreads makes it possible to organize incoming chat messages as received. This saves screen space by associating relevant messages with their corresponding threads, rather than displaying the messages upon receipt as in implementations of related technologies.

[0031] In the implementation example, subthreads can be created for several reasons. For example, subthreads make it easier to keep information about a topic together within the chat and ensure that questions are answered.

[0032] Group members may initiate a discussion if they wish to discuss a topic. While related technology applications facilitate starting discussions in the chat view, over time, messages related to one topic or discussion can become confused with other discussions or chat topics. Creating subthreads allows the group to focus on a single topic without requiring references or citations of past answers that could lead to other topics and discussions outside of this group.

[0033] Furthermore, one problem with questions and answers in group chats is that responses to questions can become detached from the original question and get mixed up with messages from other topics. For example, if a user asks a question during a chat conversation, but the person who can answer it is not online, and the chat conversation continues, the question may get sidetracked until that person comes back online. Sometimes, a user may ask multiple questions in a series of messages before other users can answer them immediately. One or more users in a group may start answering one question, and the conversation shifts, causing other questions to be forgotten. Creating a subthread for a question can help prevent it from being forgotten, as it will be marked and displayed in the chat and appear as a new item on the home screen (which is the entry point when a user opens the application).

[0034] Figure 2(b) shows a chat application with an implementation example. In particular, Figure 2(b) shows an example of a screen that generates special elements. In the implementation example, a subthread may be created when a message is created. A subthread can be created when a new message is created or by increasing the level of an already posted message. If the user enters a message (or posts a file) before sending it, that message may be marked on the element marking screen 210 as a special element (e.g., a subthread), as shown in Figure 2(b). It is also possible to create a special element and write text to the subthread.

[0035] In the implementation example, already posted messages can be leveled up without using follow-up messages. For example, a user (not necessarily the creator of the entry) can take action on elements within the chat (e.g., a long press on the touchscreen). The menu opens an element marking screen 210 as shown in Figure 2(b), where the entry can be marked as a special element.

[0036] In the implementation example, already posted messages can be leveled up by manually selecting follow-up messages. In the implementation example, users (for example, not necessarily the creator of the entry) can take action on elements within the chat (for example, by long-pressing on a touchscreen). The menu opens an element marking screen 210 as shown in Figure 2(b), where entries can be marked as special elements. The user can then proceed to the next screen and select an existing entry that should become part of a new subthread.

[0037] In the implementation example, already posted messages can be leveled up by semi-automatically selecting follow-up messages. In the implementation example, users (for example, not necessarily the creator of the entry) can take action on elements within the chat (for example, by long-pressing on the touchscreen). The menu opens an element marking screen 210 as shown in Figure 2(b), where entries can be marked as special elements. The user can then proceed to the next screen to modify pre-selected entries that should become part of a new subthread.

[0038] Subthreads can be viewed in various ways, such as through thread and subthread interfaces, color views, or split-screen views. Viewers can switch between different displays using a chat application. The displays may be different representations of the same information.

[0039] In an implementation example that includes a thread and subthread interface, as shown in Figure 2(c), the thread and subthread interface breaks down the chat into a strictly timeline-based paradigm, displaying subthreads within the chat window. Replies or answers to subthreads can be added by selecting the "+" at the end of the subthread, which adds the entry to the end of the target subthread rather than the end of the entire chat. Using this paradigm, new messages do not necessarily appear at the end of the entire chat that has been marked as unread since a certain point in time. Messages can spread throughout all threads in the entire chat. Such implementations may also utilize hints to the user that there may be new entries somewhere in the chat. An example of such a hint is a warning sign in the top bar (e.g., a button to draw the user's attention), which indicates how many new entries there are in the entire chat and can also function as a button that the user can use to jump from one new entry to another in the chat. An example implementation of the thread and subthread interface is shown in Figure 2(c). In the overview screen in Figure 2(a), the number of new entries related to a question is shown next to the question. In the example implementation, selecting a question jumps to the last unread message in the subthread within the chat.

[0040] In the example, the color view maintains a timeline-based paradigm for the chat, with subthreads marking different entries in different colors. Replies or answers to subthreads can be added by clicking the "+" next to the question, and the entries will be marked in the same color as the question. Using this paradigm, new messages will appear at the end of the entire chat, marked as unread from a certain point in time. However, questions may spread throughout the chat. Therefore, the example implementation may include buttons that allow the user to jump from one question to another within the chat. In the example implementation, it is also possible to mark a new entry as part of a single subthread using a gesture (e.g., long press) on the last answer in a subthread, as shown in Figure 2(d). As shown in Figure 2(d), related messages are color-coded, and when entered into a plus sign or other desired user interface indicator, the device scans for messages of the same color and displays or highlights the next message with the same color based on the timestamp. The overview screen shows the number of new entries related to a question next to the question. Selecting a question jumps to the first unread answer to that question in the chat.

[0041] In the example implementation of the two-screen view, the view can be implemented in the same way as the thread and sub-thread interface. Furthermore, the two screen views maintain the chat's timeline-based paradigm by marking the start of a sub-thread with an icon. Replies or answers to sub-threads can be read by tapping the sub-thread, which leads the user to a new screen showing that sub-thread and its follow-up messages. Using this paradigm, new follow-up messages for this sub-thread can be entered as normal messages on the sub-screen by simply typing in the text input at the bottom of the screen, as shown in Figure 2(e), without first selecting a special button. The overview screen shows the number of new entries related to the question next to the question. Selecting a question jumps to the sub-thread view. A separate screen is generated to represent the sub-thread to the thread, as shown in Figure 2(e). Furthermore, it is possible to provide the same interface within the separate screen, thereby facilitating the implementation of sub-threads within sub-threads and the generation of separate screens for sub-threads within sub-threads, according to the desired implementation.

[0042] One implementation example involves a document generator that can be added to a chat application. This newly introduced feature adds another level of annotation to the previous linear chat. Different ways of marking elements provide hints of importance. However, finding information later in longer chat messages can be difficult. Here, we can not only mark elements but also help extract semantically related sets of elements containing knowledge from the chat and transform them into a more readable format such as instructions, FAQs, or reports. Elements can be selected based on user choice or pre-selected by annotations (e.g., element type, star marks / markers given by the user over time) depending on the document being created. Below, an implementation example facilitates the creation of multimedia documents from a chat application.

[0043] In this implementation example, the multimedia document may be user-guided. In user-guided creation, the interface facilitates the creator's execution of a few steps. First, chat elements are selected via the interface. This process begins with the user clicking a single chat entry. Next, a selection tool provides a list of subsequent elements within the application. The user can then choose whether each element should be added to the document. The selected elements are then copied to an editing area where they can be rearranged, edited, and merged with previous or subsequent elements. New elements can be added from the local file system or the internet if necessary, and formatting can be applied. Finally, the document can be exported in the desired format.

[0044] In the implementation example, multimedia documents can be generated by a semi-automatic process. In the semi-automatic creation, several algorithms are applied after the user selects a starting point in the chat. Basic text analysis and annotation analysis (e.g., like, marker) suggest relevant entries, which may be modified by the user. In the copying process, unnecessary parts of the text (e.g., emoticons, unnecessary spaces) are removed, and elements posted by the user as a series of small messages are merged into a single text. The resulting list of elements can then be edited as described in the user guidance process, where the user is assisted by search and research algorithms to provide video scenes, images, or other materials from the internet.

[0045] In the implementation example, multimedia documents can be generated by an automated process. This automated generation can also be configured to learn from user-guided and semi-automatic modes and automatically export the desired document format using learning algorithms such as machine learning algorithms.

[0046] In addition to linear multimedia documents, hypermedia graphs can also be created using the document creation interface. A hypermedia graph requires a graph view and editor for the logic statements applied to the forks within the graph. Such an implementation allows for the creation of hypermedia documents, particularly on mobile devices, that can provide different levels of detail, difficulty, or instructions to different user groups. The mobile interface provides a document overview and interactive elements to existing documents. This document can then be extended to enable hypermedia creation on mobile devices. By recording information in the app, more detailed information and different types of media elements can be added to provide more information.

[0047] More complex subprocesses during document generation include the selection of subsequent entries, text cleaning, and entry reordering. These components are built upon the implementation of related topic detection and tracking techniques, enabling the derivation of initial topic clustering of chat messages and the tracking of the resulting detected topics. Such techniques have also been applied to newsstreams and email organization. Techniques such as key phrase extraction and conversational dynamics analysis can also be used to filter less important posts and structure the generated documents.

[0048] In the example implementation, a multimedia document viewer provides the ability to display all types of media and navigation elements. The viewer can adapt to playback devices and user characteristics by filtering the hypermedia graph. Depending on the playback device, the amount of data, along with the size of media elements, can be adjusted. In the example implementation, slower connections may concentrate on text and images, while video (where different types of media are available) is displayed using high bandwidth. Furthermore, the report generator can provide filtering capabilities to adapt the resulting document according to user input and purpose.

[0049] Furthermore, if information is missing or not presented as intended, a feature is available that allows the user to expand the document they are viewing, effectively turning a passive viewer into a co-author. Such a feature can operate in different modes.

[0050] In an example implementation of uncontrolled mode, content can be edited by the user without an approval process.

[0051] In an example implementation of creator control mode, when a user adds or edits content, a notification is sent to the multimedia / hypermedia creator. If the creator agrees, they can accept the changes. If the creator does not agree, the content is rejected or a discussion can be initiated.

[0052] In an example implementation of user-controlled mode, when a user adds / edits content, the new content is marked as unverified. Other users can then peer-review, modify, and document that content. You can vote on whether it is acceptable as "regular" content. This ensures the high quality of new information and the continuous improvement of documentation.

[0053] Figure 3 is an overall flow diagram based on an implementation example. In this implementation example, the system can be divided into three subprocesses: content collection, hypermedia document generation, and hypermedia preparation. This will be explained in detail below, as shown in Figure 3.

[0054] In step 301, the system is configured to collect content from an extended chat application, as described in Figures 2(a) to 2(e). Such content may include text, images, videos, audio, or files posted via the chat application. In step 302, after collecting the content, a hypermedia document can be generated from the content. In step 303, the hypermedia document is provided to the user as a display interface. A more detailed flow is shown in Figure 4.

[0055] Figure 4 shows the data flow in an implementation example. Specifically, Figure 4 shows the data flow between the enhanced chat application 401, the document generator 402, and the document viewer / report 403. In the enhanced chat application 401, the user sends a message at 410. This is sent to the enhanced chat UI at 411. If the message is text only, the text is added to the database and the connection to the entry is updated at 412. If the message is a media file, the media file is stored on the server at 413 and a link is added to the database. The connection to the entry is then updated. The chat is then updated for all users.

[0056] In the document generator 402, a user creating a document in 420 opens the entry selector in 421 to select a starting entry, and is then guided to the hypermedia document editor 422 in 422. The hypermedia document editor has an interface where content can be edited, and content from the internet can be added as shown in 423. The content is linked together with the hypermedia document. For chat, media files are stored on a file server, and text and structure are stored in a database. The user can select a document and add further input. Thus, an interactive hypermedia document is created and can be displayed in a viewer or the content can be exported as a document file (e.g., PDF, DOC).

[0057] In the document viewer / report 403, the user viewing the document in 430 opens the hypermedia document viewer 431. This displays the interactive hypermedia document in 432. The interactive hypermedia document 432 is created in the hypermedia document assembler 433. This is also configured to generate the document file for the user in 434 after the user instructs the hypermedia document viewer 431 to construct the document.

[0058] Figure 5 shows the content collection flow in an implementation example. Specifically, Figure 5 shows the process during content collection (e.g., a chat application). A chat / sub-chat selection is received via the user on the interface in 501. In 502, the system continues adding content. During this flow, the system may receive typed text or media via the extended chat application. In 503, elements may be marked as shown in the interface of Figure 2(b). As shown in Figure 2(b), elements can be marked as questions / discussions, to-dos, notes, reminders, events, messages for the future, etc. In particular, if an element is not marked (No), the flow proceeds to 504 to determine whether content generation is complete. If it is marked (Yes), the flow ends; otherwise (No), the flow proceeds to 502 to process additional content.

[0059] In 505, a check is performed to determine if the marked element is an instruction to start a question or discussion. If so (Yes), the flow proceeds to 506 to mark the element as the start of a question or discussion, and then proceeds to 507 to set the visibility. In 507, an interface can be provided to facilitate setting the visibility of a particular element. In the example implementation, there can be three visibility settings: single user, part of a group, and entire group. If single user or part of a group is selected, the corresponding subset of the group is selected.

[0060] If not (No), the flow proceeds to 508, where a check is performed to determine if the element is a ToDo element. If it is (Yes), the flow proceeds to 509, where the element is marked as a ToDo element. The flow then proceeds to 510, where the final due date is set. Here, an interface is provided to facilitate the selection of the date and time on which the element should remain active or when the element must be completed. The provided interface may also include an interface in 511 to facilitate reminder setting, where the interface facilitates the selection of when the reminder should be set and how far in advance the reminder should open the chat application. Furthermore, the interface can also facilitate the setting of actions in 512, which may be requests for taking a photo, answering a question, entering a value, or obtaining video or some other kind of documentation from the recipient, according to the desired implementation.

[0061] Otherwise (No), the flow proceeds to 513, where it performs a check to determine if the element is a memo element. If it is (Yes), the flow proceeds to 514, where it marks the element as a memo. The flow then proceeds to 515, where it sets the validity via an interface. Here, an interface is provided to facilitate the selection of the length of time that indicates how long the element is valid and visible to other users.

[0062] Otherwise (No), the flow proceeds to 516, where it performs a check to determine if the element is intended to be a reminder. If it is (Yes), the flow proceeds to 517, where it marks the element as a reminder. The flow then proceeds to 518, where it determines the type of reminder. If the reminder is a place and time-based reminder (Yes), the flow proceeds to 524, where it sets the location and other reminder information via the interface. In the example implementation, the location can be entered by either typing an address or marking the location on a map.

[0063] If not (No), the flow proceeds to 519 to determine if the element is a time-based reminder. If it is (Yes), the flow proceeds to 525 to provide the interface, set the start date, and proceeds to 526 to set the time for the element. If not (No), the flow proceeds to 520 to determine if the element is a location-based reminder. If it is (Yes), the flow proceeds to 521 to set the location as in the flow at 524. If not (No), the flow proceeds to 507 to set the visibility.

[0064] If, at step 516, a particular element is not a reminder (No), the flow proceeds to 522, where it is determined whether the element is event-oriented. If so (Yes), the flow proceeds to 523, marking the element as an event. The flow then proceeds to 524 to get and set the event's location, and then proceeds to get and set other information such as the event's date and time, according to the desired implementation.

[0065] If a particular element is not an event (No), the flow proceeds to 527 to mark the element as a message for the future. Then it proceeds to 525 to set the date and to 526 to set the time.

[0066] In another implementation, after the content within the extended chat application is collected as shown in 301 of Figure 3, a hypermedia document may be generated in 302 and provided to the document viewer in 303. Figure 6 shows a flowchart of document generation in an example implementation. In 601, a decision is made as to whether document generation should be a user-guided generation process (e.g., manual) or a semi-automated process.

[0067] In the user-guided document generation process (Yes), 613 provides an interface to facilitate the selection of chat inputs. Inputs are selected by marking each one through the interface. 614 provides user confirmation or an interface to determine whether the input content is acceptable. If not (No), the flow proceeds to 615, where an interface to facilitate editing of the chat inputs is provided. A interface is provided. In such an interface, input is modifiable, allowing users to delete, for example, typos, emojis, or unwanted text. Adding or deleting text is also possible.

[0068] At 616, a user confirmation or interface is provided to determine if the entry order is acceptable. If not (No), the flow proceeds to 617, where an interface is provided to facilitate rearranging the order of chat inputs according to the desired documentation. In the example implementation, there may be situations where the entry order is incorrect for the chat timeline. Single entries can also be reordered if necessary.

[0069] At step 618, user confirmation or an interface is provided to determine whether additional content should be added. If so (Yes), the flow proceeds to step 619, where an interface for adding additional content is provided. If information is missing, or if the creator knows of a better illustration than those collected during the chat, external information can be added according to the desired evidence.

[0070] At step 620, the user is given confirmation or an interface to determine whether a hyperstructure should be added. If it should be added (Yes), the flow proceeds to step 621, where the hyperlink can be added. In the example implementation, a hyperstructure can be added to provide different versions of content or different paths through documents.

[0071] If document generation is performed via a semi-automated process (No), the flow proceeds to 602, where an interface is provided to facilitate the selection of a starting entry from the chat. In automatic or semi-automatic mode, once a starting entry is selected, content generation begins by selecting entries that follow the starting entry in 603.

[0072] In sections 604, 608, and 612, a previously set decision is checked regarding whether a semi-automatic generation process is desired. If so (Yes), several interfaces are provided, which may include the following:

[0073] At 605, an interface for selection correction can be provided, where the automatic selection result at 603 can be corrected using the same interface used in user-guided mode. If the selection is OK (yes), the flow proceeds to 607, where the text is automatically cleaned, for example, as described at 615. Otherwise (no), the flow proceeds to 606, where the entry selection is corrected according to instructions from the user via the interface.

[0074] At 609, a text correction interface can be provided, where the results of automatic text cleaning can be corrected using the same interface used in user-guided mode. If the text is OK (yes), the flow proceeds to 611, where the entries are automatically rearranged as described in 617, for example. Otherwise (no), a text cleaning process is provided as shown in 610.

[0075] Next, in 622, the document structure is exported to the database 412 and file storage 413 from which the document viewer retrieved the document. When automatic generation is complete, or when the user terminates user-guided generation or semi-automatic generation, the document structure is exported to the viewer along with the document content.

[0076] Figures 7(a) to 7(c) show examples of document generation interfaces for chat applications based on implementation examples. The user guidance process interface provides a screen, as shown in Figure 7(a), where the user first defines the type of document they wish to create (e.g., how-to, FAQ, report, etc.). Then, depending on the document, structural elements can be provided into a template according to the desired implementation. For example, steps and substep elements are part of a how-to instruction template, as shown in Figure 7(b); question elements are part of an FAQ template; and elements such as title, stakeholders, introduction / purpose, time span, and progress are part of a report template. Steps can be typed in by the user or selected from existing elements. Content from chat input, to-do notes, memos, or other special elements from chat messages can be used to fill in the template, as shown in Figure 7(b). In manual mode, the user selects these inputs on a selection screen, as shown in Figure 7(c). After adding content, it can be reorganized and edited. The completed document can be expanded using other versions of hypermedia document creation during this process.

[0077] Figure 8 is a flowchart of document preparation and generation using an implementation example. At 801, an interface is provided to facilitate document selection. For example, the user can select a document from the Overview tab. At 802, it is determined whether the output device is a printer. If so (Yes), the flow proceeds to 803, where the document is generated from the multimedia document. In the implementation example, keyframes are extracted for each video and added to the document. Text is extracted for each audio and added to the document. At 804, the document is printed. If a printer is unavailable, it is downloaded to the local device.

[0078] If the output device is not a printer (No), 805 determines whether other versions of the document are available. If so (Yes), 806 provides an interface to retrieve input variables. In an example implementation, input variables for the document customization interface may include screen resolution, available bandwidth, device language, user knowledge, user gender, desired interactivity level (e.g., low / high), paths taken by other users from the same document, and other variables depending on the desired implementation. Some input variables may be determined automatically from the device, some based on user knowledge, and others may require user input (e.g., at initial use). The flow proceeds to 807, where the document is customized based on the input variables. Content is adapted, and paths within the hypermedia document may be pre-selected.

[0079] In 808, the document is loaded into the viewer, and the user can monitor or expand the document. The expansion process may be uncontrollable, controlled by the creator, or controlled by the user.

[0080] Possible implementations include content collection and chat applications. The implementation facilitates the connection and synchronization of several tools offering a wide variety of functionalities. In addition to searching within the chat, it's also possible to search shared files that have been indexed and saved within the application. Messages can be marked as future reminders. This allows the implementation to directly structure the chat from the chat view, or to mark chat elements as important.

[0081] Figure 9 shows an exemplary computing environment having exemplary computer devices suitable for use in implementation examples. The computer device 905 in the computing environment 900 may include one or more processing units, cores, processors 910, memory 915 (e.g., RAM, ROM, and / or similar), internal storage 920 (e.g., magnetic storage, optical storage, solid-state storage, and / or organic storage), and / or I / O interfaces 925, any of which can be connected to a communication mechanism or bus 930 for information communication, or embedded in the computer device 905.

[0082] Computer device 905 can be communicatively connected to an input / user interface 935 and an output device / interface 940. Either one or both of the input / user interface 935 and the output device / interface 940 may be wired or wireless interfaces and may be detachable. The input / user interface 935 may include any physical or virtual device, component, sensor, or interface available to provide input (e.g., buttons, touchscreen interfaces, keyboards, pointing / cursor controllers, microphones, cameras, braille devices, motion sensors, optical readers, etc.). The output device / interface 940 may include displays, televisions, monitors, printers, speakers, braille devices, etc. In some implementations, the input / user interface 935 and the output device / interface 940 may be embedded in or physically connected to computer device 905. In other implementations, other computer devices may function as or provide the input / user interface 935 and output device / interface 940 for computer device 905. In implementation examples including a touchscreen display, a television display, or any other form of display, the display is configured to provide a user interface such as those shown in Figures 2(a) to 2(e) and Figures 7(a) to 7(c).

[0083] Examples of computer devices 905 include, but are not limited to, advanced mobile devices (e.g., smartphones, devices mounted in vehicles and other machines, devices carried by humans or animals), mobile devices (e.g., tablets, notebook computers, laptop computers, personal computers, portable televisions, portable radios, etc.), and non-mobile devices (e.g., desktop computers, other computers, information kiosks, televisions, radios, etc., having one or more processors embedded and / or connected inside).

[0084] Computer device 905 may be communicatively connected to external storage 945 and network 950 (for example, via I / O interface 925) and capable of communicating with any number of network-connected components, devices, and systems, including one or more computer devices of the same or different configurations. Computer device 905 or any connected computer device may function, provide services, or be referred to as a server, client, thin server, general-purpose machine, dedicated-purpose machine, or other label.

[0085] The I / O interface 925 includes, but is not limited to, wired and / or wireless interfaces that utilize any communication or I / O protocol or standard (e.g., Ethernet®, 802.11x, Universal System Bus, WiMAX, modem, cellular network protocol, etc.) for communicating information to and from at least all connected components, devices, and networks in the computing environment 900. The network 950 may be any network or combination of networks (e.g., the Internet, local area network, wide area network, telephone network, cellular network, satellite network, etc.).

[0086] Computer device 905 can be used and / or communicated using computer-usable media or computer-readable media, including temporary media and non-temporary media. Temporary media include transmission media (e.g., metal cables, fiber optic materials), signals, carrier waves, etc. Non-temporary media include magnetic media (e.g., disks and tapes), optical media (e.g., CD-ROMs, digital video discs, Blu-ray discs), solid-state media (e.g., RAM, ROMs, flash memory, solid-state storage), and other non-volatile storage or memory.

[0087] Computer device 905 can be used to implement techniques, methods, applications, processes, or computer executable instructions in several exemplary computing environments. Computer executable instructions can be retrieved from temporary media and stored in non-temporary media. Executable instructions can be written in any programming language, scripting language, and machine language (e.g., C, C++, C#, Java®, Visual). It can be derived from one or more of the following: Basic, Python, Perl, JavaScript (registered trademark), etc.

[0088] The memory 915 may be configured to store or manage algorithms executed by the processor 910, for example, as shown in the flows in Figures 1, 3-6, and 8. The implementation examples described herein may be executed individually or in any combination thereof, depending on the desired implementation, and are not limited to any particular implementation example.

[0089] The processor 910 can run under any operating system (OS) (not shown) in a native or virtual environment. One or more applications can be deployed, including a logical unit 960, an application programming interface (API) unit 965, an input unit 970, an output unit 975, and an inter-unit communication mechanism 995 for different units to communicate with each other, with the OS, and with other applications (not shown). The above units and elements are modifiable in design, function, configuration, or implementation, and are not limited to those described herein. The processor 910 may be a physical processor or a central processing unit (CPU) configured to execute instructions loaded from memory 915.

[0090] In some implementations, when information or execution instructions are received by the API unit 965, they may be communicated to one or more other units (e.g., a logical unit 960, an input unit 970, and an output unit 975). In some examples, the logical unit 960 may be configured to control the flow of information between units and direct the operations provided by the API unit 965, input unit 970, and output unit 975 in some of the implementations described above. For example, the flow of one or more processes or implementations may be controlled by the logical unit 960 alone or in conjunction with the API unit 965. The input unit 970 may be configured to take input for the calculations described in the implementations, and the output unit 975 may be configured to provide output based on the calculations described in the implementations.

[0091] In an implementation example, the processor 910 may be configured to display a first message sent from a mobile device on the user interface. In the implementation example shown in Figure 2(a), message 201, sent from the mobile device to another device in 201, is placed on the display, and the processor 910 monitors for one or more second messages from the other mobile device in response to the first message sent from the mobile device. Upon receiving a second message from the other mobile device, the processor 910 is configured to generate a user interface indicator on the first message displayed on the user interface to provide a relational interface between the first message and the second message on the user interface. The user interface indicator can be configured to provide this relational interface as one of the following: a thread and sub-thread interface as shown in Figure 2(c), a color-coded relational interface as shown in Figure 2(d), or the generation of a separate screen as shown in Figure 2(e). As shown in Figures 2(c) to 2(e), such an indicator may take the form of a plus sign, an arrow, a Q sign, etc., depending on the desired implementation.

[0092] In the example shown in Figure 2(c), if the user interface indicator is configured to provide relational interfaces as thread and sub-thread interfaces, the processor 910 is configured to generate the user interface indicator by displaying a first message as the start of a thread, providing the user interface indicator as a thread extension instruction (for example, in the form of a Q symbol, a plus symbol, etc.), and, upon receiving input on the user interface indicator, displaying a second message on the user interface as a sub-thread via thread extension, as illustrated by the response to the first message shown in Figure 2(c). Furthermore, this implementation example is extendable to multiple messages, as shown in the example of multiple transmission messages in Figure 2(a). Here, multiple messages related to the transmission message can be displayed on the user interface as sub-threads via thread extension, as shown in Figure 2(c).

[0093] In the example shown in Figure 2(d), if the user interface indicator is configured to provide a relational interface as a color-coded relational interface, the processor 910 is configured to generate the user interface indicator by displaying the first message in the first color, providing the user interface indicator as an instruction to move to the next message having the first color, and, upon receiving input on the user interface indicator, displaying the second message as the next message in the first color. A message displayed in a second color different from the first color is recognized as having a different relation to the first message.

[0094] In the example shown in Figure 2(e), if the user interface indicator is configured to provide a relational interface as a separate screen, the processor 910 is configured to generate the user interface indicator by providing the relational interface as the generation of a separate screen. The processor 910 is configured to generate the user interface indicator by displaying a first message on the first display screen, providing the user interface indicator as a change instruction (e.g., an arrow) to the second screen, and, upon receiving input to the user interface indicator, changing the first display screen to the second display screen and displaying a second message on the second display screen as shown on the right side of Figure 2(e).

[0095] In the example implementation, the first message may take the form of a location-based reminder, which is configured to be sent to another mobile device when that device reaches the location associated with the location-based reminder. In the example implementation, the message can be sent to the other mobile device, but it will not appear on the other mobile device until it detects that the other mobile device is at the location indicated in the first message at the specified time. In the example implementation, the message may be either time-reminder-based only or location-reminder-based only, depending on the desired implementation.

[0096] In the implementation example, the first message can request a task to be completed. The second message can then include at least one of text, video, and image corresponding to that task. That is, a user on a mobile device can indicate that an image or video is needed to complete the task, and upon receiving the image or video, the user can confirm whether the task has been completed. Once the user determines that the task has been completed, they can delete the message from another mobile device via a confirmation interface (e.g., a reply message, a user interface indicating task completion).

[0097] In an implementation example, the processor 910 can be configured to provide a second user interface indicator, which consists of an interface for generating a document based on a first message and a second message, as shown in Figures 7(a) to 7(c). Upon receiving input to the second user interface indicator, the processor 910 is configured to select the first and second messages on the document editor, as shown in Figures 7(a) to 7(c), and to generate a document for the selected first and second messages processed in the document editor, as described in the flow of Figure 8. For example, as described in Figures 4 and 8, the processor 910 is configured to generate a document for the selected first and second messages by retrieving one or more images related to the second message and incorporating those images into the document (for example, by downloading keyframes of images or videos from the internet or a message server). In this implementation example, the processor 910 is configured to generate a document for the selected first and second messages by extracting one or more video clips related to the second message, extracting one or more keyframe images from the video clips, and incorporating these keyframe images into the document. The extraction of keyframe images can be performed according to any desired implementation known in the art. Furthermore, the processor 910 can be configured to generate the document by printing it on a printer, as shown in Figure 8.

[0098] Some parts of the detailed description are represented by algorithms and symbolic representations of computer operations. These algorithmic descriptions and symbolic representations are means used by those skilled in the field of data processing to communicate the essence of their innovations to others skilled in the field. An algorithm is a set of defined steps that lead to a desired final state or result. In the implementation examples, the steps performed require physical manipulation of tangible quantities to achieve a reliable result.

[0099] Unless otherwise stated, please understand that throughout this explanation, as will be evident from the discussion, discussions using terms such as “process,” “calculate,” “calculate,” “determine,” and “display” may include the operation and processing of computer systems and other information processing devices that manipulate and convert data represented as physical (electronic) quantities in the registers and memory of a computer system into other data similarly represented as physical quantities in the memory, registers, or other information storage, transmission, or display devices of a computer system.

[0100] Implementation examples may also relate to apparatus for performing the operations described herein. This apparatus may be specifically constructed for the required purpose and may include one or more general-purpose computers selectively started or reconfigured by a computer program. Such computer programs may be stored on computer-readable media, such as computer-readable storage media or computer-readable signal media. Computer-readable storage media may include, but are not limited to, tangible media such as optical disks, magnetic disks, read-only memory, random-access memory, solid-state devices and solid-state drives, or any other type of tangible or non-temporary media suitable for storing electronic information. Computer-readable signal media may include media such as carrier waves. The algorithms and representations presented herein are not inherently related to any particular computer or other apparatus. Computer programs may include pure software implementations containing instructions for performing the desired implementation operations.

[0101] Various general-purpose systems can be used with the programs (management programs) and modules according to the embodiments described herein. Alternatively, it may be convenient to construct more specialized devices to perform the desired method steps. Furthermore, the implementation examples are not written with reference to any particular programming language. It should be understood that the teachings of the implementation examples described herein can be implemented using various programming languages. Instructions in the programming language can be executed by one or more processing devices, such as a central processing unit (CPU), processor, or controller.

[0102] As is well known in this art, the above operations can be performed by hardware, software, or any combination of software and hardware. Various embodiments of the implementation examples can be implemented using circuits and logic devices (hardware). On the other hand, other embodiments can be implemented using instructions stored on a machine-readable medium (software). When executed by a processor, this will cause the processor to perform the method of carrying out the implementation of this application. Furthermore, some implementation examples of this application may be performed by hardware alone, and other implementation examples may be performed by software alone. Moreover, the various functions described may be performed by a single unit or by multiple components in any number of ways. When executed by software, the method may be executed by a processor such as a general-purpose computer based on instructions stored on a computer-readable medium. If necessary, the instructions may be stored on the medium in a compressed or encrypted format.

[0103] Furthermore, other implementations of this application will be apparent to those skilled in the art through the considerations herein and the practice of teachings herein. The various implementations and / or components described herein may be used individually or in any combination. The specification and implementations are to be considered merely illustrative, and the true scope and spirit of this application are intended to be shown in the appended claims. [Explanation of Symbols]

[0104] 101 Extended Chat Application 201 Message 905 Computer Devices 910 Processor

Claims

1. Computers, A means of managing messages for each of the multiple chat groups, In a first screen that displays the chat timeline of one of the aforementioned multiple groups, means for displaying multiple first messages that have been posted, Means for receiving a second message which is a response to any of the first messages, A means for managing a plurality of messages, consisting of the second message and the first message that received the second message, as a thread, and A means for displaying the list of threads on a second screen different from the first screen, It is a program designed to function as such. A program that maintains the display order of the multiple first messages displayed on the first screen in the chat timeline, even when managing a plurality of messages consisting of the second message and the first message that received the second message as the thread.

2. The program according to claim 1, wherein the second screen displays the thread in a way that allows the chat group to which the thread belongs to be identified.

3. The program according to claim 1, wherein, on the second screen, the first message constituting the thread is displayed in a manner that allows recognition of each of the threads.

4. The program according to claim 1, wherein, in the second screen, for threads in which unread messages exist, an indication is made that unread messages exist.

5. The program according to claim 4, wherein, on the second screen, when a thread containing an unread second message is selected, the program jumps to the first unread message constituting that thread.

6. The program according to claim 4, wherein, on the second screen, when a thread containing an unread second message is selected, the program jumps to the last unread message constituting that thread.

7. A method of managing messages that a computer performs, Manage messages for each of the multiple chat groups, On a first screen that displays the chat timeline of one of the aforementioned groups, multiple first messages that have been posted are displayed. Upon receiving a second message which is a response to any of the first messages, Upon receiving the second message, the multiple messages consisting of the second message and the first message that received the second message are managed as a thread. A second screen, different from the first screen, displays a list of the threads. It is a method, A method for maintaining the display order of the plurality of first messages on the chat timeline, even when managing a plurality of messages consisting of the second message and the first message that received the second message as the thread, on the first screen.

8. The processor comprises, Manage messages for each of the multiple chat groups, On a first screen that displays the chat timeline of one of the aforementioned groups, multiple first messages that have been posted are displayed. Upon receiving a second message which is a response to any of the first messages, Upon receiving the second message, the multiple messages consisting of the second message and the first message that received the second message are managed as a thread. A second screen, different from the first screen, displays a list of the threads. It is a device, A device that maintains the display order of the plurality of first messages displayed on the first screen in the chat timeline, even when managing a plurality of messages consisting of the second message and the first message that received the second message as the thread.

Images