Programs, methods and devices for message management and document generation on device

Abstract

To make it possible to know a relationship between a first message and a second message.SOLUTION: Systems and methods described herein are directed to facilitate individuals to collect, store, and automatically extract procedural knowledge from their messaging interactions with collaborative groups. Example implementations involve chat interfaces to communicate and add capability to tag, a text and media to organize a content. Example implementations also add a new thread-like structure to a previous only linear time-line of a chat. Knowledge from the chat can be extracted automatically and can be made into a high-quality multimedia document.SELECTED DRAWING: Figure 2(d)

Description

Technical Field

[0001] The present disclosure generally relates to communication platforms and devices, and more specifically to message document generation and message management on devices.

Background Art

[0002] In related art, in messaging applications such as chat, a simple and well - understood interaction metaphor of a conversation timeline is utilized to enable people to share various multimedia. Such related - art applications can assist small task - oriented user groups in coordinating with each other in group chats both in the workplace and in personal life. However, existing applications such as chat can provide functions for communication and content sharing, but it can be difficult to find the content generated and shared over a long period of time in a concise and useful way. Users of related - art implementations can only find a single word and cannot find a series of connected comments or answers such as responses to questions (which may spread across the entire chat in a conventional chat interface).

[0003] In related - art implementations, it may not be possible to organize the content for later use or to facilitate access. If this were possible, it could be useful for small task - oriented groups having a common goal or project.

[0004] Furthermore, many of the devices implementing such messaging applications as chat use displays with limited display space due to the size of the device (e.g., smartphones, tablets, etc.), so an interface that provides chat messages taking into account such limited display space is needed.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Patent Document 3

Non-Patent Documents

[0006]

Non-Patent Document 1

Non-Patent Document 2

Non-Patent Document 3

Non-Patent Document 4

[0007] The technology of the present disclosure provides a message management program based on the relationship between a first message and a second message, a method for managing messages on a mobile device, and a mobile device.

Means for Solving the Problem

[0008] Aspects of the present disclosure are programs that cause a computer to function as means for managing messages for each of a plurality of chat groups, means for displaying a plurality of first messages posted on a first display screen that displays a timeline of a chat in one of the plurality of groups, means for receiving a second message that is a response to any one of the first messages, means for managing, as a thread, a plurality of messages including the second message and the first message to which the second message was received, and means for displaying a list of the threads on a second display screen different from the first display screen.

[0009] Aspects of the present disclosure are methods for managing messages executed by a computer, which manage messages for each of a plurality of chat groups, display a plurality of first messages posted on a first display screen that displays a timeline of a chat in one of the plurality of groups, receive a second message that is a response to any one of the first messages, manage, as a thread, a plurality of messages including the second message and the first message to which the second message was received, and display a list of the threads on a second display screen different from the first display screen.

[0010] Aspects of the present disclosure further relate to a device comprising a processor that manages messages for each of a plurality of chat groups, and displays a plurality of first messages posted on a first display screen that displays the timeline of the chat of one of the plurality of groups. When a second message, which is a response to any of the first messages, is received, a plurality of messages consisting of the second message and the first message to which the second message was received are managed as a thread, and a list of the threads is displayed on a second display screen different from the first display screen.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2(a)

Figure 2(b)

Figure 2(c)

Figure 2(d)

Figure 2(e)

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7(a)

Figure 7(b)

Figure 7(c)

Figure 8

Figure 9

[0012] In the following detailed description, further details of the drawings and implementation examples of the present application are provided. Reference numerals and descriptions of overlapping elements between the drawings are omitted for clarity. The terms used throughout the specification are provided by way of example and are not intended to be limiting. For example, the use of the term "automatically" may include fully automatic or semi-automatic implementations that include the user or administrator controlling a particular aspect of the implementation according to the implementation desired by those skilled in the art in the execution of the implementation of the present application.

[0013] The implementation examples described in this specification relate to a message management and document generation system for devices such as mobile devices. In such devices, messages such as chat messages, for example, may appear in an ad-hoc manner that is not systematized or related in any way other than the time they appeared upon receipt compared to past messages. Such ad-hoc receipt of messages can be particularly problematic in devices with limited screen space such as mobile devices, where the user has to repeatedly scroll through messages to determine the context of the message. In the implementation examples, a message management interface is provided that groups messages by context (such as topics or groups), and assigns tasks and reminders to people related to a specific message context. The implementation examples further provide a system that converts such grouped messages into documents that can be shared among the participants in the message exchange.

[0014] Examples of groups that can benefit from the implementation examples described in this specification include, but are not limited to, caregivers for family (or the elderly), co-caregivers for animals, groups for assisting others or coordinating common business or projects, or members of a distribution team. These groups have the following common characteristics. Namely, solving problems, especially in the case of repetitive tasks where multiple parts of the problem may recur in the future. There are several problems that the group may have to deal with repeatedly. The group may discuss the topic and arrive at a practical solution for the future. A task list that the group has to complete, as well as deadlines that the group members can grasp, may help to reach a common goal. Since not all group members can participate simultaneously, the communication may be about different topics or occur asynchronously. Therefore, the implementation examples can facilitate the extraction of the generated knowledge for future use.

[0015] In the implementation of related technologies, an interface for a messaging application that creates multimedia documents and has a knowledge extraction system that allows users to save and edit content such as how-to instructions, frequently asked questions (FAQs), and summary reports for future use is not provided.

[0016] As an implementation example of the application interface of a message management program, there is an extended chat interface that can form chats as messages and give topics and summaries of questions that can be marked by priority. These elements are highlighted within the chat and displayed as starting points on the start screen. In the implementation example, how-to instructions, FAQs, and reports are also created based on the knowledge accumulated in a chat, which is one of the messages. This can be guided by the user or be semi-automated or automated. In the implementation of existing related technologies, chats cannot be formed. Furthermore, the function of converting the content of chats into a readable document format is not provided in the related technologies.

[0017] Figure 1 shows an overview of the system according to the implementation example. The system 100 can include an extended chat application 101, a document generator 102, and a document viewer 103. The extended chat application 101 is composed of functions that can support task-oriented small groups and is available on mobile devices and desktop computers. The document generator 102 enables the creation of multimedia documents such as how-to instructions, FAQs, and reports. The document viewer 103 is used to filter, view, and play documents that can also be generated as printable reports 104.

[0018] Figure 2(a) shows an extended chat application according to an implementation example of a management program for managing messages such as chats. Chat applications of related technologies can include functions such as sending text, image, video, and voice messages, editing images and videos, sending animated GIFs and emoticons, sending contact cards, and group communication. In the implementation example of the extended chat application, additional elements designed for task-oriented small groups are provided. Such elements can be created during conversations in the chat view. Depending on the chat entry, a dialog opens and the user can enter additional information (such as a deadline, selected group members, effective duration, etc.). Next, the created elements are marked within the chat and / or shown on an extended start / summary screen as shown in Figure 2. This not only shows the group and past chats, but also indicates other elements that require unanswered questions or actions, as well as new elements or messages added to the chat as follows.

[0019] Sub-thread: In the implementation example of a sub-thread, questions and discussions are highlighted during the chat and displayed on the start page of the application until marked as completed (see, for example, "Where can I buy flowers?" at 201 in Figure 2(a)). In the implementation example, the chat can also be divided into a strict timeline-based view to start a thread-like structure of elements. Furthermore, a sub-thread can include other sub-threads depending on the desired implementation. For example, a sub-thread may branch into other sub-threads based on the group participating in the discussion. Threads / sub-threads can have sub-threads, and depending on the desired implementation, a thread chain of any depth can be easily created.

[0020] Note: In the implementation example of the note, the note can be highlighted in the chat or marked by the recipient to hide the note from the start page. Such a note goes with the "sticky note" metaphor and can stay on the start screen until the application ends. In contrast, other discussion elements that are not "sticky" are appropriately placed in the timeline and can be removed from the screen when new messages are added.

[0021] To-Do / Task: In the implementation example, the task can be highlighted in the chat and displayed in the to-do list of an additional tab within the application. The task can be marked as completed by at least one of the specified group members. And depending on the desired implementation, there may be a deadline or a request for feedback (e.g., a photo of the result).

[0022] Message to the Future: In the implementation example, the message can be sent at a certain future time and planned to be delivered only at a specific future time.

[0023] Time-Based Reminder: In the implementation example, a time-based reminder can be created in the same way as a normal message, except that the sender specifies the reminder date / time. This reminder can be configured to be displayed (e.g., immediately) in the recipient's message timeline list, but it can also be configured to be redisplayed as the latest message to the recipient at a specific date / time.

[0024] Location-Based Reminder: In the implementation example for location-based reminders, the sender sets the reminder location instead of the date / time so that the message is redisplayed. For example, when a message is sent as a location-based reminder, the message can be held in the recipient device's queue until the recipient device goes to a specific location. In another implementation example, the message may be sent when the recipient device indicates that it is at that specific location. Other implementations can also be used depending on the desired implementation.

[0025] Time and Location-based Reminder: In an implementation example, it is possible to combine a time reminder and a location reminder so that, in addition to the date / time when the sender wants the message to be displayed, a reminder location can also be set.

[0026] Group Calendar: In an implementation example, a group calendar provides an overview of the activities of individuals and groups. The group calendar may be synchronized with other calendars. Events and appointments can be sent directly from the chat to the calendar.

[0027] Mark Message: In an implementation example, messages posted on the chat timeline can be marked as important or "liked".

[0028] Search: In an implementation example, an interface is provided to facilitate the search for specific words, images, audio, and / or video in messages such as chats. Advanced search methods such as date search and media type search are required for media search.

[0029] Each of the above functions can be sent to a set of group members. · Entire Group: All group members receive the element. · Part of Group: A defined subset of group members receive the element. · Single Member: Only one of the group members receives the element.

[0030] Adding a sub-thread as a question to start a new discussion in a subsection of the chat is a concept not present in related art chat applications. By using sub-threads, it is possible to organize incoming chat messages as received. By doing so, instead of displaying the message upon receipt as in the implementation of related art, it is possible to save space on the screen by associating related messages with the corresponding thread.

[0031] In an implementation example, sub-threads can be created for several reasons. For example, sub-threads make it easier to keep information related to a topic together within a chat and ensure that questions are answered.

[0032] Group members may wish to start a discussion about a topic. In related technology applications, starting a discussion in a chat view is made easier, but over time, messages related to one topic or discussion may be confused with other discussions and chat topics. By creating a sub-thread, the group can focus on one topic without the need to refer to or quote past answers that might otherwise wander into other topics and discussions.

[0033] Furthermore, one problem regarding questions and answers in a group chat is that the response to a question may be separated from the original question and confused with messages from other topics. For example, even if a user asks a question during a chat conversation, the person who can answer the question may not be online, and if the chat conversation continues, the question may deviate from the center of the topic until a particular person comes back online. It is also possible for a user to ask multiple questions in consecutive messages before other users can answer. One or more users in the group may start answering one question, and due to the deviation of the topic center, other questions may be forgotten. By creating a sub-thread for a question, the question can be marked and shown during the chat and displayed as a new item on the home screen (which is the entry point when the user opens the application), so that the question cannot be forgotten.

[0034] Figure 2(b) shows a chat application according to an implementation example. In particular, Figure 2(b) shows an example of a screen for generating special elements. In the implementation example, a sub-thread can be created when creating a message. The sub-thread can be created when creating a new message or by raising the level of a message that has already been posted. When a user inputs a message (or posts a file) before sending it, the message can be marked on the element marking screen 210 as a special element (such as a sub-thread) as shown in Figure 2(b). It is also possible to create a special element and write text into the sub-thread.

[0035] In the implementation example, a message that has already been posted can be raised in level without using a follow-up message. For example, a user (not necessarily the creator of the entry) can perform an action (such as a long-press operation on the touch screen) on an element in the chat. A menu opens the element marking screen 210 as shown in Figure 2(b), where the entry can be marked as a special element.

[0036] In the implementation example, a message that has already been posted can be raised in level by manually selecting a follow-up message. In the implementation example, a user (not necessarily the creator of the entry) can perform an action (such as a long-press operation on the touch screen) on an element in the chat. A menu opens the element marking screen 210 as shown in Figure 2(b), where the entry can be marked as a special element. Then the user can proceed to the next screen and select an existing entry to be part of the new sub-thread.

[0037] In the implementation example, a message that has already been posted can be promoted by semi-automatically selecting a follow-up message. In the implementation example, a user (for example, not necessarily the creator of the entry) can perform an action (such as a long-press operation on a touch screen) on an element within the chat. A menu opens the element marking screen 210 as shown in FIG. 2(b), where an entry can be marked as a special element. Then the user can proceed to the next screen and correct the pre-selected entry that is to be part of a new sub-thread.

[0038] Sub-threads can be viewed in a variety of different ways, such as through the interfaces of threads and sub-threads, a color view, or a split view of two screens. Viewers can use the chat application to switch between different displays. The displays can be different representations of the same information.

[0039] In an implementation example including the interface of a thread and a sub-thread as shown in FIG. 2(c), the interface of the thread and the sub-thread strictly decomposes the chat into a timeline-based paradigm, and shows the sub-thread in the chat window. A reply or answer to the sub-thread can be added by selecting the "+" at the end of the sub-thread, which causes an entry to be added at the end of the target sub-thread rather than at the end of the entire chat. Using this paradigm, a new message is not necessarily displayed at the end of the entire chat that is marked as unread after a certain point. The message may spread throughout all the threads of the entire chat. In such an implementation, further, a hint to the user that a new entry may be somewhere in the chat may be utilized. An example of such a hint is a warning sign (e.g., a button prompting the user's attention) in the top bar, which indicates how many new entries are in the entire chat and can also function as a button that the user can use to jump through the chat from one new entry to another. An implementation example of the interface of the thread and the sub-thread is shown in FIG. 2(c). In the overview screen of FIG. 2(a), the number of new entries regarding the question is shown beside the question. In the implementation example, when the question is selected, it jumps to the last unread message of the sub-thread in the chat.

[0040] In an embodiment, the color view maintains the chat's timeline-based paradigm, and sub-threads mark different entries with different colors. A reply or answer to a sub-thread can be added by clicking on the "+" next to the question, with the entry marked in the same color as the question. Using this paradigm, new messages are displayed at the end of the entire chat, marked as unread after a certain point. However, the question may be spread out somewhere in the chat. For this reason, an implementation example may include a button for the user to jump during the chat from one question to another. In an implementation example, it is also possible to mark a new entry as part of one sub-thread as shown in FIG. 2(d) using a gesture (e.g., long press) on the last answer of the sub-thread. As shown in FIG. 2(d), related messages are color-coded, and when entered into a plus sign or other desired user interface indicator, the device scans for messages of the same color and displays or highlights the next message with the same color based on the timestamp. On the summary screen, the number of new entries regarding a question is shown next to the question. Selecting a question jumps to the first unread answer to that question in the chat.

[0041] In the implementation example of the two-screen view, the view can be implemented in the same way as the thread and sub-thread interfaces. Further, two screen views mark the start of the sub-thread with an icon to maintain the chat timeline-based paradigm. A reply or answer to the sub-thread can be read by tapping on the sub-thread, which leads the user to a new screen showing that sub-thread and its follow-up messages. Using this paradigm, a new follow-up message for this sub-thread can be input as a normal message on the sub-screen by simply typing the text input at the bottom of the screen as shown in Fig. 2(e) without first selecting a special button. On the summary screen, the number of new entries related to the question is shown beside the question. Selecting the question jumps to the view of the sub-thread. As shown in Fig. 2(e), a separate screen is generated to represent the sub-thread for the thread. Further, it is possible to give the same interface within the separate screen, thereby facilitating the implementation of sub-threads within sub-threads and the generation of separate screens for sub-threads within sub-threads according to the desired implementation.

[0042] In the implementation example, there is a document generator that can be added to the chat application. The newly introduced function above adds a further level of annotation to the previous linear chat. The difference in the way of marking elements gives a hint of importance. However, from the message exchange such as a longer chat, it may become difficult to find information later. Here, not only marking elements, but also extracting a set of semantically related elements containing knowledge from that chat and converting it into an easier-to-read form such as a how-to guide, FAQ, or report can be assisted. The elements can be selected based on the user's selection or pre-selected by annotations (e.g., the type of element, star marks / markers given by the user over time) according to the document to be created. Below, the implementation example facilitates the creation of multimedia documents from the chat application.

[0043] In an implementation example, the multimedia document may be user-guided. In user-guided creation, the interface facilitates the maker's performance of several steps. First, a chat element is selected via the interface. This process is initiated when the user clicks on one chat entry. Next, a selection tool provides a list of subsequent elements within the application. Thereafter, the user can select for each element whether it should be added to the document. Then, the selected elements are copied to an editing area where they can be rearranged, edited, and merged with previous or subsequent elements. If necessary, new elements can be added from the local file system or the Internet, and formatting is applicable. Finally, the document can be exported in the desired format.

[0044] In an implementation example, the multimedia document can be generated by a semi-automatic process. In semi-automatic creation, after the user selects a selection start point within the chat, several algorithms are applied. Basic text analysis as well as analysis of annotations (e.g., like, marker) suggest relevant entries. This may be changed by the user. In the copy process, unwanted parts of the text (e.g., emoticons, unwanted spaces) are removed, and elements that the user posted as consecutive small messages are merged into one text. The resulting list of elements can then be edited as described in the user-guided process. Here, the user receives the assistance of search and investigation algorithms that provide video scenes, images, or other materials from the Internet.

[0045] In an implementation example, the multimedia document can be generated by an automatic process. Automatic creation can also be configured to learn from the user-guided and semi-automatic modes and automatically export the desired document format using learning algorithms such as machine learning algorithms.

[0046] In addition to linear multimedia documents, hypermedia graphs can also be created using the document creation interface. Hypermedia graphs require a graph view and an editor for the logic statements applied to the forks within the graph. In such an implementation example, hypermedia documents, which can provide different levels of detail, difficulty, or instructions for different user groups, can be created especially on mobile devices. The mobile interface provides an overview of the document and the interactive elements of existing documents. Here, it is possible to expand this document to perform hypermedia creation possible on mobile devices. By recording information in the APP, it is possible to add more detailed information as well as different types of media elements to provide more information.

[0047] More complex subprocesses during document generation are the selection of subsequent entries, text cleaning, and reordering of entries. These components are built on the implementation of related techniques for topic detection and tracking to derive the initial topic clustering of chat messages and, as a result, track the detected topics. Such techniques have also been applied to news streams and email composition. Techniques such as keyphrase extraction and discourse dynamics analysis can also be used to filter out low-importance posts and to structure the generated documents.

[0048] In the implementation example, the multimedia document viewer provides the function of displaying all types of media and navigation elements. The viewer can adapt to the playback device and user characteristics by filtering the hypermedia graph. Depending on the playback device, the data volume can be adjusted along with the size of the media elements. In the implementation example, those with slow connections may focus on text and images. On the other hand, videos are displayed using a high bandwidth (when different types of media can be used). Furthermore, the report generator can provide a filtering function to adapt the resulting document according to the user's input and purpose.

[0049] Furthermore, if there is missing information or it is not presented as intended, a function is available that allows the user to expand the document being viewed and make passive viewers co-authors. Such a function can operate in different modes.

[0050] In an implementation example of the non-controlled mode, the content can be edited by the user without an approval process.

[0051] In an implementation example of the author control mode, when the user adds / edits content, a notification is sent to the multimedia / hypermedia author. If the author agrees, the author can approve the changes. If the author does not agree, the content can be rejected or a discussion can be started.

[0052] In an implementation example of the user control mode, when the user adds / edits content, the new content is marked as unverified. Then other users can peer review the content, change the content, and vote on whether the content can be approved as "regular" content for the document This ensures the good quality of new information and the continuous improvement of the document.

[0053] Figure 3 is an overall flowchart according to an implementation example. In the implementation example, the system can be divided into three sub-processes (content collection, hypermedia document generation, hypermedia preparation). This will be described in detail below as shown in Figure 3.

[0054] At 301, the system is configured to collect content from an extended chat application as described in FIGS. 2(a) to 2(e). Such content may include text, images, videos, audio, or files posted via the chat application. At 302, after collecting the content, a hypermedia document can be generated from the content. At 303, the hypermedia document is provided to a display interface for the user. A more detailed flow is shown in FIG. 4.

[0055] FIG. 4 shows a data flow according to an implementation example. Specifically, FIG. 4 shows the data flow between an extended chat application 401, a document generator 402, and a document viewer / report 403. In the extended chat application 401, the user sends a message at 410. This is sent to the extended chat UI at 411. If the message is only text, the text is added to the database and the connection between entries is updated at 412. If the message is a media file, the media file is stored on the server at 413 and a link is added to the database. Then the connection between entries is updated. Then the chat is updated for all users.

[0056] In the document generator 402, the user who creates a document at 420 opens an entry selector at 421 to select a start entry, and then is guided to the hypermedia document editor 422 at 422. The hypermedia document editor is equipped with an interface and the content can be edited, and content from the Internet can be added as shown at 423. The content is linked together in the hypermedia document. Regarding the chat, the media file is stored in the file server, and the text and structure are stored in the database. The user can select a document and add further input. Then an interactive hypermedia document is created and displayed in the viewer or the content is exported as a document file (e.g., PDF, DOC).

[0057] In the document viewer / report 403, the user who views a document at 430 opens the hypermedia document viewer 431. This displays an interactive hypermedia document at 432. The interactive hypermedia document 432 is created by the hypermedia document assembler 433. This is also configured to generate a document file for the user at 434 after instructing the hypermedia document viewer 431 for the user to construct a document.

[0058] FIG. 5 is a diagram showing a flow of content collection according to an implementation example. Specifically, FIG. 5 shows a process during content collection (for example, a chat application). The selection of a chat / sub-chat is received on the interface via the user at 501. At 502, the system continues to add content. During this flow, the system may receive text or media typed via an extended chat application. At 503, the elements may be marked as shown in the interface of FIG. 2(b). As shown in FIG. 2(b), the elements can be marked as questions / discussions, ToDo, memo, reminder, event, future message, etc. Particularly when the elements are not marked (No), the flow proceeds to 504 and it is determined whether the content generation has ended. If marked (Yes), the flow ends, and if not (No), the flow proceeds to 502 and additional content is processed.

[0059] At 505, a check is performed to determine whether the marked element is an instruction to start a question or discussion. If so (Yes), the flow proceeds to 506, where the element is marked as the start of a question or discussion, and then proceeds to 507 to set the visibility. At 507, an interface can be provided to facilitate setting the visibility of a specific element. In the implementation example, there can be three visibility settings: single user, part of a group, and the entire group. When a single user or part of a group is selected, the corresponding subset of the group is selected.

[0060] If not (No), the flow proceeds to 508, where a check is performed to determine whether a particular element is a ToDo element. If so (Yes), the flow proceeds to 509, where the element is marked as a ToDo element. Then the flow proceeds to 510, where a deadline is set. Here, an interface is provided that facilitates the selection of whether the element is continuously valid or the date and time by which the element must be completed. The provided interface can also include an interface that facilitates reminder setting at 511, where the interface facilitates the selection of when to set the reminder and how far in advance of the deadline the reminder must open the chat application. Further, the interface can facilitate the setting of an action at 512. This can be a requirement for taking a photo, answering a question, filling in a value, obtaining a video or some other type of documentation from the recipient, according to the desired implementation.

[0061] If not (No), the flow proceeds to 513, where a check is performed to determine whether a particular element is a memo element. If so (Yes), the flow proceeds to 514, where the element is marked as a memo. The flow then proceeds to 515, where validity is set via an interface. Here, an interface is provided that can facilitate the selection of the length of time indicating how valid the element is and is displayed to other users.

[0062] If not (No), the flow proceeds to 516, where a check is performed to determine whether a particular element is directed to a reminder. If so (Yes), the flow proceeds to 517, where the element is marked as a reminder. The flow then proceeds to 518, where the type of reminder is determined. If the reminder is a location- and time-based reminder (Yes), the flow proceeds to 524, where the location and other reminder information are set via an interface. In an implementation example, the location can be input either by typing an address or by marking the location on a map.

[0063] If not (No), the flow proceeds to 519 to determine whether a particular element is a time-based reminder. If so (Yes), the flow proceeds to 525 where an interface is provided, the start date is set, and then it proceeds to 526 to set the time for the element. If not (No), the flow proceeds to 520 to determine whether the particular element is a location-based reminder. If so (Yes), the flow proceeds to 521 to set the location in the same way as the flow in 524. If not (No), the flow proceeds to 507 to set the visibility.

[0064] If at 516 the particular element is not a reminder (No), the flow proceeds to 522 where a determination is made as to whether the element is directed to an event. If so (Yes), the flow proceeds to 523 to mark the element as an event. The flow then proceeds to 524 to obtain and set the location of the event and proceeds to obtain and set other information such as the date and time of the event according to the desired implementation.

[0065] If the particular element is not an event (No), the flow proceeds to 527 to mark the element as a future message. Then it proceeds to 525 to set the date and then to 526 to set the time.

[0066] In another implementation, after the content within the extended chat application is collected as shown at 301 in FIG. 3, a hypermedia document may be generated at 302 and provided to a document viewer at 303. FIG. 6 shows a flowchart of document generation according to an implementation example. At 601, a determination is made as to whether to make the document generation a user-guided generation process (e.g., manual) or a semi-automatic process.

[0067] In the user guidance document generation process (Yes), an interface is provided at 613 to facilitate the selection of chat inputs. The inputs are selected by marking each of them via the interface. At 614, a user confirmation or interface can be provided to determine whether the input content is acceptable. If not (No), the flow proceeds to 615, where an interface is provided to facilitate the editing of chat inputs. In such an interface, the input can be changed, for example, typos, emojis, or unwanted text can be deleted. It is also possible to add or delete text.

[0068] At 616, a user confirmation or interface is provided to determine whether the entry order is acceptable. If not (No), the flow proceeds to 617, where an interface is provided to facilitate the rearrangement of the chat input order according to the desired document. In an implementation example, there may be a situation where the entry order is incorrect for the chat timeline. If necessary, a single entry can also be order-changed.

[0069] At 618, a user confirmation or interface is provided to determine whether additional content should be added. If so (Yes), the flow proceeds to 619, where an interface is provided to add additional content. If information is missing, or if the producer knows better illustrations than those collected during the chat, external information can be added according to the desired proof.

[0070] At 620, a user confirmation or interface is provided to determine whether a hyperstructure should be added. If it should be added (Yes), the flow proceeds to 621, where hyperlinks can be added. In an implementation example, the hyperstructure can be added to provide different versions of the content or different paths via the document.

[0071] If document generation is performed via a semi - automatic process (No), the flow proceeds to 602, where an interface is provided to facilitate the selection of a starting entry from the chat. In either the automatic or semi - automatic mode, when one starting entry is selected, content generation is initiated by selecting the entries following the starting entry at 603.

[0072] At 604, 608, 612, check the previously set decision regarding whether a semi - automatic generation process is desired. If so (Yes), several interfaces are provided. These may include the following.

[0073] At 605, an interface for selection correction can be provided, where the automatic selection result at 603 can be corrected using the same interface as used in the user - guided mode. If the selection is OK (yes), the flow proceeds to 607, where the text is automatically cleaned as described, for example, at 615. Otherwise (No), the flow proceeds to 606, where the entry selection is corrected according to the instructions from the user via the interface.

[0074] At 609, an interface for text correction can be provided, where the automatic text cleaning result can be corrected using the same interface as used in the user - guided mode. If the text is OK (yes), the flow proceeds to 611, where the entries are automatically rearranged as described, for example, at 617. Otherwise (No), a text cleaning process as shown at 610 is provided.

[0075] Next, at 622, the document structure is exported to the database 412 and file storage 413 from which the document was retrieved by the document viewer. When the automatic generation is complete, or when the user terminates the user - guided generation or semi - automatic generation, the document structure is exported to the viewer along with the document content.

[0076] Figures 7(a) to 7(c) show examples of a document generation interface for a chat application according to an implementation example. In the interface for the user guidance process, a screen is provided to first let the user define the type of document (e.g., how-to, FAQ, report, etc.) that the user intends to create, as shown in Fig. 7(a). Next, according to the document, structural elements can be provided to a template according to the desired implementation. For example, the step and sub-step elements are part of the how-to instruction template as shown in Fig. 7(b), the question elements are part of the FAQ template, and elements such as title, related person, introduction / purpose, time span, progress, etc. are part of the report template. The steps can be typed in by the user or selected from existing elements. Contents from chat input, todo memos, memos, or other special elements from chat messages can be used to serve as the template as shown in Fig. 7(b). In the manual mode, the user selects these inputs on a selection screen as shown in Fig. 7(c). After adding the contents, the contents can be rearranged and edited. The completed document can be extended using other versions that create hypermedia documents in this process.

[0077] Figure 8 is a flowchart of document preparation and generation according to an implementation example. At 801, an interface is provided to facilitate document selection. For example, the user can select a document on the summary tab. At 802, a determination is made as to whether the output device is a printer. If so (Yes), the flow proceeds to 803, where the document is generated from a multimedia document. In the implementation example, key frames are extracted for each video and added to the document. Text is extracted for each audio and added to the document. At 804, the document is printed. If the printer is not available, it is downloaded to a local device.

[0078] If the output device is not a printer (No), it is determined at 805 whether other versions of the document are available. If so (Yes), an interface is provided at 806 to obtain input variables. In an implementation example, the input variables for the interface for customizing the document can include screen resolution, available bandwidth, device language, user knowledge, user gender, desired level of interactivity (e.g., low / high), paths taken by other users from the same document, and others depending on the desired implementation. Some of the input variables can be automatically determined from the device, some are based on knowledge about the user, and some may require user input (e.g., at the start of use). The flow proceeds to 807 where the document is customized based on the input variables. The content is adapted so that paths within the hypermedia document can be preselected.

[0079] At 808, the document is loaded into the viewer and the user can monitor or expand the document. The expansion process can be either uncontrollable, or controlled by the producer, or controlled by the user.

[0080] In an implementation example, there can be content collection and chat applications. The implementation example facilitates the connection and synchronization of several tools that provide a very wide variety of functions. In addition to searching within the chat, it is also possible to search for shared files that are indexed and once saved within the application. It is also possible to mark messages as future reminders. This enables, in the implementation example, structuring the chat directly from the chat view or marking chat elements as important elements.

[0081] FIG. 9 illustrates an exemplary computing environment having an exemplary computer device suitable for use in an implementation example. The computer device 905 within the computing environment 900 can include one or more processing units, cores, processors 910, memory 915 (e.g., RAM, ROM, and / or the like), internal storage 920 (e.g., magnetic storage, optical storage, solid state storage, and / or organic storage), and / or an I / O interface 925, etc., and any of them can be connectable to a communication mechanism or bus 930 for information communication, or can be embedded in the computer device 905.

[0082] The computer device 905 can be communicatively coupled to an input / user interface 935 and an output device / interface 940. Either one or both of the input / user interface 935 and the output device / interface 940 can be a wired or wireless interface and can be removable. The input / user interface 935 can be physical or virtual and can include any device, component, sensor, or interface (e.g., buttons, touch screen interface, keyboard, pointing / cursor controller, microphone, camera, braille device, motion sensor, optical reader, etc.) that can be used to provide an input. The output device / interface 940 can include a display, television, monitor, printer, speaker, braille device, etc. In some implementation examples, the input / user interface 935 and the output device / interface 940 can be embedded in or physically connected to the computer device 905. In another implementation example, other computer devices can function as or provide the input / user interface 935 and the output device / interface 940 for the computer device 905. In implementation examples including a touch screen display, a television display, or any other form of display, the display is configured to provide a user interface as shown, for example, in FIGS. 2(a) - 2(e) and FIGS. 7(a) - 7(c).

[0083] Examples of computer device 905 include, but are not limited to, advanced mobile devices (e.g., smartphones, devices installed in vehicles and other machines, devices carried by humans or animals, etc.), mobile devices (e.g., tablets, notebook computers, laptop computers, personal computers, portable TVs, portable radios, etc.), and non-mobile devices (e.g., desktop computers, other computers, information kiosks, TVs, radios, etc. having one or more processors embedded and / or connected therein).

[0084] Computer device 905 may be communicatively connected to external storage 945 and network 950 (e.g., via I / O interface 925) to communicate with any number of network-connected components, devices, and systems including one or more computer devices of the same or different configurations. Computer device 905 or any connected computer device may function as, provide services as, or be referred to as a server, client, synserver, general-purpose machine, special-purpose machine, or another label.

[0085] I / O interface 925 includes wired and / or wireless interfaces that utilize any communication or I / O protocol or standard (e.g., Ethernet (registered trademark), 802.11x, Universal System Bus, WiMax, modem, cellular network protocol, etc.) for communicating information to and / or from at least all connected components, devices, and networks in computing environment 900. However, it is not limited thereto. Network 950 may be any network or combination of networks (e.g., the Internet, local area network, wide area network, telephone network, cellular network, satellite network, etc.).

[0086] The computer device 905 can be used for use and / or communication using a computer-usable medium or a computer-readable medium, including a temporary medium and a non-temporary medium. The temporary medium includes transmission media (e.g., metal cables, fiber optic materials), signals, carrier waves, etc. The non-temporary medium includes magnetic media (e.g., disks and tapes), optical media (e.g., CD ROM, digital video disks, Blu-ray disks), solid-state element media (e.g., RAM, ROM, flash memory, solid-state storage), and other non-volatile storage or memory.

[0087] The computer device 905 can be used in the implementation of technologies, methods, applications, processes, or computer-executable instructions in some exemplary computing environments. The computer-executable instructions can be retrieved from a temporary medium and stored and retrieved in a non-temporary medium. The executable instructions can originate from one or more of any programming language, scripting language, and machine language (e.g., C, C++, C#, Java®, Visual Basic, Python, Perl, JavaScript®, etc.).

[0088] The memory 915 can be configured to store or manage algorithms executed by the processor 910, such as those described in the flows in FIGS. 1, 3-6, 8. The implementation examples described herein can be executed alone or in any combination with each other according to the desired implementation and are not limited to a specific implementation example.

[0089] Processor 910 is executable under any operating system (OS) (not shown) in a native environment or a virtual environment. One or more applications can be deployed, including logical unit 960, application programming interface (API) unit 965, input unit 970, output unit 975, and an inter-unit communication mechanism 995 for the different units to communicate with each other, with the OS, and with other applications (not shown). The above units and elements can be modified in terms of design, function, configuration, or implementation and are not limited to the descriptions herein. Processor 910 may be a physical processor or a central processing unit (CPU) configured to execute instructions loaded from memory 915.

[0090] In some implementation examples, when information or execution instructions are received by API unit 965, they can be communicated to one or more other units (e.g., logical unit 960, input unit 970, output unit 975). In some examples, logical unit 960 can be configured to control the flow of information between units and direct the operations provided by API unit 965, input unit 970, and output unit 975 in some of the above implementation examples. For example, the flow of one or more processes or implementations may be controlled by logical unit 960 alone or in cooperation with API unit 965. Input unit 970 can be configured to obtain inputs for the calculations described in the implementation examples, and output unit 975 can be configured to provide outputs based on the calculations described in the implementation examples.

[0091] In an implementation example, the processor 910 may be configured to display a first message transmitted from a mobile device on a user interface. In the implementation example shown in FIG. 2(a), a message 201 transmitted from a mobile device to another device at 201 is placed on the display, and the processor 910 monitors one or more second messages from other mobile devices in response to the first message transmitted from the mobile device. When a second message is received from another mobile device, the processor 910 is configured to generate a user interface indicator on the first message displayed on the user interface to provide a relationship interface between the first message and the second message on the user interface. The user interface indicator can be configured to provide such a relationship interface as one of an interface of threads and sub-threads as shown in FIG. 2(c), a color-coded relationship interface as shown in FIG. 2(d), and generation of a separate screen as shown in FIG. 2(e). As shown in FIGS. 2(c) to 2(e), such an indicator can take the form of a plus sign, an arrow, a Q symbol, etc., depending on the desired implementation.

[0092] In the example of FIG. 2(c), when the user interface indicator is configured to provide the related interface as the interface of the thread and sub-threads, the processor 910 is configured to generate a user interface indicator by displaying a first message as the start of the thread, providing the user interface indicator as an indication of thread expansion (e.g., in the form of a Q symbol, a plus symbol, etc.), and when an input is received on the user interface indicator, displaying a second message as a sub-thread on the user interface through thread expansion as illustrated by the response to the first message shown in FIG. 2(c). Further, this implementation example is expandable to a plurality of messages as shown in the example of the plurality of transmission messages in FIG. 2(a). Here, a plurality of messages related to the transmission message can be displayed on the user interface as sub-threads through thread expansion as shown in FIG. 2(c).

[0093] In the example of FIG. 2(d), when the user interface indicator is configured to provide the related interface as a color-coded related interface, the processor 910 is configured to generate a user interface indicator by displaying a first message in a first color, providing the user interface indicator as an indication to move to the next message having the first color, and when an input is received on the user interface indicator, displaying a second message in the first color as the next message. A message displayed in a second color different from the first color is recognized as being in a different relationship from the first message.

[0094] In the example of FIG. 2(e), when the user interface indicator is configured to provide the relationship interface as a separate screen, the processor 910 is configured to generate the user interface indicator by providing the relationship interface as the generation of the separate screen. The processor 910 is configured to generate the user interface indicator by displaying a first message on a first display screen, providing the user interface indicator as a change instruction (e.g., an arrow) to a second screen, and when an input is received to the user interface indicator, changing the first display screen to the second display screen and displaying a second message on the second display screen as shown on the right side of FIG. 2(e).

[0095] In an implementation example, the first message may be in the form of a location-based reminder. This is configured to be transmitted to another mobile device when the other mobile device reaches a location related to the location-based reminder. In the implementation example, the message can be transmitted to the other mobile device, but it is not displayed on the other mobile device until the other mobile device detects that it is at the location indicated by the first message at a specified time. In the implementation example, the message may be either time reminder-based only or location reminder-based only depending on the desired implementation.

[0096] In the implementation example, the first message can request a task to be completed. And the second message can include at least one of text, video, and image corresponding to the task to be completed. That is, the user of the mobile device can indicate that an image or video is required to process the task, and when receiving the image or video, the user can confirm whether the task is fulfilled. When the user determines that the task is fulfilled, the user can delete the message from the other mobile device through a confirmation interface (e.g., a reply message, a user interface indicating task completion, etc.).

[0097] In an implementation example, the processor 910 can be configured to provide a second user interface indicator, which consists of an interface that generates a document based on a first message and a second message as shown in FIGS. 7(a) to 7(c). When receiving an input to the second user interface indicator, the processor 910 performs a selection process of the first message and the second message on a document editor as shown in FIGS. 7(a) to 7(c), and is configured to generate a document for the selected first message and second message that has been processed by the document editor as described in the flow of FIG. 8. For example, as described in FIGS. 4 and 8, the processor 910 is configured to generate a document for the selected first message and second message by retrieving one or more images related to the second message and incorporating the images into the document (for example, by downloading image or video key frames from the Internet or a message server). In an implementation example, the processor 910 is configured to generate a document for the selected first message and second message by retrieving one or more videos related to the second message, extracting one or more key frame images from the one or more videos, and incorporating the one or more key frame images into the document. The extraction of the key frame images can be performed according to any desired implementation well known in the art. Further, the processor 910 can be configured to generate a document by printing the document on a printer as shown in FIG. 8.

[0098] Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations within a computer. These algorithmic descriptions and symbolic representations are the means used by those skilled in the data processing arts to convey the substance of their innovations to others skilled in the art. An algorithm is a defined sequence of steps leading to a desired end state or result. In an implementation example, the steps executed require physical operations of tangible quantities to achieve a sure result.

[0099] Unless otherwise specified, as will be apparent from the discussion, throughout this description, discussions using terms such as "processing," "computing," "calculating," "determining," "displaying," etc., may include operations and transformations of data represented as physical (electronic) quantities within the registers and memories of a computer system, into other data similarly represented as physical quantities within the memories or registers of the computer system or other information storage, transmission, or display devices. It should be understood that this may include the operation and processing of a computer system or other information processing device.

[0100] Implementation examples may also relate to apparatus for performing the operations described herein. This apparatus may be specially constructed for the required purposes, or may include one or more general purpose computers selectively activated or reconfigured by a computer program. Such a computer program may be stored on a computer readable medium such as a computer readable storage medium or a computer readable signal medium. Computer readable storage media may include, but are not limited to, tangible media such as optical disks, magnetic disks, read only memory, random access memory, solid state devices, and solid state drives, or any other type of tangible or non-transitory media suitable for storing electronic information. Computer readable signal media may include media such as carrier waves. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. A computer program may include a pure software implementation containing instructions to perform the desired implementation operations.

[0101] A variety of general-purpose systems can be used with the programs (management programs) and modules according to the embodiments described herein. Alternatively, it may be convenient to construct a more specialized apparatus to perform the desired method steps. Further, the implementation examples are not described with reference to a specific programming language. It should be understood that the teachings of the implementation examples described herein can be implemented using a variety of programming languages. The instructions of the programming language are executable by one or more processing devices, such as, for example, a central processing unit (CPU), a processor, or a controller.

[0102] As is well known in the art, the above operations can be performed by hardware, software, or some combination of software and hardware. Various aspects of the implementation examples can be implemented using circuits and logic devices (hardware). On the other hand, other aspects can be implemented using instructions stored on a machine-readable medium (software). When executed by a processor, this will cause the processor to execute a method for implementing the present application. Further, some implementation examples of the present application may be executed by hardware only, and other implementation examples may be executed by software only. Furthermore, the various functions described may be performed by a single unit or may be distributed among a plurality of components in any number of ways. When executed by software, the method can be executed by a processor, such as a general-purpose computer, based on instructions stored on a computer-readable medium. Optionally, the instructions can be stored on the medium in a compressed format or an encrypted format.

[0103] Furthermore, other implementation aspects of the present application will be apparent to those skilled in the art from the consideration of this specification and the practice of the teachings of the present application. The various aspects and / or components of the described implementation examples can be used alone or in any combination. The specification and the implementation examples are considered to be merely illustrative, and it is intended that the true scope and spirit of the present application be shown in the appended claims.

Description of the Reference Numerals

[0104] 101 Extended Chat Application 201 Message 905 Computer Device 910 Processor

Claims

1. A computer, means for managing messages for each of a plurality of chat groups, means for displaying a plurality of first messages posted on a first display screen for displaying a timeline of a chat of one of the plurality of groups, means for receiving a second message that is a response to any one of the first messages, means for managing, as a thread, a plurality of messages consisting of the second message and the first message to which the second message was received when the second message is received, and means for displaying a list of the threads on a second display screen different from the first display screen, A program for causing the computer to function as such.

2. The program according to claim 1, wherein on the second display screen, the thread is displayed so that the chat group to which the thread belongs can be identified.

3. The program according to claim 1, wherein on the second display screen, each of the threads is displayed in a manner that allows the first message constituting the thread to be recognized.

4. The program according to claim 1, wherein on the second display screen, for a thread in which an unread second message exists, the presence of an unread message is indicated.

5. The program according to claim 4, wherein when a thread in which an unread second message exists is selected on the second display screen, the program jumps to the first unread message constituting the thread.

6. The program according to claim 4, wherein when a thread in which an unread second message exists is selected on the second display screen, the program jumps to the last unread message constituting the thread.

7. A method for managing messages, which is executed by a computer, comprising: managing messages for each of a plurality of chat groups; displaying a plurality of first messages posted on a first display screen for displaying a timeline of a chat of one of the plurality of groups; receiving a second message that is a response to any one of the first messages; Upon receiving the second message, a plurality of messages including the second message and the first message that received the second message are managed as a thread. On a second display screen different from the first display screen, a list of the threads is displayed. Method. **Claim 8** Comprising a processor, the processor: Manages messages for each of a plurality of chat groups; Displays a plurality of first messages posted on a first display screen that displays a timeline of a chat in one of the plurality of groups; Receives a second message that is a response to any one of the first messages; Upon receiving the second message, a plurality of messages including the second message and the first message that received the second message are managed as a thread; On a second display screen different from the first display screen, a list of the threads is displayed. Device.

Images