Message processing and reception methods

Abstract

A method for processing a first message received from a first terminal, said method being implemented by a second terminal and comprising: - a step of generating a second message from the first message, said second message comprising a drop-down line of content from the first message, at least one pair of markers designating, within said drop-down line, at least one segment of the content of the first message, an enrichment element associated with said segment of the content of the first message, and an instruction for formatting the display of said second message, and - a step of sending the second message to at least one recipient terminal. Figure 2 for the abbreviation

Description

Title of the invention: Methods for processing and receiving a message. Field of the invention

[0001] This invention relates to the field of interpersonal communications.

[0002] More specifically, the invention relates to a method and device for processing a first message in order to provide a second enriched message derived from this first message (for example, a reply to the first message), this method and device being intended to be implemented within communication systems such as electronic messaging systems. Previous art

[0003] Interpersonal communication tools, whether instant messaging services (such as WhatsApp®, Signal®, etc.) or email clients (such as Outlook®, Gmail®, etc.), have contributed not only to the increase in the volume of messages exchanged but also, through their functionality, to the increased complexity of the form of the messages themselves. For example, messages can be sent to a single person or to a group of people. It is also possible to send text, audio, video messages, or messages combining several of these modes (referred to herein as multimodal messages). Reply messages can include previous messages presented in the form of a discussion, etc.

[0004] This variety of forms, the addition of temporal content (i.e. content natively associated with a determined timeline, such as audio or video content), the quantity and frequency of messages tend to make communications paradoxically more complex, especially when it comes to following a thread of conversations between two or more correspondents.

[0005] Furthermore, in the particular case of messages including or based on temporal content, communication is all the more complicated as this temporal content does not lend itself easily to quick or fragmentary consultation (i.e. of only a part of this content) as a text can be.

[0006] There is a certain disparity in the possibilities offered by current communication systems depending on the modes of communication (text, audio and video) or the types of messaging (email services, instant messaging).

[0007] For example, when replying to an email, a user can reply in the body of the text or copy and paste into their reply sub-sections of the original text to which they wish to respond. The chosen method (reply in the body) (text, copy / paste into the reply text, etc.) to respond to specific elements of the received message will be different from one user to another, which may represent difficulties in understanding for the recipient of the reply.

[0008] In the case of other types of messaging, replying to a sub-part of a message may be more complicated or even impossible.

[0009] For example, in instant messaging, messages are commonly presented in threaded conversations. It is possible to select a message within a thread and reply specifically to that message, which is then quoted in the reply. For example, the quoted message appears in a box at the top of the reply, the background of which has a different color than the reply itself, in order to distinguish the quoted message from the reply.

[0010] However, it is not possible to refer to only a sub-section of a message, except by manually copying or copying and pasting that sub-section. Furthermore, to distinguish the sub-section in question from the original text of their reply, a user must insert markers, which can be of various types (for example, quotation marks, or a phrase such as "you write"...) or use color coding (when possible), before inserting their own comments or reply elements. This situation is very similar to that of a user wishing to reply to an email by referring to passages of the original message and therefore presents the same limitations.

[0011] The situation is more problematic for voice and / or video messages, for which these possibilities are absent.

[0012] By way of illustration, an audio / video message can be included in the thread of a discussion via instant messaging. The instant messaging application then acts as a player, allowing the user to listen to or view the video / audio message and then reply to it, without, however, being able to directly reference a particular passage of the audio / video content. In other examples, it is necessary to first download the file in question, or to go to an external site indicated by a link contained in the received message, and then return to the messaging application to reply to the message (again, without being able to directly reference a particular passage of the message).

[0013] The same disadvantages are observed in the case of digital answering machine type audio messaging.

[0014] All of these solutions therefore present a twofold drawback: on the one hand, they complicate the enrichment of specific sub-parts of a pre-existing message (such as formulating a reply message to a specific sub-part of a message, or transferring said enriched message, etc.), and on the other hand, they result in messages that are difficult to read and understand for their recipient (where part of the message consists of explaining the context of the reply message and what it refers to). Description of the invention

[0015] The invention remedies these drawbacks in particular by proposing a method for processing a first message received from a first terminal, said method being implemented by a second terminal and comprising: - a step for generating a second message from the first message, said second message comprising a drop-down line of content from the first message, at least one pair of markers designating, within said drop-down line, at least one segment of the content of the first message, an enrichment element associated with said segment of the content of the first message, and a display format instruction for said second message, and - a step of sending the second message to at least one recipient terminal.

[0016] No limitation is attached to the nature of the enrichment element(s) included in the second message. It may be a comment, a reply, a question prompted by the content of the first message pointed to by the segment, etc.

[0017] The second message typically corresponds to a reply from the user of the second terminal to the first message and to the user of the first terminal who sent the first message. However, it may alternatively be a message based on the first message (i.e., reproducing all or part of its content) and forwarded to a third party. Following the same logic, the recipient terminal may refer to the first terminal (if the second message is a reply to the first message), or to a third-party terminal in the context of a message forwarding.

[0018] A content flow line corresponds to a reference system ordering the content elements that make up the first message. Depending on the nature of the first message (text, video, audio), these content elements correspond respectively to words or characters, images, shots or sequences, phonemes, sounds (notes or spoken words), etc.

[0019] Thus, in the case of content natively associated with a timeline, such as typically video or audio content, the content flow line corresponds to this timeline. In the case of textual content, the content flow line corresponds to a linear ordering of the different elements composing the text, for example, an incremental numbering of each character or each word of the textual content of the message.

[0020] The markers that make up the marker pair designate points within said line of the content of the first message. In the case of video content, such markers are similar to timecodes. designating two moments in the video. In the case of text content, such markers can correspond to two numbers referring to two specific places in the text, for example to specific letters or words.

[0021] This pair of markers makes it possible to identify a content segment of the first message, for example, a passage from a video or text of the first message delimited by said markers. It is to this content segment of the first message that enrichment content of the second message is associated, in accordance with the invention.

[0022] For example, audiovisual enrichment content refers to a segment of a video contained within an initial message that is between 1 minute 30 seconds and 1 minute 45 seconds long. As another example, audio enrichment content refers to a segment of text contained between the 25th and 63rd characters of an initial message.

[0023] A display format instruction corresponds to an indication that the second message should preferably be displayed (or played) in a particular format (text, audio, audiovisual, etc.) and / or a particular presentation. For example, a display format instruction might indicate that the enrichment content of the second message should be spoken aloud rather than appearing in text form. Another example of a display format instruction might indicate that the second message should be displayed on the first terminal in such a way that the content of the first message is displayed together with the content dropdown, the marker pair(s), and, associated with these, a thumbnail corresponding to the enrichment content attached to the second message. This type of presentation is subsequently referred to as an "enriched" format.

[0024] The invention also relates to a method for receiving a second message by a receiving terminal comprising, following the sending of a first message by a first terminal to a second terminal: - a step for receiving a second message, from the second terminal, comprising a line displaying the content of the first message, at least one pair of markers designating within said line displaying at least one segment of the content of the first message, at least one enrichment element associated with said at least one segment of the content of the first message, and an instruction for formatting the display of the second message,

[0025] - displaying the content of the second message according to said display format.

[0026] The invention advantageously allows for responding to, or more generally enriching, an initial message by associating response elements or comments (also referred to as enrichment elements) with specific parts of the initial message defined by at least two markers placed on the content flow line. This communication system allows for the following modalities This results in more understandable and interactive exchanges for the recipient: the user can use the content dropdown menu as a quick way to access a segment of the content, such as a sentence or a moment in a video or audio clip. Furthermore, each enriching element in the second message is contextualized in relation to the first message (and specifically in relation to a segment of the content dropdown menu), making the response clearer for the recipient.

[0027] Furthermore, thanks to the invention, it is possible in certain embodiments to make references to the initial content of the first message by directly exploiting the markers placed on the content scroll line so as not to have to send said initial content in addition to the content of the second message (only the markers are sufficient): this offers the advantage of proposing a more synthetic and more economical communication mechanism in terms of network resources.

[0028] According to one embodiment, the first message is an audio and / or video message.

[0029] The use of the invention for processing such messages is all the more advantageous since audio and / or video content, as previously mentioned, is where quoting a passage is most complex. This embodiment thus advantageously allows users, whether recipients or senders, to achieve a simpler communication system without having to consult the entire content for each exchange, and where each enriching element is contextualized (i.e., linked to a specific part of the content of the first message) so as to facilitate understanding between users as their discussion progresses.

[0030] According to a particular embodiment, a said pair of markers included in the second message is identified from an analysis of the content of the first message.

[0031] This embodiment simplifies the selection of the received content segment(s) to which the user of the second terminal wishes to respond. For example, the analysis step can detect, in the first message, questions to which the user of the second terminal might reasonably be expected to respond, by identifying distinctive features (such as punctuation marks) in the first message that call for a reaction (comment or reply, etc.) from the second user. The analysis step can also segment the content of the first message, for example, for a first video message, by separating the passages where the author of the first message is facing the camera from the other passages (where, for example, they show a particular element), etc.

[0032] According to another particular embodiment, the processing method includes a step of obtaining information representative of a user's attention to the second terminal during the reading of the first message, and a said pair of markers included in the second message is identified from the information obtained.

[0033] This embodiment makes it possible to anticipate, when a user of the second terminal views the first message, the elements of the content of said first message that are likely to lead to some form of enrichment (comment, question, answer, etc.) on their part. For example, it could involve detecting the area looked at by the user based on the analysis of their gaze direction, the user's emotions based on the analysis of their face, etc. In the case of viewing an initial audio message, it could involve detecting external noises that could disrupt the concentration of the user of the second terminal. Of course, these are only illustrative examples and are not exhaustive.

[0034] As with content analysis, the analysis of the reactions of the user of the second terminal results in the enrichment of the content flow by adding markers at the end of this analysis. These markers can be advantageously used to help the user of the second terminal place markers indicating the content segment(s) to which they wish to respond, and thus help them formulate their message more easily and quickly.

[0035] According to a particular embodiment, at least one said enrichment element included in the second message is generated from the analysis of the content of the first message and / or information representative of the attention of the user of the second terminal.

[0036] This embodiment makes it possible, in particular, to simplify the formulation of a reply to a message by preparing the content of the second message and facilitating the task of the respondent. For example, the second message can be pre-generated following an element of the content of the first message relating to a particular question. Conversely, if a term in the first message appears not to have been understood by the user of the second terminal (based on the analysis of their reaction when they consulted the first message), according to this embodiment, a standard question relating to that term can be pre-generated. This embodiment thus advantageously streamlines exchanges by reducing user response time.

[0037] Moreover, this embodiment also makes it possible to help, in the event of a reply, the user of the first terminal to become aware of the unclear points of his message.

[0038] According to a particular embodiment, a said pair of markers included in the second message is identified from a command of a user of the second terminal.

[0039] This embodiment allows the user of the second terminal to select the sub-part of content (the content segment) that he / she wishes to enrich (with a comment, a question or an answer) autonomously and, potentially, in a more confidential way (in particular without analysis of the content of the first message or of his / her reactions).

[0040] According to a particular embodiment, the step of sending the second message is preceded by a step of formatting the second message according to settings and / or capabilities of said at least one recipient terminal.

[0041] This embodiment, which also assumes that the second terminal has information relating to the technical capabilities or user preferences of the recipient terminal, ensures that the recipient terminal is able to render the second message (for example, play audio content, display a message or a video). The reformatting may, depending on the circumstances, involve transcribing audio / video into text, compressing video, etc. This embodiment also ensures that the second message is received in a format that conforms to the one desired by the user of the recipient terminal (for example, in the case of a visually impaired user, by vocalizing text messages and converting them into audio content). Ultimately, this embodiment further improves communication between users.

[0042] According to another embodiment, the content of the first message and said at least one enrichment element are stored on a content server accessible by the first terminal, the second terminal, and the recipient terminal. This embodiment advantageously limits network costs, particularly when the same reference content is cited several times within several messages or when the second message is sent to a large number of people. In such a case, the centralized server allows for optimal management of message distribution to the various recipients.

[0043] This embodiment also allows for saving messages sent by users, making it possible to consult these messages later.

[0044] Finally, it offers the possibility of delegating to said server certain analyses that may be implemented, such as content analysis or user attention analysis.

[0045] According to another embodiment, the processing method further includes a step of requesting validation of the second message by the user of the second terminal before sending said second message.

[0046] This embodiment advantageously allows the author of the second message to ensure the consistency and general content of the second message, which helps to improve the overall quality of the discussion.

[0047] The invention also relates to a device for processing a first message received from a first terminal, said processing device being embedded in a second terminal and comprising: - a generation module, configured to generate a second message from the first message, said second message comprising a drop-down line of content from the first message, at least one pair of markers designating, within said drop-down line, at least one segment of the content of the first message, at least one enrichment element associated with said at least one segment of the content of the first message and an instruction for the display format of said second message, - a sending module, configured to send the second message to at least one recipient terminal.

[0048] The invention also relates to a device for receiving a second message embedded in a receiving terminal and comprising: - a receiving module configured to receive a second message received from a second terminal following the sending of a first message by a first terminal to the second terminal, said second message comprising a drop-down line of the content of the first message, at least one pair of markers designating, within said drop-down line, at least one segment of the content of the first message, at least one enrichment element associated with said at least one segment of the content of the first message, and a display format instruction for the second message; and - a display module configured to display the content of the second message according to said display format.

[0049] The invention further relates to a computer program, comprising program code instructions for the implementation of a process according to any one of the particular embodiments described above, when these programs are executed by a processor.

[0050] Such instructions can be stored permanently in a non-transient memory medium of a terminal implementing the message processing or reception method according to the invention.

[0051] This program may use any programming language, and be in the form of source code, object code, or code intermediate between source code and object code, such as in a partially compiled form, or in any other desirable form.

[0052] The invention also relates to a computer-readable information or recording medium on which a computer program such as those mentioned above is recorded.

[0053] The recording medium can be any entity or device capable of storing the program. For example, the medium may include a means of storage, such as a ROM (Read Only Memory), for example a CD ROM (Compact Disc Read-Only Memory) or a microelectronic circuit ROM, or a magnetic recording medium, for example a mobile medium, a hard disk drive or an SSD (Solid State-Drive).

[0054] On the other hand, the recording medium can be a transmissible medium, such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio, or by other means, so that the computer program it contains is executable remotely. The programs according to the invention can, in particular, be downloaded onto a network, for example, an Internet-type network.

[0055] Alternatively, the recording medium may be an integrated circuit in which one of the programs is incorporated, the circuit being adapted to execute or to be used in the execution of a message processing method or a message receiving method according to the invention.

[0056] The invention also relates to a communication system, said system comprising: - a device for processing a first message, and - a device for receiving a second message.

[0057] Finally, the invention relates to a data structure of a second message comprising: - the content of the first message to which the second message refers, (it may be a file (text, video, etc.) or information such as a link allowing access to this content), - a content dropdown line associated with the content of the message Msgl, - at least one pair of markers designating, within said dropdown line, at least one segment of the content of the first message, - an enrichment element associated with said segment of the content of the first message, - a display format instruction for said second message.

[0058] Furthermore, it can be envisaged, in other embodiments, that the processes and devices according to the invention present in combination all or part of the aforementioned characteristics. Brief description of the figures

[0059] Other features and advantages of the present invention will become apparent from the description given below, with reference to the attached figures which illustrate an example of an embodiment without any limiting character.

[0060] Figure [Fig. 1] represents, in its environment, a system according to the invention, in a particular embodiment,

[0061] Figure [Fig.1A] schematically represents the hardware architecture of a device for processing a first message according to one embodiment of the invention,

[0062] Figure [Fig.1B] schematically represents the hardware architecture of a device for receiving a second message according to one embodiment of the invention.

[0063] Figure [Fig.2] schematically represents the components of a second message according to the invention,

[0064] Figure [Fig.3] describes the steps of the processes for processing a first message and a second message according to one embodiment,

[0065] Figure [Fig.4] describes an example of a second message displayed according to a rich format instruction,

[0066] Description of the invention Description of a system conforming to the invention

[0067] Figure [Fig.1] represents, in its environment, a SYS system according to the invention, in a particular embodiment, said system being configured to implement message processing and reception methods according to the invention.

[0068] In the embodiment described here, the SYS system includes at least a first communicating terminal T1 and a second communicating terminal T2, connected to a network R.

[0069] Terminals T1 and T2 are, for example, smartphones, tablets or personal computers capable of sending and receiving asynchronous messages (such as email, "chat", instant messaging, etc.) in text, audio, video or any other format known to a person skilled in the art.

[0070] The network R is, for example, a mobile telephone network or a fixed network (copper or fiber). It may be composed of one or more (sub-)networks such as, for example, Wi-Fi networks or mobile networks operated, where applicable, by the same operator or by different operators.

[0071] According to a particular embodiment, the SYS system may also include a server S, not shown, connected to the network R and capable of communicating with the terminals T1 and T2. The server S is in particular capable of hosting the messages sent by at least one of the two terminals.

[0072] According to the invention, the T2 terminal is a message processing device according to the invention equipped with: - of an ER2 sending and receiving module, and - of a GR message generation module.

[0073] In the embodiment described herein, it is further equipped: - of an AA attention analysis module for a user of terminal T2, and - of an AC content analysis module.

[0074] However, the AA and AC modules are optional.

[0075] According to other embodiments, the attention analysis and content analysis module(s) are hosted within an S server.

[0076] The Tl terminal is equipped with an ER1 send and receive module and an AR message display module.

[0077] According to the invention, when terminal T1 sends a first message Msgl to terminal T2, the latter is able to respond in the form of a second message Msg2, which is an enriched version of the message Msgl, as described in more detail below.

[0078] Fig. 1 A shows the simplified structure of the T2 terminal configured to implement the message processing method in a particular embodiment.

[0079] In this embodiment, the terminal T2 includes a send and receive module ER2 adapted for receiving and transmitting calls and information. Furthermore, in the particular embodiment of the invention described herein, the steps performed by the terminal T2, within the framework of implementing the message processing method of the present invention, are implemented by means of instructions in a computer program PG2. For this purpose, the terminal T2 has the classic architecture of a computer and includes, in particular, a memory MEM2, a processing unit UTR2, equipped, for example, with a processor PROC2, and controlled by the computer program PG2 stored in memory MEM2. The memory MEM2 is a storage medium within the meaning of the invention. The computer program PG2 includes instructions for implementing the steps of the message processing method according to the invention as described later with reference to Figure [Fig. 3].

[0080] The PG2 program defines functional modules of the message processing device which include in particular: - the GR generation module, which is configured to generate a second message from the first message, said second message comprising a drop-down line of content from the first message, at least one pair of markers designating, within said drop-down line, at least one segment of the content of the first message, an enrichment element associated with said segment of the content of the first message and a display format instruction for said second message, - the ER2 sending module, which is configured to send the second message to at least one recipient terminal.

[0081] In the particular embodiment described here, the PG2 program further defines the functional modules for attention analysis AA and content analysis AC of the terminal T2.

[0082] Fig. 1B presents the simplified structure of a terminal T1 configured to implement the process of receiving and processing a second terminal in a particular embodiment.

[0083] In this embodiment, the terminal Tl includes a sending and receiving module ER1 adapted to receive and send calls and information.

[0084] Furthermore, in the particular embodiment of the invention described herein, the steps executed by the terminal Tl, within the framework of implementing the method for processing a second message of the present invention, are implemented by means of instructions in a computer program PG1. For this purpose, the terminal Tl has the conventional architecture of a computer and includes, in particular, a memory MEM1, a processing unit UTR1, equipped, for example, with a processor PROC1, and controlled by the computer program PG1 stored in memory MEM1. The memory MEM1 is a storage medium within the meaning of the invention. The computer program PG1 includes instructions for implementing the steps of the method for receiving a second message according to the invention, which are described later with reference to Figure [Fig. 3].

[0085] The PG1 program thus defines functional modules of the terminal Tl for receiving a second message which include: - the ER1 send and receive module, which is configured to receive a second message received from a second terminal following the sending of a first message by a first terminal to the second terminal, said second message comprising a drop-down line of the content of the first message, at least one pair of markers designating, within said drop-down line, at least one segment of the content of the first message, at least one enrichment element associated with said at least one segment of the content of the first message, and a display format instruction for the second message; and - the AR display module, which is configured to display the content of the second message according to said display format.

[0086] Description of the components of a second message according to the invention;

[0087] Figure [Fig.2] represents two messages Msgl and Msg2.

[0088] The first Msgl message is a message sent from terminal Tl to terminal T2 and including CL content. For example, the Msgl message is a message sent by terminal Tl from an instant messaging service.

[0089] The second message Msg2 is, in the embodiment described here, a reply message to the first message Msgl, sent by terminal T2 to terminal TL. The second message is a data structure within the meaning of the invention, comprising the following elements: - the content Cl of the first message Msgl to which the second message refers, (this can be a file (text, video, etc.) or information such as a link to access this content), - a content flow line L1 associated with the content Cl of the message Msgl, - at least one pair of markers Ma2 placed on the content flow line Ll, - at least one enrichment element C2 associated with the pair of markers Ma2, the message Msg2 containing as many enrichment elements as there are pairs of markers placed on the content flow line Ll, and - an IR2 instruction for displaying the message Msg2.

[0090] Within this structure, the initial content from a first message to which the second message refers can be a content file (text, video, etc.) or a link referring to this content.

[0091] The nature of the content flow line varies according to the nature of the content of the Msgl message.

[0092] By way of example, as mentioned previously, if the content Cl is audio or audiovisual content, the content flow line corresponds to the timeline of that content. If the content Cl is text, the content flow line may correspond to a linear arrangement of the different elements composing the text, for example, an incremental numbering of each character or each word of the textual content of the message. Alternatively, the content flow line may correspond to a timeline of the text spoken according to a predefined phrasing.

[0093] If such a content flow line Ll is not present in the message Msgl, it can be generated by the terminal T2, in a way known per se, during the generation of the second message Msg2.

[0094] It should be noted that in the case of a multimodal message (i.e. a message including, for example, textual and audiovisual elements or textual and audio elements, etc.), the timeline reflects this multimodal aspect by integrating, if necessary, several different content timelines, one after the other, corresponding respectively to each of the modes present in the message.

[0095] For example, when the content of the first message consists of a text of 2000 characters and a 3-minute video file, the content flow line may include a first content flow line numbered from 1 to 2000, where each increment corresponds to each character of the text in the reading direction, and a second temporal content flow line numbered from 0 to 180 where each increment corresponds to one second of the video content and is associated with the image of the video displayed at that second.

[0096] According to another example, when the first message is composed of the same elements as in the previous example (a text and a video), the content flow line can correspond to a time-based content flow line numbered from 0 to 500 where each increment from 0 to 319 corresponds to one second and is associated with a passage of the text of the spoken message, then, for seconds from 320 to 500, with one second of the content associated with the image of the video displayed that second.

[0097] According to the invention, at least one pair of Ma2 markers is placed on the content flow line. This pair of markers makes it possible to identify a segment of the content of the first message Msgl (for example, a passage in a text or a segment of a video) to which enrichment content can subsequently be associated.

[0098] No limitation is attached to the form taken by these markers.

[0099] By way of illustration, a pair of markers can correspond to two numbers referring to two text characters in a text or to two images in a video.

[0100] For example, depending on the content flow line to which it relates, a pair of numbered markers "137-248" may designate:

[0101] - the set of characters contained between the 137th and 248th character of the text of the first message Msgl intended to be quoted in the second message Msg2 and associated with an enrichment element; or - to the video segment contained between the 137th and 248th frame of a video content corresponding to the first message Msgl intended to be quoted and associated with an enrichment element in the second message Msg2.

[0102] The markers can also be associated with metadata (such as an indication of the modality that enabled them to be obtained (attention analysis, content analysis, manual annotation by the user).

[0103] It should be noted that the content flow line may also contain markers relating to reactions from a previous recipient of the first message where appropriate, such as those produced during step E301B described below.

[0104] The IR2 display format instruction is a command intended to set the display (or rendering) format of the second Msg2 message on the receiving terminal to which the second Msg2 message is sent, in this case the T2 terminal.

[0105] According to this embodiment, the display format instruction IR2 corresponds to a command intended to display the second message Msg2 in a so-called "enriched" format, showing both the content of the first message Msgl, the content dropdown line L1 and, delimited by the markers Ma2 placed on the dropdown line L1, the enrichment elements referring to the content segments of the first Msgl message delimited by these markers. This format is described in more detail with reference to figure [Fig.4].

[0106] According to another example, the display format instruction IR2 may indicate that the enrichment elements must be vocalized in order to be heard by the recipient of the second message. Such a display format instruction requires the implementation of vocalization and speech synthesis solutions known to those skilled in the art and not described here, such as, for example, eSpeak®, Festival®, or 15.ai®.

[0107] According to another example, the IR2 display format instruction can instruct the receiving terminal not to display the entire contents of the first Msgl message.

[0108] According to yet another example, the IR2 display format instruction can instruct the receiving terminal to display the enrichment elements in text format. Such a display format instruction can rely on the implementation of well-known speech-to-text automatic speech recognition solutions such as, for example, Dragon Naturally Speaking®, CMU Sphinx®, or Whisper®.

[0109] It should be noted that a display format instruction for the second message Msg2 may not be implemented by the terminal receiving the second message, for example because that terminal is not capable of displaying the display format indicated by the instruction or because the terminal is configured locally to display the second message in another format.

[0110] We will now describe with reference to [Fig.3] the main steps of a process for processing the message Msgl implemented by the first terminal T1 and of a process for receiving the message Msg2 implemented by the second terminal T2, in a particular embodiment. [YES] Description of the main steps in the processes for processing a first message and receiving a second message according to a particular embodiment of the invention

[0112] Figure [Fig.3] describes the steps of the process of processing a first message and of the process of receiving a second message.

[0113] In E301, the second terminal T2 receives from the first terminal T1 the message Msgl containing content Cl, T1, T2 as described in Figure [Fig. 2]. According to the embodiment described here, the content Cl is video content. Alternatively, the content Cl can be text, audio, or a combination of several of these forms (including video).

[0114] In E301A, the message generation module GR of the second terminal T2 creates a content unfolding line L1, associated with the content Cl of the message Msgl.

[0115] The content flow line can be created for example from the extraction of metadata from the content of the first Msgl message (in the case of a video or audio content) or following the implementation of text analysis solutions (character and / or word counter, reading time calculation via solutions such as wordcounter®, Gorby®, or any other solution known to the person skilled in the art).

[0116] In the example of a video-type content Cl considered here, the GR message generation module reuses the video's timeline line as the L1 timeline line of content Cl.

[0117] In the embodiment described here, in E301B, an analysis of the attention of the recipient of the first message Msgl, i.e. of the user of the second terminal T2 in the example considered here, is also carried out by the attention analysis module AA of the terminal T2, when said user consults the content Cl.

[0118] This analysis can be performed using techniques known to those skilled in the art, based in particular, but not exclusively, on capturing the user's face. The user's face can be captured, for example, by the front-facing camera of the T2 terminal, if it has one. The captured images are then analyzed by emotion recognition solutions based, for example, on convolutional neural networks, long-term memory, extreme learning machines, or hybrid solutions combining these different architectures or others.

[0119] Among the areas of analysis, the following may be cited as illustrative and non-limiting examples: - Analysis of the user's gaze and the areas viewed, taking into account the user's gaze axis, - where applicable, if the user interface implements a pointer, analysis of the pointer's position (finger, cursor, etc.), - where applicable, if the message content is textual in nature, analysis of the displayed text areas, - analysis of the facial expression of the recipient of the first message (including in the case of an audio message if, for example, the user seems not to hear a passage of the audio message correctly). - etc.

[0120] The analysis of the attention of the user of the terminal T2 may be based on any other technique known to the person skilled in the art, as a substitute or in addition to those mentioned above.

[0121] Following this analysis, markers corresponding to the information from this analysis are placed on the content flow line Ll.

[0122] In the case of markers associated with user attention, the positioning of the markers on the content flow line can take into account the user's reaction times in order, for example, to position the markers slightly upstream at the moment of detection of the variation in attention.

[0123] Such information may, for example, take the form of metadata reported on the content flow line Ll, here the content timeline. If the content Cl is text, this information may take the form of references or annotations associated with the various passages of the text concerned.

[0124] This metadata may include the context of marker generation (gaze analysis, facial expression).

[0125] In another embodiment, the analysis of the attention of the user of terminal T2 can be carried out by an entity other than the second terminal T2, such as, for example, by a server communicating with the second terminal T2. In this embodiment, the second terminal T2 sends the data relating to the reactions of the user of terminal T2 that it has captured to said server and, at the end of the analysis, the server sends the information resulting from this analysis to terminal T2.

[0126] These markers can subsequently be used to help the user of the T2 terminal select parts of content to which he / she wishes to respond.

[0127] It should be noted that step E301B is optional.

[0128] In the embodiment described here, in E301C, a content analysis Cl of the first message Msgl is further performed by the content analysis module AC of terminal T2.

[0129] Such content analysis may, for example, rely on the implementation of technologies known to those skilled in the art, such as: - segmentation of video content into scenes and / or shots, - transcription of audio or audiovisual content into textual content, - phoneme analysis, - shape analysis, - location recognition, - facial recognition, - context recognition aimed in particular at reducing ambiguities of certain acronyms or proper names, whether written or spoken, - text or transcription analysis,

[0130] In another embodiment, the analysis of the content Cl of the first message Msgl can be carried out by an entity other than the second terminal T2, such as by example by a server communicating with the second terminal T2. In this embodiment, the second terminal T2 sends the content Cl of the first message Msgl to this server and, after the analysis, the server sends to the terminal T2 the information resulting from the analysis of the content.

[0131] At the end of this step, markers corresponding to the relevant information from the content analysis are placed on the content flow line L1.

[0132] For example, markers are placed for: - to delimit the different shots or scenes of a video content, or to, - place text elements from the transcription of audio or video content, - indicate the names of people appearing on screen (if facial recognition allows identification of people in the video content),

[0133] Such information relating to content analysis may take the form of metadata reported on the content flow line, here the timeline of content Cl. According to other examples, if content Cl is a text, this information relating to user attention may take the form of references or annotations associated with the different passages of the text concerned.

[0134] These markers can subsequently be used to help the user select parts of content to which they wish to respond.

[0135] It should be noted that step E301C is also optional.

[0136] In E302, terminal T2 receives an instruction from a user to place a pair Ma2 of markers within the content line Ll.

[0137] For example, the user of terminal T2 can enter the position of the pair of markers Ma2 via a touch screen type interface, cursor or any other human-machine interface known to a person skilled in the art.

[0138] This pair of markers is intended to delimit a segment (indicate a part of the content) Cl to which the user of terminal T2 wishes to respond.

[0139] The user's input of marker pairs can be facilitated by the attention analysis steps E301B and the content analysis step E301C when implemented. If necessary, markers are already pre-positioned on the content line at the end of these steps. It is possible to display only a portion of the markers generated by the analysis steps E301B and E301C to avoid information overload that would complicate the actions of the user on the second terminal. The selection between displayed and hidden markers can be performed automatically according to predefined rules or following manual configuration of the GR message generation module. The criteria for whether a marker pair is displayed can be diverse and varied. For example, it could depend on the number of markers produced at the end of the analysis steps. attention and content analysis, proximity of other markers on the content flow line, a confidence index or weight assigned to markers, etc.

[0140] In E303, the second terminal T2 receives an instruction from the user of the second terminal to associate said pair of markers Ma2 with a content C2, also referred to as "enrichment element".

[0141] The generation of the enrichment element C2 can take place upstream or downstream of step E302. This enrichment element can be an answer to a question present in the first message, a comment or even a question etc.

[0142] In the example considered here, it is assumed that the user of the second terminal T2 films an enrichment content (a reply or a comment related to the passage of the first message) delimited by the pair of markers Ma2 positioned during the step E302 and then associates this enrichment content with this pair of markers.

[0143] This is, however, only an illustrative and non-limiting example of the invention: the enrichment element C2 may alternatively be textual, audio, etc., or correspond to a combination of several of these modes. It may be produced using input (keyboard) and / or recording (camera, microphone) methods and devices known to those skilled in the art, but not described in detail here. The generation of the enrichment element C2 may also implement content editing tools and interfaces (text editor, editing tool, etc.) known per se.

[0144] According to the embodiment described herein, the content and / or user attention analysis steps of terminal T2 are used to pre-generate the C2 enrichment element of the Msgl message.

[0145] For example, data from the analysis of the attention of the user of the second terminal T2 can be used to enable the message generation module GR to pre-generate questions with respect to terms that the user of the second terminal does not seem to have understood.

[0146] According to another example, the data from the analysis of the content of the Msgl message can be used to generate images or search for quotations related to words or phonemes identified within the CL content

[0147] This pre-generation step of a C2 enrichment element of the Msgl message is based on the implementation of a Retrieval Augmented Generation module or fine tuning technique, or any other solution known to the person skilled in the art (keyword search, image generation, etc.).

[0148] It should be noted that this step of preparing the enrichment element is optional.

[0149] Steps E302 and E303 are iterated as many times as necessary depending on the number of enrichment elements and marker pairs that the user of terminal T2 wishes to attach to the second message. For example, if the user of terminal T2 wishes to react to two passages of the message Msgl, they select two content segments by placing two marker pairs on the content flow line and associate an enrichment element with each of these two marker pairs.

[0150] In E304, an IR2 instruction for displaying the second message is produced by the message generation module GR. This display instruction can be generated automatically or set manually by the user of terminal T2.

[0151] In the example considered here, this display format instruction instructs the display of message Msg2 in an enriched format, that is, by displaying both the first message Msgl together with the content dropdown line L1 and the various enrichment contents C2 added by the user of terminal T2. A representation of a message Msg2 displayed according to such an instruction is illustrated in Figure [Fig.4], which is described in detail later.

[0152] According to another example, it may consist of an instruction to present the second message Msg2 in a particular format (audio, video, text, a combination of these formats, etc.).

[0153] The implementation and compliance of the IR2 instruction for the display format of the second message Msg2 are conditioned by the display capabilities of the terminal receiving the second message (Tl in the example considered here) and / or by the user preferences and / or configuration parameters of this receiving terminal.

[0154] In another embodiment, the Msg2 message may further include additional information such as markers obtained at the end of steps E301B and E301C but which the user has not selected or associated with an enrichment element during steps E302 and E303.

[0155] According to this embodiment, the message Msg2 is also adapted, by the message generation module, to the preferences of the user of the terminal to which the second message is addressed and to the capabilities of that terminal. If necessary, the sending of said message is then preceded by a reformatting step of the second message according to the settings and / or capabilities of said terminal to which the second message is addressed.

[0156] This reformatting step can be implemented with transcription and / or vocalization tools known to those skilled in the art and described above.

[0157] According to such an embodiment, the preferences and / or capabilities of the recipient terminal(s) are communicated to the T2 terminal via any means known to a person skilled in the art (information associated with the first message, information provided by a third-party server, etc.).

[0158] Optionally, the sending of the second message Msg2 is preceded by a validation step of said second message by the user of terminal T2. This step allows the user to reread the message and, if necessary, modify its content.

[0159] In E305, terminal T2 sends the second message Msg2 to a recipient terminal. In the example considered here, message Msg2 is a reply message to message Msgl, sent to the first terminal Tl.

[0160] Alternatively, it can be envisaged that the message Msg2 is a message derived from the message Msgl transferred to one or more third terminals in addition to or in replacement of the terminal Tl.

[0161] For example, if it is known that terminal Tl does not have an optimal connection to network R, this reformatting can lead to (by implementing technical means similar to those mentioned above) a transcription of audio / video content into text content or to lowering the video quality of the content in order to reduce the bandwidth required to view it.

[0162] According to another example, if it is known that the recipient of the second message is visually impaired, the reformatting of the second message can consist of vocalizing the textual contents and converting said textual messages into audio messages by using implementation by technical means similar to those mentioned above.

[0163] In E306, the receiving terminal, i.e., in the example considered here, the terminal Tl, receives the second message Msg2 and displays it according to the IR2 instructions as interpreted by said terminal Tl.

[0164] In the example chosen here and in accordance with the IR2 instruction for displaying the second message established in E304, the terminal Tl displays the message Msg2 in an enriched format consisting of displaying: - the initial message Msgl and - the content flow line L1 including the pair of markers Ma2 associated with content C2.

[0165] This display format advantageously allows the presentation of all content elements including (enrichment elements) of the second message Msg2 by contextualizing them in relation to the passages to which they refer within the first message Ml.

[0166] This display format of the second message also invites, without obligating, the user of the first terminal T1 to continue the conversation initiated with the user of the second terminal T2 according to a message processing method received in accordance with the invention, that is to say by selecting in turn an element of the content C2 to which he wishes to respond in particular.

[0167] Figure [Fig.4] represents, by way of illustration, an example of displaying the message Msg2 on the display screen EA of the terminal Tl according to an instruction IR2 of display format called "enriched".

[0168] According to this embodiment, the second message comprises two enrichment elements C2 and C2' associated respectively with pairs of markers Ma2 and Ma2' placed on the content flow line.

[0169] When the user of the receiving terminal wishes to view the second message, they can click on the thumbnail corresponding to either of the enrichment elements C2, C2'. If so, this content is "played" and displayed on the content viewing area ZA (typically a video player or any other corresponding player compatible with the format of the enrichment elements).

[0170] Thanks to this enriched display format, the user of the receiving terminal can also move the CUR cursor along the content drop-down line (or directly onto the representation of the initial content of the first message Cl). In this case, the display area shows the content of the first message corresponding to the moment selected by the cursor. The user can thus view the part of the first message to which a particular enrichment element refers.

Claims

Demands

1. A method for processing a first message received from a first terminal, said method being implemented by a second terminal and comprising: - a step of generating a second message from the first message, said second message comprising a line of content from the first message (L1), at least one pair of markers (Ma2) designating, within said line of content, at least one segment of the content of the first message, at least one enrichment element (C2) associated with said at least one segment of the content of the first message and a display format instruction for said second message (IR2), and - a step (E305) of sending the second message to at least one recipient terminal.

2. A method for processing a first message according to claim 1 wherein the first message is an audio and / or video message.

3. A method for processing a first message according to claim 1 or 2 wherein a said pair of markers included in the second message is identified from an analysis by the second terminal of the content of the first message.

4. A method for processing a first message according to any one of claims 1 to 3 comprising a step of obtaining information representative of the attention of a user of the second terminal when reading the first message and wherein said pair of markers included in the second message is identified from the information obtained.

5. A method for processing a first message according to claim 3 or 4 wherein at least one said enrichment element included in the second message is generated from the analysis of the content of the first message (Msgl) and / or information representative of the attention of the user of the second terminal.

6. A method for processing a first message according to any one of claims 1 to 5 wherein a said pair of markers included in the second message is identified from a command of a user of the second terminal.

7. A method for processing a first message according to any one of claims 1 to 6, wherein the step of sending the second The message is preceded by a formatting step of the second message according to the settings and / or capabilities of at least one recipient terminal.

8. A method for processing a first message according to any one of claims 1 to 7 wherein the content of the first message and said at least one enrichment element are stored on a content server accessible by the first terminal, the second terminal and the recipient terminal.

9. A method for processing a first message according to any one of claims 1 to 8 comprising a step of requesting validation of the second message by a user of the second terminal before sending said second message.

10. A method for receiving a second message by a receiving terminal comprising, following the sending of a first message by a first terminal to a second terminal: - a step (E306) of receiving a second message, from the second terminal, comprising a line of content of the first message, at least one pair of markers (Ma2) designating within said line of content at least one segment of the content of the first message, at least one enrichment element associated with said at least one segment of the content of the first message and an instruction for displaying the second message, - the display of the content of the second message (Msg2) according to said display format.

11. A processing device for a first message received from a first terminal, said processing device being embedded in a second terminal and comprising: - a generation module, configured to generate a second message from the first message, said second message comprising a drop line of content from the first message (L1), at least one pair of markers (Ma2) designating, within said drop line, at least one segment of the content of the first message, at least one enrichment element (C2) associated with said at least one segment of the content of the first message and a display format instruction for said second message (IR2), - a sending module, configured to send the second message to at least one recipient terminal.

12. A device for receiving a second message embedded in a receiving terminal and comprising: - a receiving module configured to receive a second message received from a second terminal following the sending of a first message by a first terminal to the second terminal, said second message comprising a line of unfolding of the content of the first message, at least one pair of markers (Ma2) designating within said unfolding line, at least one segment of the content of the first message, at least one enrichment element associated with said at least one segment of the content of the first message and an instruction for the display format of the second message; and - a display module configured to display the content of the second message (Msg2) according to said display format.

13. Computer program, comprising program code instructions for implementing a method according to any one of claims 1 to 10, when said program is executed on a computer.

14. Information or recording medium (MEM), readable by a computer on which a computer program according to claim 13 is recorded.

15. Communication system (SYS), said system comprising: - a processing device according to claim 11 configured to process a first message received from a first terminal, and - a receiving device according to claim 12 configured to process a second message sent by the first processing device following the receipt of the first message.

16. Data structure of a second message comprising: - content (Cl) from a first message (Msgl) to which the second message refers, or information enabling access to this content, - a content dropdown line L1 associated with the content Cl of the message Msgl, - a dropdown line of the content of the first message, - at least one pair of markers designating, within said dropdown line, at least one segment of the content of the first message, - at least one enrichment element (C2) associated with said at least one segment of the content of the first message, - a display format instruction for said second message.

Images