Contextualization of an interactive computer application

Abstract

The invention relates to a method implemented by an electronic device, the method comprising: obtaining a context of the electronic device; determining at least one first intention of at least one interactive computer application based on at least one element of the context of the electronic device that is not linked to a human-machine interface of the at least one interactive computer application, said at least one first intention being determined from among at least one intention set out by the at least one interactive computer application, said at least one set-out intention being associated with at least one action to be performed via the at least one interactive computer application; providing said at least one determined first intention to the at least one interactive computer application.

Description

Description Title: Contextualizing an interactive computer application technical field

[0001] This disclosure falls within the domain of data processing by interactive computer applications. Previous technique

[0002] An interactive computer application, or simply "interactive application," is an application that allows a user to command actions through simple interaction (using natural language or a mouse click, for example) with the application itself. The application may also assist the user in formulating these commands (for example, by simplifying the process). The action may be performed by the interactive application itself or by another application (or service, or tool) to which the interactive application is linked, for example, to simplify the user's experience with that other application. For instance, some chatbots are interactive computer applications that present themselves as a dialogue window, allowing the user to perform tasks (or to have tasks performed).In this case, the interaction takes place via a dialog box displayed on the user's terminal screen. The dialog agent interacts with the user in natural language, for example, through a question-and-answer exchange. These natural language interactions allow the user to determine an action to be performed that corresponds to their objectives, to define all the parameters of the action, and finally to execute the action once all the parameters are known. Natural language interactions with the dialog agent thus prevent the user from having to manually enter the action command (for example, directly through an interface in another application that performs the action), as the action command interface may be less user-friendly than a dialog agent and may even be unfamiliar to the user.For example, if the user wishes to book a meeting room via a service such as an online electronic booking tool, the user's dialogue with the interactive computer application can determine the day and time of booking, the number of participants, etc., before proceeding with the booking via the online tool.

[0003] Another type of interactive application involves voice assistants. Such a voice assistant allows a user to perform one or more tasks via a voice dialogue with the assistant. In this case, the interaction can, for example, take place via a microphone and speakers, as if the user were having a dialogue with a human assistant.

[0004] Interactive computer applications are increasingly used, for example in the professional sphere, to perform tasks such as booking meeting rooms, train or plane tickets, etc. Interactive computer applications can provide a simpler and more user-friendly solution than standard interfaces.

[0005] State-of-the-art interactive applications require the user to precisely define the purpose of their interaction (to understand the command to be executed and obtain its input parameters). This leads to numerous interactions with the interactive application, thus increasing interaction time and limiting the potential benefits of these applications. In particular, interactions with state-of-the-art interactive applications can sometimes appear tedious and a waste of time to the user.

[0006] Therefore, there is a need to facilitate exchanges between an interactive computer application and its users, in order to perform an action by the interactive computer application or another application. Summary

[0007] This disclosure improves the situation.

[0008] A method is proposed, implemented by an electronic device, for the processing of data by at least one interactive computer application, comprising: obtaining a context of the electronic device; providing, to at least one interactive computer application, at least one first intention, exposed by said at least one interactive computer application, from at least one element of the context of the electronic device not linked to a human-machine interface of the at least one interactive computer application, said at least one exposed intention being associated with at least one action to be carried out via the at least one interactive computer application.

[0009] In particular, a method implemented by an electronic device is proposed, comprising: obtaining a context of the electronic device; determining at least one first intention of at least one interactive computer application from at least one element of the context of the electronic device not linked to a human-machine interface of the at least one interactive computer application, said at least one first intention being determined from at least one intention exposed by the at least one interactive computer application, said at least one exposed intention being associated with at least one action to be performed via the at least one interactive computer application; providing said at least one first determined intention to the at least one interactive computer application.

[0010] By "context of the electronic device", we mean any data relating to the use of the electronic device or its environment at a given time (or instant).

[0011] The context of an electronic device can, for example, include data displayed on the device's screen, sound data captured by a microphone or emitted by a speaker connected to the device, environmental data such as the device's location or ambient light, etc. The context can also take into account the location and date, the person's physical posture, eye tracking, etc.

[0012] The term "at least one element of the electronic device's context not linked to a human-machine interface of at least one interactive computer application" refers to at least one element of the context that is not affected by at least one interactive computer application. For example, this could be at least one context element not belonging to a human-machine interface of at least one interactive computer application, or even a context element from an application other than at least one interactive computer application.

[0013] The term "at least one intention exposed by at least one interactive computer application" refers to a goal that the application's interactions allow to be pursued. For example, at least one computer application can expose, via an API (Application Programming Interface), at least one intention that it is capable of using.

[0014] The term "expressed intention" refers to an intention detected through analysis of the device's context, particularly captured contextual elements. This could be, for example, a goal or objective expressed in a sentence or text (corresponding to a user utterance, a document excerpt (email, etc.), one or more sentences spoken and recorded by a microphone, etc.), an automatically generated description of an image in the context, and so on. It could also be an intention implicitly expressed by a user (for example, the sentence "I need to buy my train ticket" can reveal the expressed intention "to book a train"). An expressed intention allows us to categorize the intention of an end user within the context of an interaction with at least one interactive computer application and can be based on text extracted from any type of captured contextual element (text, audio, image, etc.).

[0015] An expressed intention can be associated with the contextual elements that enabled its detection.

[0016] The term "specific intention" refers to a specific (selected) intention, chosen from among those expressed by an interactive application, as the intention to be provided to the interactive application. This intention can be determined by correlating an intention expressed within the context of the electronic device with the intentions expressed by the interactive application.

[0017] For example, natural language processing (NLP) tools allow us to associate words in the context with intentions expressed within that context. For instance, sentence patterns can be recognized to identify the intention expressed by the words in the context. Natural language processing tools can then correlate these expressed intentions with the various intentions displayed by the interactive application.

[0018] Different user utterances can thus convey the same expressed intention, correlated (semantically, for example) with the same stated intention. For example, the utterances "What is your phone number?", "Where can I reach you by phone?", or even "Please give me the phone number." could correspond to the same expressed intention "retrieve a phone number", which can be correlated with the intention exposed by the interactive application "retrieve_phone_number".

[0019] In another example, the user statements "Remember to find a place to meet tomorrow from 2pm to 4pm", or "We need to book a meeting room at 2pm tomorrow for 2 hours" can correspond to the same expressed intention "book a room", which can be correlated with the exposed intention "book_room" of the interactive application.

[0020] Automatic recognition of the user's expressed intention can help the interactive computer application respond "directly" (more directly than prior art solutions), via the selection of the corresponding exposed intention.

[0021] Analyzing the words in the context (for example, classifying them) to obtain expressed intentions, and then correlating these expressed and exposed intentions, allows us to establish a link between the words identified in the context and the exposed intentions. Both an expressed intention and an exposed intention are alphanumeric strings, comprising at least one word in natural language, or a word fragment such as a root or lemma of that word. An expressed or exposed intention can be a synthetic phrase, for example, or a statement in natural language. Examples of expressed intentions (such as "book a room") and examples of exposed intentions (such as "room_booking") are presented later.

[0022] An expressed intention is associated with the contextual elements that enabled its recognition (or detection). For simplicity, in this request, the contextual elements that enabled the detection of an expressed intention are associated with the (determined) intention correlated with that expressed intention. Thus, in this request, the contextual elements "related" to a determined intention refer to the contextual elements associated with an expressed intention correlated with that determined intention.

[0023] The at least one exposed intention can be exposed by the interactive application in several ways. For example, the at least one exposed intention can be obtained via a file or via one or more messages (via sockets, APIs (English acronym meaning "Application Programming Interface", in French "Application Programming Interface") REST (English acronym meaning "Representational State Transfer", in French "Representational State Transfer"), etc.), such as JSON (English acronym meaning "JavaScript Object Notation", in French "JavaScript Object Notation") type messages, describing a list of intentions exposed by the interactive application (i.e. a part of its interface).

[0024] The term "action to be performed via the interactive computer application" refers to an action that can be initiated by the interactive computer application itself or through the issuance, by the interactive application, of a computer command to another application, such as an online tool or service. For example, if an expressed intention of the interactive computer application is to reserve a room via an online room booking service, this The stated intention can be associated with a room booking action, for example via a booking service API.

[0025] The method described in this application, in at least some of its embodiments, allows for the automatic determination, among the intentions expressed by the interactive computer application, of an intention semantically similar to an intention expressed within the context of the electronic device, and provides this intention to the interactive computer application, which can then convert this determined intention into an action. Such an embodiment can therefore help facilitate the interaction between a user of the electronic device and their environment. For example, in some embodiments, the number of messages to be exchanged between an interactive application and the user in order to trigger an action can be limited (for example, reduced) compared to the number of interactions that would be necessary if the user were to use the interactive application alone.For example, in some cases, the interaction with the user may be reduced to a single message to which the user can respond with a confirmation click. The process, in at least some of its embodiments, can therefore help facilitate communication between an interactive computer application and at least some of its users, with the aim of enabling the interactive computer application or another application to perform an action.

[0026] According to another aspect, a computer program is proposed comprising instructions for the implementation of all or part of a process as defined in this application, in any of its embodiments, when this program is executed by a processor.

[0027] In another aspect, a non-transient, computer-readable recording medium is proposed, on which such a program is recorded.

[0028] According to another aspect, an electronic device is proposed comprising at least one computing unit configured to execute all or part of a process (i.e. method) as defined in this application, in its various embodiments. In particular, an electronic device is proposed comprising at least one configured computing unit, for data processing by at least one interactive computer application, to: obtain a context of the electronic device; provide, to at least one interactive computer application, at least one first intention, exposed by said at least one interactive computer application, from at least one element of the context of the electronic device not linked to a human-machine interface of the at least one interactive computer application, said at least one exposed intention being associated with at least one action to be carried out via the at least one interactive computer application.

[0029] A "processing unit" is defined as an electronic component capable of performing electronic or computer calculations to carry out a specific function. A processing unit can refer to any type of processor or electronic component capable of performing numerical calculations. For example, a processing unit can be an integrated circuit, an ASIC (from the English acronym An application-specific integrated circuit (ASIC) can be defined as a microcontroller, a microprocessor, a digital signal processor (DSP), a processor, or a graphics processing unit (GPU). A computing unit according to the invention is not limited to a particular type of computing architecture. For example, a processor can implement a Harvard or Von Neumann architecture.

[0030] The term "memory" refers to a digital electronic element (device) used to store data. Various types of memory can be used in the invention, such as read-only memory (ROM), random-access memory (RAM), volatile memory, or flash memory. An electronic device according to the invention may be equipped with one or more non-volatile memories of various types, such as mass storage, flash memory, ROM, or solid-state drives (SSDs). A device according to the invention may also include one or more random-access memories, such as RAM, DRAM, SRAM, DPRREAM, VRAM, eDRAM, or 1T-SRAM.

[0031] The features described in the following paragraphs may optionally be implemented, independently of each other or in combination with each other:

[0032] In at least one embodiment, the process includes: providing at least one element of the context, relating to at least one first intention, to at least one interactive computer application.

[0033] In particular, in some embodiments, the process includes providing at least one element of the context, relating to at least one first determined intention, to at least one interactive computer application.

[0034] For example, contextual elements useful for configuring and / or specifying the action to be performed and / or establishing an interaction to configure the action can be transmitted. For example, if the determined intent is to book a train ticket, and the context indicates, via the information displayed on the screen, a reservation for a train from Paris to Dijon in two days, at least a portion of the displayed information can be provided to the interactive application.

[0035] Providing at least one element of context relating to at least one first intention determined to at least one interactive computer application makes it possible to provide the interactive application with certain known information enabling it to carry out the action, and / or to establish an interaction in order to configure the action to be carried out.

[0036] In at least one embodiment, said supply takes into account a correlation between at least one so-called expressed intention obtained from said context, and said first intention.

[0037] In particular, in at least one embodiment, said provision takes into account a correlation between at least one so-called expressed intention obtained from said context, and said first intention. Thus, in certain embodiments, said at least one first intention is determined by correlation between at least one so-called expressed intention obtained from said context with said at least one intention expressed by at least one interactive computer application.

[0038] The term "correlation between at least one expressed intention and at least one intention displayed by at least one interactive computer application" refers to determining whether at least one expressed intention corresponds to at least one intention displayed by at least one interactive application. This correspondence depends on the semantic proximity between the expressed intention and the displayed intention. It may be an exact, literal correspondence, or a partial correspondence between the expressed and displayed intentions.

[0039] Determining at least one first intention by correlation between at least one so-called expressed intention obtained from said context with said at least one intention exposed by at least one interactive computer application makes it possible to ensure that the at least one first determined intention is semantically consistent with the intentions expressed in the context, and therefore provide the at least one interactive computer application with at least one first determined intention consistent with the context.

[0040] In at least one embodiment, said context element includes at least one word rendered or acquired via a human-machine interface of the electronic device and said correlation takes into account a semantic proximity between said at least one expressed intention and said first intention.

[0041] In at least one embodiment, the method includes detecting said at least one word rendered or acquired via said human-machine interface of the electronic device.

[0042] Thus, in certain embodiments, said context includes at least one word rendered or acquired via a human-machine interface of the electronic device; said correlation takes into account a semantic proximity between said at least one expressed intention and said at least one intention exposed by the at least one interactive computer application; obtaining the context involves detecting said at least one word rendered or acquired via said human-machine interface of the electronic device.

[0043] The term "a human-machine interface of the electronic device" refers to a human-machine interface integrated into or coupled with the electronic device.

[0044] For example, it could be a word displayed on at least one screen of the electronic device, and / or captured via at least one microphone of the electronic device, and / or played back via at least one speaker of the electronic device, and / or present in a description extracted from an image displayed on at least one screen, and / or captured by at least one camera of the electronic device.

[0045] The ongoing detection of at least one dead element, whether rendered or acquired, to obtain / enrich the context (which itself allows for the elicitation of the expressed intention), and the consideration, in the correlation, of a semantic proximity between said at least one expressed intention and said at least one exposed intention, helps to determine "as early as possible" the at least one initial intention (i.e., as soon as contextual elements are detected that allow for the elicitation of an expressed intention in proximity). semantics with at least one of the intentions exposed by at least one interactive computer application).

[0046] In some embodiments, the process includes recognizing, among at least one word, at least one named entity associated with said at least one expressed intention.

[0047] By "named entity" we mean a word, or a group of words that can be categorized into classes such as names of people, names of organizations or companies, names of applications, names of places, quantities, distances, values, dates, email, or predefined categories (in a file, a database, ...).

[0048] Recognizing at least one named entity associated with at least one first determined intention within a given word can help identify contextual elements related to the expressed intention. Named entities can themselves be contextual elements. In some embodiments, these contextual elements can be provided to the interactive computer application, in association with the determined intention, to facilitate interaction with a user of the device (these contextual elements being likely to form parameters of the action to be performed).

[0049] In at least one embodiment, the process includes filtering at least one first intention and providing said at least one first intention after filtering.

[0050] Thus, in some embodiments, the process involves: filtering at least one first determined intention; providing said at least one first determined intention after filtering.

[0051] Filtering by at least one initial determined intention allows us to provide an intention exposed by the interactive application and correlated with an expressed intention only if it meets filtering criteria.

[0052] In embodiments where context is provided to the interactive application in addition to the determined intent, the context elements can also be filtered before being sent to the interactive application, for example to provide only the elements relevant to the expressed intent correlated with the determined intent.

[0053] In some embodiments, the filtering is configurable.

[0054] The filtering settings allow the user, or an administrator of the electronic device, to limit the use of the interactive application (via its stated intentions).

[0055] In some embodiments, filtering is performed according to the context of the electronic device.

[0056] Filtering based on context can help refine filtering rules and / or allow for more precise filtering settings. For example, filtering can be based on contextual elements such as location, date, or time.

[0057] In one embodiment, the filtering of a given intention takes into account an exclusion rule based on context elements associated with an expressed intention correlated with the given intention.

[0058] For example, an interactive application may expose a "recipe" intent associated with an "exclusion" rule "chocolate", in which case an exposed "recipe" intent will only be provided to the interactive application if the context does not mention "chocolate".

[0059] According to another example, certain stated intentions that imply payment may only be permitted for a user with an "adult" profile.

[0060] In at least one embodiment, said method includes starting said interactive computer application once said first intention has been determined.

[0061] In some embodiments, the interactive computer application is configured to emit at least one message on said human-machine interface of said interactive computer application, the message taking into account said first determined intention and said at least one context element relating to said at least one first determined intention.

[0062] The term "sending at least one message on the human-machine interface" means using an output of the human-machine interface to produce a message understandable by the user. The message could be, for example, a text message, but also an audio message, a video message, or a visual message such as the emission of a specific light or the execution of a particular movement by a robotic arm.

[0063] The sending of a message taking into account said first determined intention and said at least one element of context makes it possible to send a message dependent on the context of the device to the user, for example in order to obtain validation or clarification from the user.

[0064] In some embodiments, the interactive computer application is configured to: receive, via the input human-machine interface of said interactive computer application, at least one response message; configure at least one action to be performed based on at least one response message.

[0065] For example, the interactive computer application could be a virtual assistant integrating a conversational robot (also called a "chatbot" according to English terminology), and the conversational robot can exchange messages with the user via a messaging system, in order to confirm that the first determined intention provided to the conversational robot corresponds to an actual intention of the user (i.e. an action that the user actually wishes to order) and / or to collect additional information.

[0066] Similarly, the interactive computer application can be a voice assistant establishing a discussion with the user, in order to confirm that the first determined intention provided to the chatbot corresponds to an actual intention of the user and / or to collect additional information.

[0067] For example, if a specific intent, "train booking," aims to reserve a train ticket, and if contextual information unrelated to the interactive application (e.g., extracted from an email) provided to the interactive application indicates that the user wants to book a train from Paris to Dijon in two days, the chatbot or virtual assistant can present the user with train times from Paris to Dijon in two days, and ask them which time suits them, before triggering the action of booking the train ticket.

[0068] Receiving a reply message, and setting up the action to be taken based on the reply message, can help to clarify the user's actual intention in order to set up the action to be taken, while limiting exchanges by taking into account information extracted from the context by the interactive application, before its exchange with the user.

[0069] In some embodiments, the context of the electronic device includes context elements of the environment of the electronic device.

[0070] Taking into account contextual elements of the electronic device's environment can help to ensure that the determined intent is provided and / or to filter determined intents only when certain conditions are met (e.g., only during office hours, or when the device is in a given location) and / or to enrich the context provided to the interactive application with environmental data.

[0071] In some embodiments, at least one of the steps of obtaining a context of the electronic device, and determining at least a first intention, is carried out continuously.

[0072] By "is carried out continuously", we mean the execution of a step at successive intervals in time without significant stopping, for example periodically with a defined time interval (configurable and / or constant for example).

[0073] Continuously performing at least one of the steps ensures that the determination of at least one initial expressed intention is ongoing (or almost ongoing or regular), so as to detect an expressed intention within the context of the device "as early as possible."

[0074] In some embodiments, at least one of the steps of obtaining a context of the electronic device, and determining at least a first intention is carried out during the occurrence of an event representative of a change in the context of the electronic device.

[0075] Performing at least one of the steps when an event representative of a change in context occurs can help limit the frequency of determinations of expressed intentions, by performing a determination of expressed intentions only in situations (such as a change in the context of the device) making it possible to detect a new expressed intention (leading by correlation with the exposed intentions to a new determination of an exposed intention).

[0076] In some embodiments, the interactive computer application is an interactive computer application that is currently running.

[0077] Providing the specified intent to a running interactive computer application, that is, an interactive application running in the background, makes it possible to limit the time required before the interactive application processes the specified intent.

[0078] In some embodiments, the process includes, prior to providing at least one first determined intention to the interactive computer application, starting said interactive computer application.

[0079] Starting the interactive application before the specified intent is provided ensures that the application only starts when that intent is available. This helps limit the CPU load and memory usage of the interactive application. Brief description of the drawings

[0080] Other features, details, and advantages will become apparent upon reading the detailed description below and analyzing the attached drawings, on which: Fig. 1

[0081] [Fig. 1] shows an example of a dialogue between a user and an interactive computer application according to the state of the art. Fig. 2

[0082] [Fig. 2] shows an example of an electronic device suitable for implementing the process of the present application according to at least some of its embodiments. Fig. 3

[0083] [Fig. 3] shows an example of the process according to certain embodiments. Fig. 4

[0084] [Fig. 4] shows an example of dialogue between a user and an interactive computer application according to certain embodiments. Description of the implementation methods

[0085] Reference is now made to Figure 1.

[0086] The Diall dialogue is an example of a dialogue between a user ut1 and an interactive computer application Applntl according to the state of the art.

[0087] The Diall dialogue takes place via an interactive application interface, presented as a chatbot, between a user Ut1 and the interactive application Applntl. In this example, the user wants to reserve a room. All messages are exchanged through a chat window within the interactive application, where the user can type and read messages.

[0088] In step Et1, user Ut1 starts the interactive application Applntl, or alternatively, starts an online booking tool, which causes the launch of an interactive application Applntl dedicated to interactions with users of this tool.

[0089] In a second step Et2, the interactive application Applntl writes the message "What can I do for you?".

[0090] In a third step Et3, user Ut1 writes the message "Book a room".

[0091] In a fourth step Et4, the interactive application Applntl writes the message "On which site?".

[0092] In a fifth step Et5, user Ut1 writes the message "On site S".

[0093] In a sixth step Et6, the interactive application Applntl writes the message "What day?".

[0094] In a seventh step Et7, user Ut1 writes the message "on April 19th from 2pm to 3pm".

[0095] In an eighth step Et8, the interactive application Applntl writes the message "From what time?".

[0096] In a ninth step Et9, user Ut1 writes the message "From 2pm to 3pm".

[0097] In a tenth step Et10, the interactive application Applntl writes the message "For how many people?".

[0098] In an eleventh step Et1, user Ut1 writes the message "5".

[0099] In a twelfth step Et12, the interactive application Applntl writes the message "Let's recap: you want to reserve room A bridge on April 19th from 2pm to 3pm for 5 people. OK?".

[0100] In a thirteenth step Et13, user Ut1 writes the message "yes".

[0101] In a fourteenth step Et14, the interactive application Applntl makes the reservation, via a room reservation API.

[0102] In a fifteenth step Et15, the interactive application Applntl writes the message "Room reserved!".

[0103] In this example, the interactive application Applntl allows the user to book a room through a simple conversation. However, the number of exchanges with the Applntl application required for this booking is high. For example, a message exchange with a question from the interactive application and a response from the user is necessary for each booking parameter (day, time, room, number of people, etc.). This limits the ease of use of the interactive application and increases the exchange time. In this example, the exchange lasted more than a minute. Current state-of-the-art solutions therefore do not fully exploit the potential of interactive applications to simplify user actions.

[0104] Figure 2 shows an example of an electronic device suitable for implementing the method of this application according to at least some embodiments.

[0105] The electronic device Disp includes at least one computing unit Cale capable of running a probe orchestration module Sond, a context building module Cont and an interactive computer application Applnt2.

[0106] The term "probe" refers to a computer module capable of obtaining information from an Int (software or hardware) interface of the electronic device.

[0107] The term "at least one human-machine interface of said device" means at least one human-machine interface coupled to the device, that is to say, a human-machine interface integrated into the device, or a human-machine interface connected to the device by means of wired or wireless communication.

[0108] For example, at least one human-machine interface Int may include a screen Ecr, and / or at least one microphone Micr and / or at least one speaker Htp.

[0109] The probes can thus be capable of obtaining information from at least one interface. For example, a probe associated with the screen (Ecr) can provide the content displayed on the screen, a probe associated with the screen and the microphone (Micr) can detect the sounds picked up by the microphone, a probe associated with the screen and the speaker (Htp) can detect the sounds and words emitted by the speaker, and so on. Other probes can retrieve all types of data related to the environment of the electronic device: location, temperature, ambient light, etc.

[0110] The Sond probe orchestrator is capable of coordinating probes and extracting information about user activity: acquired or rendered words (textual or spoken), HTML content of an email, switching between active windows, names of active applications, visual content of applications, audio and video streams, etc. In certain embodiments, the Sond probe orchestrator can process the raw data provided by the probes to extract useful information. For example, the probe orchestrator can implement a character recognition algorithm to detect words or named entities, or to identify the names of active applications from a video stream.

[0111] The interactive computer application (or simply "interactive application") Applnt2 is a computer application capable of interacting with a user via at least one interface (Int) to help the user perform actions related to an online tool / service (such as a room booking service, airline ticket booking, hotel booking, etc.). The interactive computer application Applnt2 could, for example, be a virtual assistant in the form of a dialogue agent such as a chatbot, or a voice assistant.

[0112] The interactive application exposes at least one intent. As mentioned above, an exposed intent corresponds to a goal that interactions within the interactive application allow the user to pursue, linked to an action performed through the application. Intents can be exposed, for example, via an API, allowing the context building module Cont to understand the intents exposed by the interactive application Applnt2.

[0113] At least one exposed intent is associated with at least one action to be performed by the interactive application. For example, an exposed intent "room_booking" could be associated with an action to actually book the room, which could be executed by the interactive application via an API of a room booking service. If the interactive application Applnt2 exposes several intents, each exposed intent can be associated with one or more actions; for example, the exposed intent "room_booking" with an action to actually book a room, the exposed intent "train_booking" with an action to actually book a train ticket, and so on.

[0114] Although only one interactive application, Applnt2, is shown in Figure 2, in some embodiments several interactive applications can be used. For example, different interactive applications can expose different intentions. It is thus possible, for instance, to have a "general-purpose" interactive application that handles (exposes) many intentions, or conversely, several "specialized" interactive applications that expose only one or a limited number of intentions. If several expressed intentions are detected, they can be correlated with different exposed intentions, which can then be provided, if necessary, to one or more interactive applications.

[0115] The Cont context building module is capable of interacting with the Sond probe orchestrator as well as the Applnt2 interactive computer application.

[0116] The interaction between the context building module Cont and the probe orchestrator Sond allows the context building module to gather information enabling it to construct a context. Semantic analysis of the context identifies expressed intentions and groups / filters contextual elements based on these identified intentions. These expressed intentions are then correlated with exposed intentions to determine one or more exposed intentions to be provided to one or more interactive applications.

[0117] The interaction between the Cont context building module and the Applnt2 interactive application provides the Applnt2 interactive application with context information that helps for a smoother interaction with the user.

[0118] The electronic device Disp can take different forms depending on the embodiment. For example, it can be a personal computer, a tablet, a smartphone, or any other type of electronic device capable of interacting with a user via one or more interfaces.

[0119] In some embodiments, a computer system in which the different modules are executed by a plurality of computing units distributed across several computing devices can be used, rather than a single device. For example, the interactive computer application Applnt2 can be implemented and executed on an electronic device separate from the probe orchestrator Sond and the context building module Cont.

[0120] Reference is now made to Figure 3, which illustrates some embodiments of the process (or method) of the present application.

[0121] The P3 process is a process implemented by at least one electronic device, for example the Disp device.

[0122] In the illustrated example, the P3 process may include an S31 step of obtaining a context of the electronic device.

[0123] Step S31 can, for example, be executed by the context building module Cont, from the data provided by the probe orchestrator Sond.

[0124] The Cont context building module can, for example, detect, from the data provided by the Sond probe orchestrator, changes in the display of a device screen (e.g., changes in active windows, changes in the visual rendering of an application...), and perform processing such as useful area analysis, word detection or OCR (Optical Character Recognition) in order to extract useful information displayed on the screen.

[0125] Similarly, the Cont context building module can analyze sounds captured by a microphone or emitted by a speaker, near the device for example.

[0126] This could include, for example, sounds captured and / or played back by / for a user of the device and / or by a third party).

[0127] The resulting context can take various forms. For example, the context can be formalized using a markup format. For instance, the content displayed by the "Outlook" email application could be formalized as follows: { 'process': 'outlook.exe', 'title': 'actions to prioritize', 'data': [{ 'type': 'text', 'bbox': {x:12, y:34, w:750, h:560} 'value': 'Hello, Would it be possible to arrange a visit [...] Best regards, JF' }], }

[0128] In the illustrated example, the P3 process may include an S32 determination of at least one first intention of at least one interactive computer application.

[0129] Depending on the embodiment, steps S31 and S32 can be executed sequentially or simultaneously. Similarly, depending on the embodiment, the first determined intention can be obtained by correlation with an explicitly expressed (literally) intention included in the context, or with an implicitly expressed intention, deduced from elements of the context without being literally included in the context itself.

[0130] In the illustrated example, the determination of at least one first intention can be carried out by correlation with an expressed intention obtained from at least one element of the electronic device's context that is not linked to a human-machine interface of the interactive computer application; that is, it can be determined from at least one context element that was not obtained through inputs / outputs related to the interactive application itself. For example, in some embodiments, at least one first intention can be determined by correlation with an expressed intention obtained solely from context element(s) not obtained through inputs / outputs related to the interactive application itself.

[0131] For example, if the interactive computer application takes the form of a chatbot with a conversation window, at least one initial intent can be determined by correlation with an expressed intent obtained from contextual elements unrelated to the conversation window. These could be, for example, contextual elements related to a window in another application (originating from such a window, for instance), or contextual elements related to a voice conversation (such as a voice conversation initiated by the user) in a computer chat application unrelated to the interactive computer application itself.

[0132] In some embodiments, the construction of the context and / or the determination of at least one first intention may include the recognition of named entities, for example via the use of classifiers, rule engines, or supervised learning engines, for example of the LLM (Large Language Model) type.

[0133] For example, determining at least one initial intention might include: a semantic analysis of the elements of the obtained context, leading to a grouping of at least some context elements based on their semantics (a context element may belong to several groupings). The semantic analysis might include recognizing named entities in the context related to an expressed intention; associating an expressed intention, representative of the semantics of the grouped elements, with at least one (e.g., each) grouping of context elements; and determining at least one initial intention among the intentions displayed by at least one interactive application, by correlating the detected expressed intention with the intentions displayed by the interactive application.

[0134] For example, if a new email appears on the screen in an email application (such as "Outlook" ©) and contains the text: "Hello, would it be possible to arrange a visit to the JFLCORP company on November 18th? Also, would it be possible to reserve a room on November 20th from 10:00 AM to 3:00 PM for a debriefing session with the manager? Best regards, JF.", the following context can be constructed from the email: [{ 'process': 'outlook.exe', 'title': 'actions to prioritize', 'data': [{ 'type': 'intent', 'value': 'organize a visit', 'metadata' :[{ 'category': 'Name Entity', 'type': 'company', 'value': 'JFL CORP', 'knowledgeLevel': 0.57, },{ 'category': 'Name Entity', 'type': 'date', 'value': '18 / 10 / 2023' } ] },{ 'type': 'intent', 'value': 'book a room, 'metadata' :[{ 'category': 'Name Entity', 'type': 'interval', 'value': '10 AM - 3 PM', 'knowledgeLevel': 0.57, },{ 'category': 'Name Entity', 'type': 'date', 'value': '20 / 10 / 2023' },{ 'category': 'info', 'type': 'subject', 'value' feedback on the visit with the manager' } ] }], }]

[0135] Here we observe that the context is linked to the 'outlook.exe' application and includes several expressed intentions, in this case "organize a visit" and "book a room". The expressed intentions are correlated with the intentions exposed (processable) by at least one interactive computer application, based on their semantic proximity, to determine (identify) at least one exposed intention that is semantically close to at least one expressed intention.

[0136] For this purpose, the exposed intentions can be retrieved, depending on how they are exposed. For example, intentions can be exposed via web services (REST API, SOAP, etc.), HTTP requests, websockets, and so on.

[0137] In the example context above, the words identified by the various sensors (e.g., displayed on a screen, captured by a microphone, played back by a speaker, etc.) were used to detect two expressed intentions: "organize a visit" and "book a room." In our example, a single intention, "book_a_room," is presented by the interactive application. The expressed intention "book a room" is, for example, more strongly correlated with the presented intention "book_a_room" than the expressed intention "organize a visit." The latter may not be correlated with any presented intention. Since the expressed intention "book a room" is correlated with the presented intention "book_a_room," the presented intention "book_a_room" is therefore determined to be provided to the interactive application.The context elements provided to the interactive application, in association with the determined intent, can be limited to context elements relating to the expressed intentions correlated with an exposed intention (i.e. a determined intention) (in this example the single determined intention "room reservation"), for example to become the following context:. [{ 'process': 'outlook.exe', 'title': 'actions to prioritize', 'data': [{ 'type': 'knowledge, 'metadata' :[{ 'category': 'Name Entity', 'type': 'company', 'value': 'JFL CORP', 'knowledgeLevel': 0.57, } ] },{ 'type': 'intent', 'value': 'book a room', 'metadata' :[{ 'category': 'Name Entity', 'type': 'interval', 'value': '10 AM - 3 PM', 'knowledgeLevel': 0.57, },{ 'category': 'Name Entity', 'type': 'date', 'value': '20 / 10 / 2023' },{ 'category': 'info', 'type': 'subject', 'value' / feedback on the visit with the manager' } ] }], }]

[0138] In this example, the expressed intention "organize a visit", which could not be correlated with any intention exposed by the computer application, was removed from the context to be provided to the interactive application, but some data, in this case the name of the visiting company, was retained for possible use by the interactive application.

[0139] The correlation between expressed and revealed intentions can be established in various ways, for example, through the use of natural language processing (NLP) techniques such as classifiers, rule engines, semantic proximity calculations, and by leveraging a database of expressions associated with the revealed intention. Thus, a revealed intention can be determined if the words used correspond exactly to the intention. expressed, but also in the case of the use of synonyms, abbreviations, words in foreign languages, etc.

[0140] The association of contextual elements with an expressed intention and / or the correlation between an expressed intention and a stated intention can be carried out after preliminary training phases, in certain embodiments. These training phases can rely on training sentences associated with the expressed intention to allow for the subsequent classification of contextual elements as associated with at least one expressed intention (the same contextual element can be classified (associated) with several expressed intentions). For example, an expressed intention "to reserve a room" can be associated with different words, groups of words, and / or training sentences, such as "I would like to reserve a room," "We would need a room for the...", "meeting," etc.The context can therefore be associated with an expressed intention based on a semantic proximity with certain words, or groups of words or phrases used for training related to this expressed intention, rather than the name of the expressed intention alone, which further improves detection.

[0141] Expressed (and determined) intentions can be associated with named entities recognized in the context. For example, the expressed intention "Book a room" is associated with the named entities "date" and "interval," which are known parameters of a room reservation. Thus, inferring (e.g., extracting) an expressed intention from at least one word can involve recognizing in the context the named entities associated with the expressed intention—in this case, the reservation dates and intervals mentioned in the email. Other information, which does not constitute named entities, can be identified and associated with the expressed intention as information. For example, the information "debriefing of the visit with the manager" has been identified and can be associated with the expressed intention.

[0142] These named entities and additional information form contextual elements related to the expressed intent under consideration, and therefore to at least one initial intent correlated with that intent. The named entities and additional information can thus be provided to the interactive computer application in addition to the identifier of the determined intent, in order to allow the interactive application to have the most precise information possible to configure the action and / or establish an interaction with the user.

[0143] The context of the electronic device may also include contextual elements of the electronic device's environment.

[0144] For example, the context may include the device's position, the time, the ambient light, or the user's presence near the device.

[0145] As will be explained in more detail below, these elements make it possible to provide contextual information related to a specific intention for the interactive computer application and / or to filter specific intentions based on the device's environment (or, alternatively, to filter expressed intentions before correlating them with displayed intentions). Contextual elements relating to the environment make it possible, for example, to know if it is day or night, if the device is used in a personal or professional setting, etc.

[0146] Steps S31 and / or S32 can be carried out in different ways depending on the embodiments.

[0147] For example, in some embodiments, steps S31 and / or S32 can be performed continuously, that is, constantly at regular intervals. For example, steps S31 and / or S32 can be performed every second, 10 seconds, minutes, etc., or at variable intervals until an intention is determined, in order to ensure that intention determination occurs continuously, with configurable temporal precision. Performing steps S31 and / or S32 makes it possible to limit (in a deterministic manner) the delay between the occurrence of contextual elements representative of an expressed intention and the possible delivery of an exposed intention correlated with that expressed intention to at least one computer application.

[0148] In some embodiments, steps S31 and / or S32 may be carried out when an event representative of a change occurs in the context of the electronic device.

[0149] Such an event could be, for example: the detection of a new word; a change of window; the opening of a message; etc.

[0150] Put another way, steps S31 and / or S32 are carried out in this case when an event representative of a change in context occurs, which makes the occurrence of the conditions for detecting a new expressed intention more likely.

[0151] Performing steps S31 and / or S32 when an event representative of a change in the context of the electronic device occurs reduces the computational load of at least one electronic device computing unit while retaining the ability to determine the intentions to be provided to at least one computer application when necessary, since steps S31 and / or S32 are executed only when an event representative of a change in the context of the device makes the expression of a new intention in the context more likely.

[0152] The process may include an S33 check to determine whether an exposed intent has been identified. The S33 step consists of verifying whether an exposed intent that is semantically close to an expressed intent, and therefore can be provided to the interactive computer application, has been identified. This check may be performed following an S35 filter (see below) and may be optional in some embodiments.

[0153] When no validly provided intention for the interactive computer application has been determined (e.g., the output of step S33 is negative), the process can return to step S31, and so on until an intention is determined.

[0154] When, on the contrary, an intention has been determined, and where appropriate filtered, the P3 process then includes a provision S34 of said at least one determined intention to said interactive computer application.

[0155] Thus, the interactive application benefits from prior knowledge of the intended purpose from the moment it is launched / invoked. For example, at least one contextual element not originating from its own interface (external to the interactive application's interaction history with the user, for instance) may be provided along with the intended purpose. At least one initial intention can therefore be determined by correlation with an intention expressed, for example, from contextual elements of applications other than the interactive application itself. For instance, at least one initial intention can be determined through text from emails, words captured via a microphone, and so on.

[0156] Providing the determined intention to the interactive application thus makes it possible to limit the (explicit) interactions between the interactive application and the user before the implementation of a command (calling the online service for example), since the knowledge of at least one element of context not from its own interface makes it possible to determine at least one first intention without interaction via the interface of the interactive computer application itself.

[0157] Providing the specific intent to the interactive application can thus help more generally to facilitate exchanges between an interactive computer application and its users, with a view to performing an action through the interactive computer application or another application.

[0158] The provision of the specified intention to the interactive application may, in certain embodiments, be conditional upon contextual elements of the electronic device's environment (such as a type of common use ("work", "leisure", etc.). For example, in some embodiments, specified intentions associated with a "work" type of use may only be sent during office hours (and not outside of them).

[0159] According to some embodiments, the interactive computer application may be an interactive computer application already running (for example, if the interactive computer application is a chatbot, virtual assistant or voice assistant running continuously in the background) or not (for example, if the interactive computer application is a chatbot, virtual assistant or voice assistant launched and run on demand).

[0160] In the case where the interactive computer application is an interactive computer application already running, the determined intent (and at least one possible context element) can be provided directly to the running interactive computer application.

[0161] Otherwise, the process may include starting (launching) the interactive computer application prior to providing the determined intent (and at least one possible contextual element).

[0162] As noted above, at least one element of context relating to the expressed intent, for example named entities, can be provided in addition to the determined exposed intent, in order to facilitate interaction between the interactive application and the user.

[0163] For example, the interactive computer application can be invoked without user action as soon as an intention is determined, and if necessary filtered, and the determined intention can be provided to the interactive computer application along with context elements related to the determined intention, so that the first interaction of the interactive application is already targeted to the actual needs of the user (and takes into account provided context elements, without requiring the user to enter them).

[0164] Using the context example associated with the "Outlook.exe" application above, the expressed intent "book a room" is linked to the entities named "Interval" (value "10am-3pm") and "Date" (value "10 / 20 / 2023"). In this example, the two entities named "Interval" (value "10am-3pm") and "Date" (value "10 / 20 / 2023") can be provided to the interactive application along with the determined intent "book a room," which is correlated with the expressed intent "book a room." Without the user even needing to invoke the interactive application, it can send them an initial personalized message based on the context, such as: "Would you like me to book a room on November 20th from 10am to 3pm? If so, for how many people?"

[0165] Such implementations can help facilitate interaction between the application and the user by limiting the number of exchanges (some exchanges can even be avoided). For example, unlike the example in Figure 1, the interactive application in the example in Figure 3 does not need to send a message to request the stated intention to pursue, nor the date or time slot. A simple message requesting confirmation of the information already detected may, for instance, suffice in some cases before performing the action associated with the determined intention.

[0166] Thus, the interactive application can, upon being invoked via the current request process (and therefore upon its initial interaction with the user), send messages (such as questions) that depend on the provided context, and configure the action to be performed based on the provided context and the responses received to these messages. Sending context-dependent messages can reduce the number of messages exchanged with a user to configure the action to be performed, and can therefore help facilitate interaction between the user and the interactive application.

[0167] Furthermore, the sending of the determined intent and associated parameters can occur automatically as soon as the electronic device's interfaces transmit information that allows for the detection of an expressed intent. For example, an intent to be sent can be determined as soon as a change in context makes it possible to detect a new expressed intent: detection of a new word in displayed text, a new word exchanged in a voice conversation, receipt of a new email, a window switching, etc. The user therefore does not need to initiate the interaction with an interactive application such as a chatbot; this is done automatically.

[0168] To this end, an exposed intention can be provided to the interactive application as soon as it has been determined to be correlated with an expressed intention, and filtered where appropriate.

[0169] In some embodiments, the S33 process may include an S35 filtering of at least one determined first intention.

[0170] At least one initial intent is therefore conditionally provided to the interactive computer application. Providing at least one initial intent to the computer application conditionally ensures that only those intents that meet at least one filtering criterion are provided to the interactive computer application.

[0171] The filtering criteria can be of different types.

[0172] For example, the determined intentions can be filtered according to the context in which the method is executed. For example, if the electronic device is a professional electronic device and / or if the process is performed during office hours, only the determined intentions relating to a professional application can be retained.

[0173] The filtering can be configured, for example by a user or administrator of the electronic device.

[0174] For example, the user or administrator can choose to filter exposed / determined intents based on classes of exposed intents (e.g., intents related to a work goal, a personal goal, etc.) or contextual elements. For instance, the user can choose to keep only exposed / determined intents related to work applications. The filtering parameters can thus be modified according to the user's or administrator's needs.

[0175] Filtering can also be done based on the context of the electronic device.

[0176] For example, filtering may only retain expressed or determined intentions related to a given company, a given person, etc.

[0177] Filtering can also be performed based on environmental data of the electronic device. For example, filtering can depend on the device's location, the time of day, the temperature and / or ambient light, etc.

[0178] For example, if the time and / or position of the device indicate that the device is used in a professional setting (location in offices and / or office hours for example), only the stated / determined intentions related to a professional context can be retained.

[0179] Reference is now made to figure 4.

[0180] Figure 4 shows an example of a Dial4 dialogue between a user Ut4 and an interactive computer application Applnt4 according to certain embodiments.

[0181] In this example, the following email was received:

[0182] "Hello, would it be possible to organize a visit to the JFLCORP company on November 18th? Also, would it be possible to reserve a room on November 20th from 10am to 3pm for a debriefing session with the manager? Sincerely, JF."

[0183] In this example, already studied above, in a first step Et41, a context is established, and an expressed intention "book a room" is detected, leading to the following context (associated with the determined intention "room_booking") which is sent to the interactive computer application Applnt4: [{ 'process': 'outlook.exe', 'title': 'actions to prioritize', 'data': [{ 'type': 'knowledge, 'metadata' :[{ 'category': 'Name Entity', 'type': 'company', 'value': 'JFL CORP', 'knowledgeLevel': 0.57, } ] },{ 'type': 'intent', 'value': 'reserve room', 'metadata' :[{ 'category': 'Name Entity', 'type': 'interval', 'value': '10 AM - 3 PM', 'knowledgeLevel': 0.57, },{ 'category': 'Name Entity', 'type': 'date', 'value': '20 / 10 / 2023' },{ 'category': 'info', 'type': 'subject', 'value' / feedback on the visit with the manager' } ] }], }]

[0184] In a second step (Et42), the interactive application sends an initial message to validate the information contained in the context: "Would you like me to reserve a room on November 20th from 10am to 3pm?". A notification is displayed with the message.

[0185] In a third step Et43, the user Ut4 clicks on the notification, which validates the message.

[0186] In a fourth step Et44, the interactive application Applnt4 opens a dialog window with the phrase "For how many people?".

[0187] In a fifth step Et45, user Ut4 responds "5".

[0188] In a sixth step Et46, the interactive application Applnt4 sends the message "Let's recap: you want to reserve a room on November 20th from 10am to 3pm for 5 people. OK?".

[0189] In a seventh step Et47, user Ut4 writes the message "yes".

[0190] In an eighth step Et48, the interactive application Applnt4 makes the reservation, via a room booking API.

[0191] In a ninth step Et49, the interactive application Applnt4 writes the message "Room reserved!".

[0192] It is observed that the number of interactions between the user and the interactive application in the Dial4 dialogue is significantly lower than the number of interactions between the user and the interactive application in the prior art Diall dialogue. The reduction in the number of interactions within the framework of the invention is primarily due to the fact that the user does not need to launch the interactive application themselves, nor manually express their intention, nor provide certain information already contained in the email (notably the date and time range). The interaction between the user and the interactive application can therefore be facilitated. Thus, in this example, the duration of the Dial4 dialogue can be less than 10 seconds, whereas it could be more than 1 minute in the case of the prior art Diall dialogue.

[0193] This disclosure is not limited to the examples of process, product, computer program and electronic device described above, only by way of example, but encompasses all the variants that a person skilled in the art may consider in the context of the protection sought.

Claims

Claims

1. Method (P3) implemented by an electronic device for processing data by at least one interactive computer application, comprising: obtaining (S31) a context of the electronic device; providing (S32, S34), to at least one interactive computer application, at least one first intention, exposed by said at least one interactive computer application, from at least one element of the context of the electronic device not linked to a human-machine interface of the at least one interactive computer application, said at least one exposed intention being associated with at least one action to be carried out via the at least one interactive computer application.

2. Method according to claim 1, comprising: providing at least one element of the context, relating to the at least one first intention, to the at least one interactive computer application.

3. A method according to any preceding claim, wherein said method comprises starting said interactive computer application once said first intention has been determined.

4. A method according to any preceding claim, wherein said providing takes into account a correlation between at least one so-called expressed intention obtained from said context, and said first intention.

5. Method according to the preceding claim, wherein: said context element comprises at least one word rendered or acquired via a human-machine interface of the electronic device; said correlation takes into account a semantic proximity between said at least one expressed intention and said first intention.

6. Method according to the preceding claim, where the method comprises detecting said at least one word rendered or acquired via said human-machine interface of the electronic device.

7. Method according to the preceding claim, comprising recognizing, among the at least one word, at least one named entity associated with said at least one expressed intention.

8. Method according to any one of the preceding claims, comprising: filtering the at least one first intention; providing said at least one first intention at the end of the filtering.

9. The method of claim 6, wherein the filtering is performed based on the context of the electronic device.

10. A method according to any preceding claim, wherein obtaining a context of the electronic device, and / or determining the at least one first intention to be provided is performed continuously.

11. Method according to any one of the preceding claims, in which obtaining a context of the electronic device, and determining the at least one first intention to be provided is carried out upon the occurrence of an event representative of a change in the context of the electronic device.

12. Electronic device comprising at least one computing unit configured, for the processing of data by at least one interactive computer application, to: obtain (S31) a context of the electronic device; provide (S32, S34), to at least one interactive computer application, at least one first intention, exposed by said at least one interactive computer application, from at least one element of the context of the electronic device not linked to a human-machine interface of the at least one interactive computer application, said at least one exposed intention being associated with at least one action to be carried out via the at least one interactive computer application.

13. Computer program product comprising instructions for implementing the method according to any one of claims 1 to 11, when said program is executed by a processor.

14. A non-transitory, computer-readable recording medium on which is recorded a computer program comprising instructions for implementing the method according to any one of claims 1 to 11, when said program is executed by a processor.

Images