Contextualizing an interactive computer application

Abstract

The invention relates to a method implemented by an electronic device, comprising: obtaining a context of the electronic device; determining at least one first intention of at least one interactive computer application from at least one element of the electronic device's context not related to a human-machine interface of the at least one interactive computer application, said at least one first intention being determined from among at least one intention exposed by the at least one interactive computer application, said at least one exposed intention being associated with at least one action to be performed via the at least one interactive computer application; providing said at least one determined first intention to the at least one interactive computer application. Abstract Figure: Figure 2

Description

Title of the invention: Contextualization of an interactive computer application Technical field

[0001] The present disclosure relates to the field of data processing by interactive computer applications. Prior art

[0002] An interactive computer application, or simply "interactive application" is an application allowing a user to command actions, via a simple interaction (in natural language or by mouse click for example) with the interactive computer application, and assisting for example the user in the formulation of this command (so as for example to simplify its formulation for the user). It may be an action to be carried out by the interactive application itself or an action which must be carried out by another application (or service, or tool) with which this interactive computer application is associated, so as for example to simplify the use of this other application for the user.For example, some chatbots are interactive computer applications that present themselves as a dialog window with the user, allowing them to perform tasks (or order their completion). In this case, the interaction is carried out via a dialog window displayed on a screen of a user's terminal. The chatbot carries out interactions in natural language with the user, via a question-and-answer game for example. Natural language interactions make it possible to determine an action to be performed corresponding to the user's objectives, to determine all the parameters of the action to be performed and finally to carry out this action when all the parameters for carrying out the action are known.Natural language interactions with the dialog agent thus avoid the user having to enter the action command themselves (for example directly via an interface of another application allowing the action to be carried out), the action command interface possibly being less user-friendly than a dialog agent and possibly also being unknown to the user. For example, if the user wishes to reserve a meeting room via a service such as an online electronic reservation tool, the user's dialogue with the interactive computer application can make it possible to determine the day and time of reservation, the number of participants, etc. before making the reservation via the online tool.

[0003] Another type of interactive application concerns voice assistants. Such a voice assistant allows a user to perform one or more tasks via a voice dialogue with the assistant In this case, the interaction can, for example, take place via a microphone and speakers, as if the user were carrying out a dialogue with a human assistant.

[0004] Interactive computer applications are increasingly used, for example in the professional field, to carry out tasks such as booking meeting rooms, train or plane tickets, etc. Interactive computer applications can provide a simpler and more user-friendly solution for users than standard interfaces.

[0005] Interactive applications according to the state of the art require that the user precisely define the object of his exchange (to understand the command to be launched and obtain its input parameters), which induces numerous interactions with the interactive computer application, and thus increases the time of the interaction, and limits the benefit likely to be provided by these interactive computer applications. In particular, interactions with interactive applications according to the state of the art can sometimes appear boring and synonymous with wasting time for a user.

[0006] There is therefore a need to facilitate exchanges between an interactive computer application and its users, with a view to carrying out an action by the interactive computer application or another application. Summary

[0007] The present disclosure improves the situation.

[0008] A method implemented by an electronic device is proposed, comprising: obtaining a context of the electronic device; determining at least one first intention of at least one interactive computer application from at least one element of the context of the electronic device not linked to a human-machine interface of the at least one interactive computer application, said at least one first intention being determined from at least one intention exposed by the at least one interactive computer application, said at least one exposed intention being associated with at least one action to be carried out via the at least one interactive computer application; providing said at least one determined first intention to the at least one interactive computer application.

[0009] By "context of the electronic device" is meant any data relating to the use of the electronic device or its environment at a given time (or instant).

[0010] A context of the electronic device may for example include data displayed on a screen of the device, sound data captured by a microphone or emitted by a loudspeaker connected to the device, environmental data such as the location of the device or the ambient brightness, etc. The context may also take into account the location and date, the physical posture of the person, eye tracking, etc.

[0011] The term "at least one element of the context of the electronic device not linked to a human-machine interface of the at least one interactive computer application" means at least one element of the context which is not affected by the at least one interactive computer application. It may for example be at least one element of context not belonging to a human-machine interface of the at least one interactive computer application, or even a context element of an application other than the at least one interactive computer application.

[0012] "At least one intention exposed by the at least one interactive computer application" means a goal that the interactions of the application allow to be pursued. For example, the at least one computer application can expose via an API (English acronym meaning "Application Programming Interface") at least one intention that it is able to use.

[0013] “Expressed intention” means an intention detected by analyzing the context of the device and in particular captured context elements. This may be, for example, a goal or objective expressed in a sentence or text (corresponding to a user statement, an extract from a document (email, etc.), one (or more) sentences spoken orally and captured by a microphone, etc.), an automatically generated description of an image of the context, etc. It may also be an intention expressed implicitly by a user (for example, the sentence “I have to buy my train ticket” can make it possible to detect the expressed intention “book a train”). An expressed intention makes it possible to categorize the intention of an end user in the context of an exchange with the at least one interactive computer application, and may be based on a text extracted from any type of captured context element (text, audio, image, etc.).

[0014] An expressed intention can be associated with the contextual elements which enabled its detection.

[0015] "Determined intent" means a determined (selected) intent, from among the intents exhibited by an interactive application, as an intent to be provided to the interactive application. The intent may be determined by correlation between an intent expressed in the context of the electronic device and the intents exhibited by the interactive application.

[0016] For example, natural language processing tools (NLP tools) allow words in the context to be associated with intentions expressed in the context. For example, sentence patterns can be recognized to identify the intention expressed by the words in the context. Natural language processing tools then make it possible to correlate the expressed intentions thus obtained with the different intentions exposed by the interactive application.

[0017] Different user statements can thus make it possible to obtain the same expressed intention, correlated (semantically for example) to the same exposed intention. For example, the statements "What is your telephone number?", "Where can I reach you by telephone?", or even "Give me the telephone number, please." could correspond to the same expressed intention "retrieve a telephone number", this being able to be correlated to the intention exposed by the interactive application "retrieve_telephone_number".

[0018] In another example, the user statements “Think about finding a place to meet tomorrow from 2 p.m. to 4 p.m.”, or “We need to reserve a meeting room at 2 p.m. tomorrow for 2 hours” may correspond to the same expressed intention “reserve a room”, which may be correlated with the expressed intention “room_reservation” of the interactive application.

[0019] Automatic recognition of the user's expressed intention can help the interactive computer application to respond "directly" to it (more directly than prior art solutions), via the selection of the corresponding stated intention.

[0020] The analysis of words in the context (for example, their classification) to obtain expressed intentions, then the correlation between the expressed intentions and the exposed intentions, makes it possible to establish a link between the words identified in the context and the exposed intentions. An expressed intention and an exposed intention are both presented as a string of alphanumeric characters, comprising at least one word in natural language, or a word extract such as a root or a lemma of this word. An expressed intention or an exposed intention can be a synthetic group of words, for example, or a statement in natural language. Examples of expressed intentions (such as "reserve a room") and examples of exposed intentions (such as "reservation_room") are presented below.

[0021] An expressed intention is associated with the context elements that allowed its recognition (or detection). For simplicity of language, in the present application, the context elements that allowed the detection of an expressed intention are associated with the (determined) intention correlated with this expressed intention. Thus, in the present application, the context elements "relating" to a determined intention refer to the context elements associated with an expressed intention correlated with this determined intention.

[0022] The at least one exposed intent may be exposed by the interactive application in several ways. For example, the at least one exposed intent may be obtained via a file or via one or more messages (via sockets, APIs (English acronym meaning "Application Programming Interface"), REST (English acronym meaning "REpresentational State Transfer"), etc.), such as JSON type messages (English acronym meaning "JavaScript Object Notation"), describing a list of intentions exposed by the interactive application (i.e. part of its interface).

[0023] By "action to be performed via the interactive computer application" is meant an action that can be triggered by the interactive computer application itself or via the transmission, by the interactive application, of a computer command to another application, for example an online tool or service. For example, if an exposed intention of the interactive computer application is a reservation of a room via an online room reservation service, this exposed intention can be associated with an action of reserving the room, for example via an API of the reservation service.

[0024] The method of the present application, in at least some of its embodiments, makes it possible to automatically determine, among the intentions exposed by the interactive computer application, an exposed intention semantically close to an intention expressed in the context of the electronic device, and to provide it to the interactive computer application, which will be able to convert this determined intention into an action. Such an embodiment can therefore help to facilitate the interaction of a user of the electronic device and its environment. For example, in certain embodiments, the number of messages to be exchanged between an interactive application and the user, in order to trigger an action, can be limited (for example reduced) compared to the number of interactions which would be necessary if the user used the interactive application only.For example, in some cases, the exchange with the user may be reduced to a single message to which the user may respond by a confirmation click. The method, in at least some of its embodiments, may therefore help to facilitate exchanges between an interactive computer application and at least some of its users, with a view to performing an action by the interactive computer application or another application.

[0025] According to another aspect, there is provided a computer program comprising instructions for implementing all or part of a method as defined in the present application, in any one of its embodiments, when this program is executed by a processor.

[0026] According to another aspect, there is provided a non-transitory, computer-readable recording medium on which such a program is recorded.

[0027] According to another aspect, there is provided an electronic device comprising at least a computing unit configured to execute all or part of a process (i.e. method) as defined in the present application, in its various embodiments.

[0028] The term “computing unit” means an electronic component capable of performing electronic or computer calculations to perform a specific function. A computing unit may designate any type of processor or electronic component capable of performing digital calculations. For example, a computing unit may be an integrated circuit, an ASIC (from the English acronym “Application-Specific Integrated Circuit”, literally in French “integrated circuit specific to an application”, a microcontroller, a microprocessor, a DSP (from the English acronym “Digital Signal Processor”, literally in French “digital signal processor”), a processor, a GPU (from the English acronym “Graphics Processing Unit”, literally in French “graphics computing unit”). A computing unit according to the invention is not limited to a particular type of computing architecture. For example, a processor may implement a Harvard or Von Neumann type architecture.

[0029] The term "memory" means a digital electronic element (device) used to store data. Different types of memories may be used in the invention, such as a read-only memory, a random access memory, a volatile memory or a flash memory. An electronic device according to the invention may be equipped with one or more non-volatile memories which may be of different types such as mass memories, flash memory, read-only memories or SSD memories. A device according to the invention may also comprise one or more random access memories such as RAM, DRAM, SRAM, DPRREAM, VRAM, eDRAM or 1T-SRAM memories.

[0030] The characteristics set out in the following paragraphs may, optionally, be implemented, independently of one another or in combination with one another:

[0031] In some embodiments, the method comprises providing at least one element of the context, relating to the at least one first determined intention, to the at least one interactive computer application.

[0032] For example, context elements useful for configuring and / or specifying the action to be performed and / or establishing an interaction with a view to configuring the action can be transmitted. For example, if the determined intention consists of reserving a train ticket, and the context indicates, via the information displayed on the screen, a reservation for a train from Paris to Dijon in two days, at least a portion of the information displayed can be provided to the interactive application.

[0033] The provision of at least one context element relating to the at least one first intention determined to the at least one interactive computer application makes it possible to provide the interactive application with certain known information making it possible to carry out the action, and / or to establish an interaction in order to configure the action to be carried out.

[0034] In some embodiments, said at least one first intention is determined by correlation between at least one so-called expressed intention obtained from said context with said at least one intention exposed by the at least one interactive computer application.

[0035] By "correlation between at least one so-called expressed intention and at least one intention exposed by the at least one interactive computer application" is meant a determination of whether at least one expressed intention corresponds to at least one intention exposed by the at least one interactive application. This correspondence is a function of a semantic proximity between the expressed intention and the exposed intention. It may be an exact, literal correspondence, or a partial correspondence between the expressed intention and the exposed intention.

[0036] Determining the at least one first intention by correlation between at least one so-called expressed intention obtained from said context with said at least one intention exposed by the at least one interactive computer application makes it possible to ensure that the at least one first determined intention is semantically consistent with the intentions expressed in the context, and therefore to provide the at least one interactive computer application with at least one first determined intention consistent with the context.

[0037] In some embodiments, said context comprises at least one word rendered or acquired via a human-machine interface of the electronic device; said correlation takes into account a semantic proximity between said at least one expressed intention and said at least one intention exposed by the at least one interactive computer application; obtaining the context comprises detecting said at least one word rendered or acquired via said human-machine interface of the electronic device.

[0038] “A human-machine interface of the electronic device” means a human-machine interface integrated or coupled to the electronic device.

[0039] For example, it may be a word displayed on at least one screen of the electronic device, and / or captured via at least one microphone of the electronic device and / or rendered via at least one loudspeaker of the electronic device and / or present in a description extracted from an image displayed on at least one screen and / or captured by at least one camera of the electronic device.

[0040] The ongoing detection of the at least one death rendered or acquired to obtain / enrich the context (itself allowing the expressed intention to be obtained), and the taking into account, in the correlation, of a semantic proximity between said at least one expressed intention and said at least one exposed intention, help to determine “as soon as possible” the at least one first intention (i.e. from the detection of context elements allowing an intention expressed in semantic proximity to be obtained with at least one less than the intentions expressed by the at least one interactive computer application).

[0041] In some embodiments, the method comprises recognizing, among the at least one word, at least one named entity associated with said at least one expressed intention.

[0042] By "named entity" is meant a word, or a group of words categorizable into classes such as names of people, names of organizations or companies, names of applications, names of places, quantities, distances, values, dates, email, or previously defined categories (in a file, a database, ...).

[0043] Recognition, among the at least one word, of at least one named entity associated with said at least one first determined intention may help to identify context elements associated with the expressed intention. The named entities may themselves be context elements. These context elements may be provided, in certain embodiments, to the interactive computer application, in association with the determined intention, to facilitate interaction with a user of the device (these context elements being likely to form parameters of the action to be performed).

[0044] In certain embodiments, the method comprises: filtering the at least one first determined intention; providing said at least one first determined intention at the end of the filtering.

[0045] Filtering the at least one first determined intention makes it possible to provide an intention exposed by the interactive application and correlated with an expressed intention only if it satisfies filtering criteria.

[0046] In embodiments where a context is provided to the interactive application in addition to the determined intent, the context elements may also be filtered before sending to the interactive application, for example to provide only those elements relevant to the expressed intent correlated with the determined intent.

[0047] In certain embodiments, the filtering is configurable.

[0048] The filtering setting allows the user, or an administrator of the electronic device, to limit the use of the interactive application (via its stated intentions).

[0049] In some embodiments, the filtering is performed based on the context of the electronic device.

[0050] Performing filtering based on context can help refine filtering rules and make filtering settings more precise. For example, filtering can be done based on context elements such as location, date, or time.

[0051] In one embodiment, the filtering of a determined intention takes into account an exclusion rule based on context elements associated with an intention. expressed correlated with the determined intention.

[0052] For example, an interactive application may expose an intent "recipe" associated with an exclusion rule "chocolate", in which case an exposed intent "recipe" will only be provided to the interactive application if the context does not mention "chocolate".

[0053] According to another example, certain stated intentions which involve payment may only be authorized for a user with an “adult” profile.

[0054] In certain embodiments, the interactive computer application is configured to transmit at least one message on said human-machine interface of said interactive computer application, the message taking into account said first determined intention and said at least one context element relating to said at least one first determined intention.

[0055] The term “emitting at least one message on said human-machine interface” means using an output of the human-machine interface to produce a message understandable by the user. The message may for example be a text message, but also an audio message, a video message or even a visual message such as the emission of a given light, or the performance of a particular movement by a robotic arm.

[0056] The transmission of a message taking into account said first determined intention and said at least one context element makes it possible to transmit a message dependent on the context of the device intended for the user, for example with a view to obtaining validation or clarification from the user.

[0057] In some embodiments, the interactive computer application is configured to: receive, via the input human-machine interface of said interactive computer application, at least one response message; configure the at least one action to be performed based on the at least one response message.

[0058] For example, the interactive computer application may be a virtual assistant integrating a conversational robot (also called a "chatbot" in English terminology), and the conversational robot may exchange messages with the user via a messaging system, in order to confirm that the first determined intention provided to the conversational robot corresponds to an effective intention of the user (i.e. to an action that the user actually wishes to order) and / or to collect additional information.

[0059] Similarly, the interactive computer application may be a voice assistant establishing a discussion with the user, in order to confirm that the first determined intention provided to the conversational robot corresponds to an effective intention of the user and / or to collect additional information.

[0060] For example, if a specific intention* train reservation » aims to reserve a ticket train, and if contextual information unrelated to the interactive application (e.g. extracted from an email) provided to the interactive application indicates that the user wishes to book a train from Paris to Dijon in two days, the conversational robot or virtual assistant may present to the user the timetables for trains from Paris to Dijon in two days, and ask him which timetable suits him, before initiating the action of booking the train ticket.

[0061] Receiving a response message, and setting the action to be performed based on the response message, can help to specify the effective intention of the user in order to set the action to be performed, while limiting the exchanges by taking into account the information extracted from the context by the interactive application, before its exchange with the user.

[0062] In some embodiments, the context of the electronic device comprises context elements of the environment of the electronic device.

[0063] Taking into account contextual elements of the environment of the electronic device can help to only carry out the provision of the determined intention and / or filter the determined intentions when certain conditions are met (for example, only during office hours, or when the device is in a given location) and / or to enrich the context provided to the interactive application with environmental data.

[0064] In some embodiments, at least one of the steps of obtaining a context of the electronic device, and determining the at least one first intention, is performed continuously.

[0065] By "is carried out continuously" is meant the execution of a step at successive intervals in time without significant stopping, for example periodically with a defined time interval (parameterizable and / or constant for example).

[0066] Continuously carrying out at least one of the steps makes it possible to ensure that the determination of at least one first intention exposed takes place permanently (or in a quasi-permanent or regular manner), so as to detect “as early as possible” an intention expressed in the context of the device.

[0067] In some embodiments, at least one of the steps of obtaining a context of the electronic device, and determining the at least one first intention is performed upon the occurrence of an event representative of a change in the context of the electronic device.

[0068] Performing at least one of the steps upon the occurrence of an event representative of a change in the context can help to limit the frequency of determinations of expressed intentions, by only performing a determination of expressed intentions in situations (such as a change in the context of the device) making possible a possible detection of a new expressed intention. (leading by correlation with the stated intentions to a new determination of an stated intention).

[0069] In some embodiments, the interactive computer application is a running interactive computer application.

[0070] Providing the determined intent to a running interactive computer application, i.e., an interactive application running in the background, makes it possible to limit the time required before the interactive application processes the determined intent.

[0071] In some embodiments, the method comprises, prior to providing the at least one first determined intention to the interactive computer application, starting said interactive computer application.

[0072] Starting said interactive computer application prior to providing the determined intention makes it possible to start the interactive application only when a determined intention can be provided to it. And therefore helps to limit the CPU load consumed by the interactive application as well as its memory occupation. Brief description of the drawings

[0073] Other characteristics, details and advantages will appear on reading the detailed description below, and on analyzing the attached drawings, in which: Fig.l

[0074] [Fig.l] shows an example of dialogue between a user and an interactive computer application according to the state of the art. Fig. 2

[0075] [Fig.2] shows an example of an electronic device suitable for implementing of the method of the present application according to at least some of its embodiments. Fig. 3

[0076] [Fig.3] shows an example of a method according to certain embodiments. Fig. 4

[0077] [Fig.4] shows an example of a dialogue between a user and an application in interactive computing according to certain embodiments. Description of the embodiments

[0078] Reference is now made to [Fig.l].

[0079] The Diall dialog is an example of a dialog between a user utl and an interactive computer application Applntl according to the state of the art.

[0080] The Diall dialogue is carried out via an interface of an interactive application in the form of a chatbot, between a user Utl and the interactive application Applntl. In this example, the user wishes to reserve a room. All Messages are sent through a chat window in the interactive application, in which the user can enter and read messages

[0081] In a step Etl, the user Utl starts the interactive application Applntl, or alternatively, starts an online reservation tool, which causes the start of an interactive application Applntl dedicated to interactions with the users of this tool.

[0082] In a second step Et2, the interactive application Applntl writes the message “What can I do for you?”.

[0083] In a third step Et3, the user Utl writes the message “Book a room”.

[0084] In a fourth step Et4, the interactive application Applntl writes the message “On which site?”.

[0085] In a fifth step Et5, the user Utl writes the message “On site S”.

[0086] In a sixth step Et6, the interactive application Applntl writes the message “What day ? ".

[0087] In a seventh step Et7, the user Utl writes the message “April 19 from 2 p.m. to 3 p.m.”

[0088] In an eighth step Et8, the interactive application Applntl writes the message “From what time?”.

[0089] In a ninth step Et9, the user Utl writes the message “From 2 p.m. to 3 p.m.”

[0090] In a tenth step EtlO, the interactive application Applntl writes the message “ For how many people?

[0091] In an eleventh step Etl, the user Utl writes the message “5”.

[0092] In a twelfth step Etl2, the interactive application Applntl writes the message “Let’s recap: you would like to reserve room A bridge on April 19 from 2 p.m. to 3 p.m. for 5 people. OK?”.

[0093] In a thirteenth step Etl3, the user Utl writes the message “yes”.

[0094] In a fourteenth step Etl4, the interactive application Applntl makes the reservation, via a room reservation API.

[0095] In a fifteenth step Etl5, the interactive application Applntl writes the message “Room reserved!”.

[0096] In this example, the interactive application Applntl therefore allows the user to reserve a room via a simple conversation. However, the number of exchanges with the interactive application Applntl necessary for this reservation is high. For example, an exchange of messages with a question from the interactive application and a response from the user is necessary for each parameter of the reservation (day, time, room, number of people, etc.). This limits the ease of use of the interactive application, and increases the exchange time. In this example, the exchange time will have lasted for example more than a minute. State-of-the-art solutions therefore do not allow to fully exploit the potential of interactive computer applications to simplify the actions to be carried out by users.

[0097] [Fig.2] shows an example of an electronic device suitable for implementing the method of the present application according to at least certain embodiments.

[0098] The electronic device Disp comprises at least one computing unit Cale capable of executing a probe orchestration module Sond, a context construction module Cont and an interactive computer application Applnt2.

[0099] The term “probe” means a computer module capable of obtaining information from an Int interface (software or hardware) of the electronic device.

[0100] The term “at least one human-machine interface of said device” means at least one human-machine interface coupled to the device, i.e. a human-machine interface integrated into the device, or a human-machine interface connected to the device by wired or wireless communications means.

[0101] For example, the at least one human-machine interface Int may comprise a screen Ecr, and / or at least one microphone Micr and / or at least one loudspeaker Htp.

[0102] The probes may thus be able to obtain information from at least one interface. For example, a probe associated with the screen Ecr may be able to provide the content displayed on the screen, a probe associated with the screen to the microphone Micr may be able to detect the sounds picked up by the microphone, a probe associated with the screen to the loudspeaker Htp may be able to detect the sounds and words emitted by the loudspeaker, etc. Other probes may be able to recover any type of data relating to the environment of the electronic device: location, temperature and ambient brightness, etc.

[0103] The probe orchestrator Sond is capable of coordinating the probes, and extracting information relating to the user's activity: words acquired or rendered (textually or vocally), HTML contents of an email, change of active windows, name of active applications, visual content of applications, audio streams, video, etc. The probe orchestrator Sond may be capable, in certain embodiments, of processing the raw data provided by the probes to extract useful information therefrom. For example, the probe orchestrator may implement a character recognition algorithm to detect words, or named entities, or detect the name of active applications, from a video stream.

[0104] The interactive computer application (or more simply “interactive application”) Applnt2 is a computer application capable of interacting with a user via at least one Int interface to help the user perform actions in connection with an online tool / service (such as a room reservation service, plane tickets, a hotel, etc.). The interactive computer application Applnt2 can by example being a virtual assistant in the form of a dialoguing agent such as a chatbot, or a voice assistant.

[0105] The interactive computer application exposes at least one intention. As indicated above, an exposed intention corresponds to a goal that the interactions of the interactive application allow to be pursued in connection with an action to be carried out via the interactive application. The exposure of intentions can be done for example by means of an API, and allows the context construction module Cont to know the intentions exposed by the interactive application Applnt2.

[0106] The at least one exposed intention is associated with at least one action to be performed by said interactive application. For example, an exposed intention “room_reservation” may be associated with an action of actually reserving the room, which may be executed by the interactive application via an API of a room reservation service. If the interactive computer application Applnt2 exposes several intentions, each exposed intention may be associated with one or more actions, for example the exposed intention “room_reservation” with an action of actually reserving a room, the exposed intention “train_reservation” with an action of actually reserving a train ticket, etc.

[0107] Although only one interactive application Applnt2 has been shown in [Fig. 2], in some embodiments several interactive applications can be used. For example, different interactive applications can expose different intentions. It is thus possible, for example, to have a “generalist” interactive application dealing with (exposing) numerous intentions, or on the contrary several “specialized” interactive applications exposing only one or a limited number of intentions. If several expressed intentions are detected, they can be correlated with different exposed intentions which can be provided, if necessary, to one or more interactive applications.

[0108] The context construction module Cont is able to interact with the probe orchestrator Sond as well as the interactive computer application Applnt2.

[0109] The interaction between the context construction module Cont and the probe orchestrator Sond allows the context construction module to collect information allowing it to construct a context. The semantic analysis of the context makes it possible to identify expressed intentions and group / filter the elements in context according to the identified expressed intentions and then correlate these expressed intentions with the exposed intentions in order to determine one or more exposed intentions to be provided to one or more interactive applications.

[0110] The interaction between the context construction module Cont and the interactive application Applnt2 makes it possible to provide the interactive application Applnt2 with context information helping to facilitate smoother interaction with the user.

[0111] The electronic device Disp may take different forms depending on the embodiments. For example, it may be a personal computer, a tablet, a smartphone, or any other type of electronic device capable of interacting with a user via one or more interfaces.

[0112] In some embodiments, a computing system in which the different modules are executed by a plurality of computing units distributed across multiple computing devices may be used, rather than a single device. For example, the interactive computing application Applnt2 may be implemented and executed on an electronic device separate from the probe orchestrator Sond and the context construction module Cont.

[0113] Reference is now made to [Fig. 3], which illustrates certain embodiments of the process (or method) of the present application.

[0114] The method P3 is a method implemented by at least one electronic device, for example the device Disp.

[0115] In the illustrated example, the method P3 may comprise a step S31 of obtaining a context of the electronic device.

[0116] Step S31 can for example be executed by the context construction module Cont, from the data provided by the probe orchestrator Sond.

[0117] The context construction module Cont can for example detect, from the data provided by the probe orchestrator Sond, changes in the display of a screen of the device (e.g. change of active windows, change of visual rendering of an application, etc.), and carry out processing such as analyses of useful areas, word detection or OCR (English acronym meaning “Optical Character Recognition”) in order to extract the useful information displayed on the screen.

[0118] Similarly, the Cont context construction module can analyze sounds captured by a microphone or emitted by a loudspeaker, near the device for example.

[0119] This may, for example, involve sounds captured and / or rendered by / for a user of the device and / or by a third party).

[0120] The context thus obtained can be presented in different ways. For example, the context can be formalized via a tag format. For example, the content displayed by the “Outlook” email application could be formalized in the following way:

[0121] {

[0122] 'process': 'outlook.exe',

[0123] 'title' : 'actions to prioritize',

[0124] 'data' : [{

[0125] 'type' : 'text',

[0126] 'bbox': {x:12, y:34, w:750, h:560}

[0127] 'value': 'Hello, Would it be possible to organize a visit [...] Kind regards, JF'

[0128] }],

[0129] }

[0130] In the illustrated example, the method P3 may comprise a determination S32 of at least one first intention of at least one interactive computer application.

[0131] According to the embodiments, steps S31 and S32 may be executed sequentially or simultaneously. Similarly, according to the embodiments, the first determined intention may be obtained by correlation with an expressed intention included explicitly (literally) in the context, or an implicitly expressed intention only, deduced from elements of the context without being literally included in the context itself.

[0132] In the illustrated example, the determination of the at least one first intention can be carried out by correlation with an expressed intention obtained from at least one element of the context of the electronic device not linked to a human-machine interface of the interactive computer application, that is to say that it can be determined from at least one context element which has not been obtained by inputs-outputs linked to the interactive application itself. For example, in certain embodiments, the at least one first intention can be determined by correlation with an expressed intention obtained from only context element(s) which have not been obtained by inputs-outputs linked to the interactive application itself.

[0133] For example, if the interactive computer application is in the form of a chatbot with a conversation window, the at least one first intention may be determined by correlation with an expressed intention obtained from context elements that are not linked to the conversation window. This may be, for example, context element(s) relating to a window of another application (from such a window, for example), or context element(s) relating to a voice conversation (such as a voice conversation carried out by the user) in a computer chat application unrelated to the interactive computer application itself.

[0134] In some embodiments, the construction of the context and / or the determination of the at least one first intention may comprise the recognition of named entities, for example via the use of classifiers, rule engines, or supervised learning engines, for example of the LLM (Large Language Model) type.

[0135] For example, the determination of at least one first intention may comprise: - a semantic analysis of the elements of the context obtained, leading to a grouping of at least certain context elements according to their se semantic (a context element that can belong to several groupings). Semantic analysis may include recognition of named entities in the context relating to an expressed intention; - an association with at least one (for example each) grouping of context elements of an expressed intention, representative of the semantics of the grouped elements; - a determination of at least one first intention among the intentions expressed by at least one interactive application, by correlation between the at least one expressed intention detected and the intentions expressed by the at least one interactive application.

[0136] For example, if a new email appears on the screen in an email application (such as "Outlook" ©) and includes the text: "Hello, would it be possible to organize a visit to the company JFLCORP on November 18? In addition, would it be possible to reserve a room on November 20 from 10:00 to 3:00 p.m. for a restitution of the visit with the manager? Kind regards, JF.", the following context can be constructed from the email:

[0137] [{

[0138] 'process': 'outlook.exe',

[0139] 'title' : 'actions to prioritize',

[0140] 'data' : [{

[0141] 'type' : 'intent',

[0142] ‘value’ : ‘organiser une visite’,

[0143] ‘metadata’ :[{

[0144] ‘category’ : ‘Name Entity’,

[0145] ‘type’ : ‘compagny’,

[0146] ‘value’:‘JFLCORP’,

[0147] ‘knowledgeLevel’ : 0.57,

[0148] },{

[0149] ‘category’:‘Name Entity’,

[0150] ‘type’ : ‘date’,

[0151] ‘value’ / 18 / 10 / 2023’

[0152] }

[0153] },{

[0154] ‘type’ : ‘intent’,

[0155] ‘value’ : ‘réserver une salle,

[0156] ‘metadata’ :[{

[0157] ‘category’:‘Name Entity’,

[0158] ‘type’ : ‘interval’,

[0159] ‘value’:‘10h - 15h’,

[0160] ‘knowledgeLevel’ : 0.57,

[0161] },{

[0162] ‘category’ : ‘Name Entity’,

[0163] ‘type’ : ‘date’,

[0164] ‘value’ :‘20 / 10 / 2023’

[0165] },{

[0166] 'category' : 'info',

[0167] 'type' : 'subject',

[0168] 'value': 'restitution of the visit with the manager'

[0169] }

[0170] }],

[0171] }]

[0172] Here we see that the context is linked to the application 'outlook.exe', and includes several expressed intentions, here "organize a visit" and "reserve a room". The expressed intentions are correlated with the intentions exposed (processable) by at least one interactive computer application, according to their semantic proximity, to determine (identify) at least one exposed intention semantically close to at least one expressed intention.

[0173] For this purpose, the exposed intentions can be retrieved, depending on how the intentions are exposed. For example, the intentions can be exposed via web services (REST API, SOAP, etc.), HTTP requests, websockets, etc.

[0174] In the context example above, the words identified by the different probes (for example displayed on a screen, picked up by a microphone, rendered by a loudspeaker, etc.) were for example used to detect two expressed intentions “organize a visit” and “book a room”. In our example, a single intention “room_reservation” is exposed by the interactive computer application. The expressed intention “book a room” is for example more strongly correlated with the exposed intention “room_reservation”, than the expressed intention “organize a visit”. This may not be correlated with any exposed intention. The expressed intention “book a room” being correlated with the exposed intention “room_reservation”, the exposed intention “room_reservation” is therefore determined as to be provided to the interactive application.The context elements provided to the interactive application, in association with the determined intent, may be limited to the context elements relating to the expressed intents correlated with an exposed intent (i.e. a determined intent) (in this example the single determined intent “room reservation”), for example to become the following context: .

[0175] [{

[0176] 'process': 'outlook.exe',

[0177] 'title' : 'actions to prioritize',

[0178] 'data' : [{

[0179] 'type' : 'knowledge,

[0180] 'metadata' :[{

[0181] 'category': 'Dwarf Entity',

[0182] 'type' : 'compagny',

[0183] 'value' : 'JFLCORP',

[0184] 'knowledgeLevel': 0.57,

[0185] } ]

[0186] },{

[0187] 'type' : 'intent',

[0188] 'value' : 'reserve a room',

[0189] ‘metadata’ :[{

[0190] ‘category’ : ‘Name Entity’,

[0191] ‘type’ : ‘interval’,

[0192] ‘value’:‘10h - 15h’,

[0193] ‘knowledgeLevel’ : 0.57,

[0194] },{

[0195] ‘category’:‘Name Entity’,

[0196] ‘type’ : ‘date’,

[0197] ‘value’ :‘20 / 10 / 2023’

[0198] },{

[0199] ‘category’ : ‘info’,

[0200] ‘type’ : ‘subject’,

[0201] ‘value’ :‘restitution de la visite avec le gérant’

[0202] }

[0203] }],

[0204] }]

[0205] In this example, the expressed intention “organize a visit”, which could not be correlated with any intention exposed by the computer application, was removed from the context to be provided to the interactive application, but certain data, in this case the name of the visiting company, was retained for possible use by the interactive application.

[0206] The correlation between expressed intentions and exposed intentions can be done in different ways, for example via the use of automatic natural language processing or “NLP” techniques of classifiers, rule engines, calculation of semantic proximity, taking into account a database of expressions associated with the stated intention, etc. Thus, a stated intention can be determined if the words used correspond exactly to the expressed intention, but also in the case of the use of synonyms, abbreviations, words in foreign languages, etc.

[0207] The association of context elements with an expressed intention and / or the correlation between an expressed intention and an exposed intention may be carried out after prior training phases, in certain embodiments; These training phases may rely on training sentences associated with the expressed intention to allow a subsequent classification of context elements as associated with at least one expressed intention (a single context element may be classified (associated) with several expressed intentions. For example, an expressed intention “reserve a room” may be associated with different words, groups of words and / or training sentences, such as “I would like to reserve a room”, “We would need a room for the...”, “meeting” etc.The context can therefore be associated with an expressed intention based on a semantic proximity with certain words, or groups of words or phrases used for training relating to this expressed intention, rather than just the name of the expressed intention, which makes it possible to further improve detection.

[0208] The expressed intentions (and the determined intentions) may be associated with named entities recognized in the context. For example, the expressed intention “Book a room” is associated with the named entities “date” and “interval”, which form known parameters of a room reservation. Thus, the deduction (for example the extraction) of an expressed intention from the at least one word may comprise the recognition in the context of the named entities associated with the expressed intention, in this case the reservation dates and intervals mentioned in the email. Other information, which does not form named entities, may be identified and associated with the expressed intention in the form of information. For example, the information “return of the visit with the manager” has been identified and may be associated with the expressed intention.

[0209] These named entities and additional information form context elements relating to the expressed intention considered, and therefore to the at least one first determined intention correlated with the expressed intention considered. The named entities and additional information can thus be provided to the interactive computer application in addition to the identifier of the determined intention, in order to allow the interactive application to have information that is as precise as possible to configure the action and / or establish an interaction with the user.

[0210] The context of the electronic device may also include context elements of the environment of the electronic device.

[0211] For example, the context may include the device's position, the time, the lum- ambient light or the presence of the user near the device.

[0212] As will be explained in more detail below, these elements make it possible to provide context elements relating to a determined intention to the interactive computer application and / or to carry out a filtering of the determined intentions according to the environment of the device (or alternatively to filter the expressed intentions before their correlation with exposed intentions). The context elements relating to the environment make it possible, for example, to know whether it is day or night, whether the device is used in a personal or professional context, etc.

[0213] Steps S31 and / or S32 can be carried out in different ways depending on the embodiments.

[0214] For example, in certain embodiments, steps S31 and / or S32 may be performed continuously, i.e. permanently at regular intervals. For example, steps S31 and / or S32 may be performed every second, 10 seconds, minutes, etc. or at variable intervals until an intention is determined, in order to ensure that the determination of intentions occurs continuously, with configurable temporal precision. Performing steps S31 and / or S32 makes it possible to limit (in a deterministic manner) the delay between the occurrence of context elements representative of an expressed intention, and the possible provision of an exposed intention correlated with this expressed intention to the at least one computer application.

[0215] In certain embodiments, steps S31 and / or S32 may be performed upon the occurrence of an event representative of a change in the context of the electronic device.

[0216] Such an event can for example be: - detection of a new word; - a window change; - opening a message; - etc.

[0217] In other words, steps S31 and / or S32 are carried out in this case when an event representative of a change in the context occurs, which makes the occurrence of the conditions for detecting a new expressed intention more likely.

[0218] Performing steps S31 and / or S32 upon the occurrence of an event representative of a change in the context of the electronic device makes it possible to reduce the computational load of at least one electronic device computation unit while retaining the possibility of determining the intentions to be provided to the at least one computer application when necessary, since steps S31 and / or S32 are only executed when an event representative of a change in the context of the device makes the expression of a new intention in the context more likely.

[0219] The method may comprise a verification S33 of the determination of an intention exposed. Step S33 consists of verifying whether an intention exposed in semantic proximity with an expressed intention and therefore able to be provided to the interactive computer application has been determined. This verification can be carried out following a filtering S35 (see below) and can be optional in certain embodiments.

[0220] Where no intention can be validly provided to the application in interactive computing has not been determined (output of step S33 is negative for example), the method can return to step S31, and so on until an intention is determined.

[0221] When, on the contrary, an intention has been determined, and where appropriate filtered, the method P3 then comprises a provision S34 of said at least one determined intention to said interactive computer application.

[0222] Thus, the interactive computer application benefits from its launch / invocation, from prior knowledge related to the determined intention, for example at least one context element not originating from its own interface (external to the interaction history of the interactive application with the user for example) can be provided with the determined intention. The at least one first intention can thus be determined by correlation with an intention expressed for example from application context elements other than the interactive computer application. For example, the at least one first intention can be determined using text from emails, words acquired via a microphone, etc.

[0223] Providing the determined intention to the interactive application thus makes it possible to limit the (explicit) interactions between the interactive application and the user before implementing a command (calling the online service for example), since the knowledge of at least one context element not originating from its own interface makes it possible to determine the at least one first intention without interaction via the interface of the interactive computer application itself.

[0224] Providing the determined intention to the interactive application can thus help more generally to facilitate exchanges between an interactive computer application and its users, with a view to performing an action by the interactive computer application or another application.

[0225] The provision of the determined intent to the interactive application may be conditioned in certain embodiments on contextual elements of the environment of the electronic device (such as a type of current use (“work”, “leisure”, etc.). For example, in certain embodiments, the determined intents associated with a “work” type of use may only be sent during office hours (and not outside).

[0226] According to some embodiments, the interactive computer application may be an already running interactive computer application (e.g., if the interactive computer application is a chatbot, virtual assistant or voice assistant running continuously in the background) or not (for example, if the interactive computer application is a chatbot, virtual assistant or voice assistant launched and executed on demand).

[0227] In the case where the interactive computer application is an already running interactive computer application, the determined intent (and the at least one possible context element) may be provided directly to the running interactive computer application.

[0228] Otherwise, the method may comprise starting (launching) the interactive computer application prior to providing the determined intention (and the at least one possible context element).

[0229] As indicated above, at least one context element relating to the expressed intent, for example named entities, may be provided in addition to the determined exposed intent, in order to facilitate the interaction between the interactive application and the user.

[0230] For example, the interactive computer application can be invoked without user action as soon as an intention is determined, and if necessary filtered, and the determined intention can be provided to the interactive computer application accompanied by context elements relating to the determined intention, so that the first interaction of the interactive application is already targeted on the actual needs of the user (and takes into account provided context elements, without asking the user to enter them).

[0231] Taking the example of context associated with the application "Outlook.exe" above, an expressed intention "book a room" is associated with the entities named "Interval" (value "10 a.m.-3 p.m."), and "date" (value "10 / 20 / 2023"). In this example, the two entities named "Interval" (value "10 a.m.-3 p.m."), and "date" (value "10 / 20 / 2023") can be provided to the interactive computer application at the same time as the determined intention "room reservation" correlated with the expressed intention "book a room". Without the user even needing to invoke the interactive application, it will be able to send him a first personalized message depending on the context, such as: "Would you like me to book a room on November 20 from 10 a.m. to 3 p.m.? If so, for how many people?".

[0232] Such embodiments can help to facilitate the interaction between the application and the user, by limiting the number of exchanges (some exchanges can for example be avoided). For example, unlike the example of [Fig.l], the interactive application in the example of [Fig.3] does not have to send a message to request the stated intention to be pursued, nor the date nor the time slot. A simple message requesting confirmation of the information already detected can for example example may be sufficient in certain cases before performing the action associated with the determined intention.

[0233] Thus, the interactive application can emit, upon its invocation via the method of the present application (and therefore upon its initial interaction with the user), messages (questions for example) dependent on the context provided, and configure the action to be performed according to the context provided and the responses received to these messages. The emission of messages dependent on the context provided can make it possible to reduce the number of messages to be exchanged with a user to configure the action to be performed, and can therefore help to facilitate the interaction between the user and the interactive computer application.

[0234] Furthermore, the sending of the determined intention and the associated parameters can be done automatically, as soon as the interfaces of the electronic device allow information to be perceived making it possible to detect an expressed intention. For example, an intention to be sent can be determined as soon as a change in the context makes it possible to detect a new expressed intention: detection of a new word in a displayed text, of a new word exchanged in a voice conversation, reception of a new email, change of window, etc. The user therefore does not need to initialize the interaction with an interactive application such as a chatbot himself, this is done automatically.

[0235] For this purpose, an exposed intention can be provided to the interactive application as soon as it has been determined to be correlated with an expressed intention, and if necessary filtered.

[0236] In some embodiments, the method S33 may comprise a filtering S35 of the at least one first determined intention.

[0237] The at least one first determined intention is therefore provided to the interactive computer application conditionally. Providing the at least one first determined intention to the computer application conditionally makes it possible to provide the interactive computer application with only the determined intentions which satisfy at least one filtering criterion.

[0238] The filtering criteria can be of different types.

[0239] For example, the determined intentions may be filtered according to the execution context of the method. For example, if the electronic device is a professional electronic device and / or if the method is executed during office hours, only the determined intentions relating to a professional application may be retained.

[0240] Filtering can be configured, for example by a user or an administrator of the electronic device.

[0241] For example, the user or administrator can choose to filter the intentions exposed / determined in relation to classes of exposed intents (e.g., intents related to a professional goal, a personal goal, etc.), or context elements. For example, the user can choose to keep only exposed / determined intents related to professional applications. The filtering parameters can thus be modified according to the needs of the user or the administrator.

[0242] Filtering can also be performed based on the context of the electronic device.

[0243] For example, the filtering may retain only the expressed or determined intentions related to a given company, a given person, etc.

[0244] Filtering may also be performed based on environmental data of the electronic device. For example, filtering may depend on the location of the device, time, temperature and / or ambient brightness, etc.

[0245] For example, if the time and / or position of the device indicate that the device is used in a professional context (location in offices and / or office hours for example), only the intentions exposed / determined linked to a professional context may be retained.

[0246] Reference is now made to [Fig.4].

[0247] [Fig.4] shows an example of a Dial4 dialogue between a Ut4 user and an interactive computer application Applnt4 according to certain embodiments.

[0248] In this example, the following email was received:

[0249] “Hello, would it be possible to organize a visit to the JFLCORP company on the 18th? November? Also, would it be possible to reserve a room on November 20th from 10:00 to 3:00 p.m. for a restitution of the visit with the manager? Kind regards, JF.

[0250] In this example, already studied above, in a first step Et41, a context is established, and an expressed intention “reserve a room” is detected, to result in the following context (associated with the determined intention “room_reservation”) which is sent to the interactive computer application Applnt4:

[0251] [{

[0252] 'process': 'outlook.exe',

[0253] 'title' : 'actions to prioritize',

[0254] 'data' : [{

[0255] 'type' : 'knowledge,

[0256] 'metadata' :[{

[0257] 'category': 'Name Entity',

[0258] 'type' : 'compagny',

[0259] 'value':'JFLCORP',

[0260] 'knowledgeLevel': 0.57,

[0261] } ]

[0262] },{

[0263] ‘type’ : ‘intent’,

[0264] ‘value’:‘reserver salle’,

[0265] ‘metadata’ :[{

[0266] ‘category’:‘Name Entity’,

[0267] ‘type’ : ‘interval’,

[0268] ‘value’ : ‘10h- 15h’,

[0269] ‘knowledgeLevel’ : 0.57,

[0270] },{

[0271] ‘category’:‘Name Entity’,

[0272] ‘type’ : ‘date’,

[0273] ‘value’ :‘20 / 10 / 2023’

[0274] },{

[0275] ‘category’ : ‘info’,

[0276] ‘type’ : ‘subject’,

[0277] ‘value’ :‘restitution de la visite avec le gérant’

[0278] }

[0279] }],

[0280] }]

[0281] In a second step Et42, the interactive application sends a first message to validate the information contained in the context: “Would you like me to reserve a room on November 20 from 10:00 to 3:00 p.m.?” A notification is displayed with the message.

[0282] In a third step Et43, the user Ut4 clicks on the notification, which validates the message.

[0283] In a fourth step Et44, the interactive application Applnt4 opens a dialog window with the sentence “For how many people?”.

[0284] In a fifth step Et45, the user Ut4 responds “5”.

[0285] In a sixth step Et46, the interactive application Applnt4 sends the message “Let’s recap: you want to reserve a room on November 20 from 10:00 to 3:00 p.m. for 5 people. OK?”.

[0286] In a seventh step Et47, the user Ut4 writes the message “yes”.

[0287] In an eighth step Et48, the interactive application Applnt4 proceeds to the re reservation, via a room reservation API.

[0288] In a ninth step Et49, the interactive application Applnt4 writes the message “Room reserved!”.

[0289] We note that the number of interactions between the user and the application in interactive in the Dial4 dialogue is much lower than the number of interactions between the user and the interactive application in the dialogue according to the state of the art Diall. The reduction in the number of interactions in the context of the invention is notably due to the fact that the user does not need to launch the interactive application himself, nor to manually express his intention, nor to provide certain information already contained in the email (in particular date and time slot). The interaction between the user and the interactive application can therefore be facilitated. Thus, in this example, the duration of the Dial4 dialogue can for example be less than 10 seconds, whereas it could be more than 1 minute in the case of the dialogue according to the state of the art Diall.

[0290] The present disclosure is not limited to the examples of method, computer program product and electronic device described above, only by way of example, but it encompasses all the variants that may be envisaged by those skilled in the art within the framework of the protection sought.

Claims

Claims

1. Method (P3) implemented by an electronic device for processing data by at least one interactive computer application, comprising: - obtaining (S31) a context of the electronic device; - determining (S32) at least one first intention of at least one interactive computer application from at least one element of the context of the electronic device not linked to a human-machine interface of the at least one interactive computer application, said at least one first intention being determined from at least one intention exposed by the at least one interactive computer application, said at least one exposed intention being associated with at least one action to be performed via the at least one interactive computer application; - providing (S34) said at least one first determined intention to the at least one interactive computer application.

2. Method according to claim 1, comprising: - providing at least one element of the context, relating to the at least one first determined intention, to the at least one interactive computer application.

3. Method according to any one of the preceding claims, wherein said at least one first intention is determined by correlation between at least one so-called expressed intention obtained from said context, with said at least one intention exposed by the at least one interactive computer application.

4. Method according to the preceding claim, in which: - said context comprises at least one word rendered or acquired via a human-machine interface of the electronic device; - said correlation takes into account a semantic proximity between said at least one expressed intention and said at least one intention exposed by the at least one interactive computer application; - obtaining the context comprises detecting said at least one word rendered or acquired via said human-machine interface of the electronic device.

5. Method according to the preceding claim, comprising recognizing, among the at least one word, at least one named entity associated with said at least one expressed intention.

6. Method according to any one of the preceding claims, comprising: - carrying out a filtering of the at least one first determined intention; - providing said at least one first determined intention at the end of the filtering.

7. A method according to claim 6, wherein the filtering is performed based on the context of the electronic device.

8. Method according to any one of the preceding claims, in which at least one of the steps of obtaining a context of the electronic device, and determining the at least one first intention is carried out continuously.

9. Method according to any one of the preceding claims, in which at least one of the steps of obtaining a context of the electronic device, and determining the at least one first intention is carried out upon the occurrence of an event representative of a change in the context of the electronic device.

Images