Order conversion process

Abstract

A method for converting a command, along with a corresponding computer program, storage medium, and interpreter, is proposed. The command is expressed, at least in part, in natural language and concerns the state of a piece of equipment, into a logical instruction. The proposed technique involves obtaining the logical instruction through an interpreter of said command. This interpreter uses a language model with a query that includes the command and a set of interpretation rules comprising only first-order interpretation rules. The resulting logical instruction conforms to what would be produced by interpreting the command using a second interpretation rule derived from the set of first-order rules. This second interpretation rule is distinct from the first interpretation rules. (See abstract figure: Figure 1)

Description

Title of the invention: Method for converting a command Technical field

[0001] The present disclosure relates to a method for converting a command and to a corresponding computer program, recording medium and interpreter. Prior art

[0002] Currently, to control electronic equipment, such as lamps, roller shutters, a heating system, robots, computers, etc. (there is a very wide variety of these), using natural language commands, whether oral or written, it is necessary to have a natural language interpretation mechanism, called NLU (Natural Language Understanding), built specifically to take into account all the commands that can be formulated. However, such a system generally poses at least two problems. On the one hand, it is tedious to build, because it requires anticipating, for each command, all possible formulations. On the other hand, even with a good level of anticipation, such a system generally proves insufficient, because many variants of a formulation will have been forgotten.Furthermore, the larger the number of NLU rules, to accommodate a wide variety of formulations, the more complex the system becomes to maintain. Summary

[0003] The present disclosure improves the situation.

[0004] According to one aspect, there is provided a method for converting a command, expressed at least in part in natural language and concerning a state of equipment, into a logical instruction, the method comprising obtaining the logical instruction via an interpreter of said command requesting a language model with a query comprising the command and a set of interpretation rules comprising only first interpretation rules, the logical instruction being consistent with what would be produced by an interpretation of the command by application of a second interpretation rule derivable from the set of first rules, the second rule of interpretation being distinct from the first rules of interpretation.

[0005] According to another aspect, there is provided a computer program comprising instructions which, when the program is implemented by a processor, lead to implement said process.

[0006] According to another aspect, there is provided a non-transitory recording medium readable by a computer on which said computer program is recorded.

[0007] According to another aspect, there is provided an interpreter of a command, expressed at least in part in natural language and concerning a state of a piece of equipment, into a logical instruction, the interpreter being configured to obtain the logical instruction by requesting a language model with a query comprising the command and a set of interpretation rules comprising only first interpretation rules, the logical instruction being consistent with what would be produced by an interpretation of the command by application of a second interpretation rule derivable from the set of first rules, the second interpretation rule being distinct from the first interpretation rules.

[0008] The present disclosure makes it possible, in each of its aspects, to optimize communication between users and any equipment controlled by computer means. It facilitates interaction in natural language while offering the possibility of precise and reliable execution of commands by means of a transformation into logical instruction.

[0009] In one example, the logic instruction is adapted to cause a change in state of the equipment from a current state to a desired state when the logic instruction is executed by a logic reasoning engine.

[0010] This has the advantage of making interactions with automated systems more accessible and intuitive for users, while maintaining high accuracy in the execution of commands. It also reduces the risk of errors due to misunderstandings or incorrect interpretations of natural language, thus improving the reliability of the system.

[0011] In one example, the logic instruction is adapted to cause a transmission of a signal specifying the current state when the logic instruction is executed by a logic reasoning engine.

[0012] It is thus possible, for example, to provide immediate feedback to the user on the current state of the equipment, which can be crucial in situations where knowledge of the current state of the equipment is necessary to make informed decisions or to ensure safety.

[0013] In one example, the set of first rules contains a rule relating to a designation of the equipment and a rule relating to a designation of the desired state.

[0014] This has the advantage of facilitating system configuration and customization, as users can specifically designate the desired equipment and state by using their own language. This makes the proposed technique more flexible and easier to use, even for those who are not familiar with the technology.

[0015] In one example, the request further includes descriptive data of the current state of the equipment.

[0016] Taking into account the current state of the equipment when interpreting commands strengthens the robustness and reliability of the proposed technique. This ensures that the actions taken are always relevant and appropriate to the current situation.

[0017] In one example, the method further comprises or the system further provides for transmitting the logic instruction to a logic reasoning engine for execution of the logic instruction.

[0018] The logical reasoning engine participates in the execution of commands and contributes to a smooth interaction between the user and the system responsible for processing the commands. Brief description of the drawings

[0019] Other characteristics, details and advantages will appear on reading the detailed description below, and on analyzing the attached drawings, in which: Fig.l

[0020] [Fig.l] represents a system for processing an order in an exemplary embodiment. Fig. 2

[0021] [Fig.2] represents a processing circuit adapted to implement a method of converting a command, expressed at least in part in natural language, into a logical instruction. Fig. 3

[0022] [Fig.3] represents, in the form of a flowchart, an algorithm allowing the processing circuit of [Fig.2] to implement the aforementioned conversion method. Description of the embodiments

[0023] In the following description, identical reference numerals designate identical parts or parts having similar functions.

[0024] [Fig.l] represents the main elements of a system (100) suitable for processing a command expressed at least partly in natural language and for managing a set of equipment.

[0025] The elements of the system are implemented by computer means, for example by one or more processing circuits (200) such as that shown in [Fig.2], which comprises a processor (202) having a data processing function, connected to a memory (204) having a data storage function, and to a communication interface (206) having a function of receiving and / or transmitting data.

[0026] The system may have a home automation function, the managed equipment being installed in a home and including, for example, lamps, a heating system, shutters, a security system, multimedia equipment, etc.

[0027] Alternatively, the system may be intended for the management of industrial equipment, such as robots installed on one or more sites. Its functions may be to control such machines or to obtain information on their status, particularly for the purposes of safety and efficiency.

[0028] Another possible area of ​​application is customer service, with the aim of enabling agents to provide more precise and relevant responses in real time to requests from their customers.

[0029] The system can also find an application in the fields of medical assistance and / or accessibility, allowing people with reduced mobility to control different medical or home assistance devices, in order to improve their quality of life.

[0030] A human-machine interface (102) allows the system to receive a command spoken in natural language by a user, for example a resident in the case of a home automation system or an operator in the case of an industrial equipment management system. The interaction can be provided in any format appropriate in the case at hand, in particular text or voice. The command represents an underlying intention of the user.

[0031] The user's intention may concern a control of managed equipment, in particular a change of state of the managed equipment to a desired state. For example, in the case of a home automation system, it may involve turning on or off the living room lamp.

[0032] Alternatively, the user's intention may relate to a request for information, which may or may not be related to a state of a managed device. For example, the user's intention, as a resident, may be to know whether, at a current time, the living room lamp is on. Alternatively, the user's intention may be to obtain an answer to a general knowledge question unrelated to the set of managed devices.

[0033] A command interpreter (104) is provided to enable the command thus received to be interpreted. For this, the interpreter is capable of requesting one or more modules of the system, in particular a language model (106), LLM in English for “Large Language Model”, a database (108) and / or a natural language generator (110) and / or a natural language processing processor or “NLU processor” (not shown). The result of the interpretation of the command by the command interpreter (104) is a logical formula representing the user's underlying intention.

[0034] A language model (106) is a service whose use is now democratized. Its simplified operating principle is based on the generation of a sequence of output words, carried out by a successive word-by-word prediction. This prediction is based on a sequence of words provided as input. Modern language models, in particular those based on transformer-type architectures, integrate many other elements into their operation, such as attention, lexical embeddings and layers of neurons.

[0035] The database (108) contains descriptive information relating to the equipment managed by the system. Taking the example of the living room lamp in a home automation system, the database can be designed to store descriptive information indicating the current state of the lamp, i.e. whether it is for example on or off, and in the case of a lamp having adjustable parameters such as intensity or color, the current value of these parameters. A suitable mechanism can be provided for updating the descriptive information in the database each time the state of the corresponding equipment changes.

[0036] The database (108) also contains predetermined natural language understanding rules, or NLU rules, and the natural language generator (110) is configured to access and apply these NLU rules to interpret received commands.

[0037] It may be provided that the command interpreter first requests the natural language generator or the NLU processor to attempt to interpret the command and, only in the event of failure, then requests the language model. This sequential approach makes it possible to take advantage of already provided NLU rules, in particular NLG (Natural Language Generation) rules which explain how to go from a logical formula to a natural language sentence, and may have advantages in terms of speed and resource savings. Alternatively, it may be provided that the command interpreter always requests both the natural language generator or the NLU processor to obtain a first interpretation result and the language model to obtain a second interpretation result. This parallel approach makes it possible to implement cross-validation which reinforces the robustness of the interpretation.It also allows the implementation, by appropriate artificial intelligence means, of an algorithm for the continuous improvement of the NLU rules applicable by the natural language generator. The choice between one of these two approaches may depend on the specific needs of the system, the types of commands expected and the available resources.

[0038] The command interpreter can be configured to access all or part of the database content, in particular descriptive data and NLU rules or NLG rules, with a view to passing all or part of this content to the language model. Filtering or masking mechanisms may be provided for various reasons. One reason may be to prevent, for example, sensitive information contained in the database from being passed to the language model. Another reason may be to avoid passing to the language model information deemed useless for interpreting the command.

[0039] Examples of exchanges between the command interpreter and the aforementioned modules are detailed later in this document through [Fig.3] and several specific examples.

[0040] The system is also configured to implement a logical reasoning engine (112) and an action factory (114) to ultimately execute actions as soon as they are deemed relevant.

[0041] In particular, the command interpreter may be configured to transmit the logical formula to the logical reasoning engine (112), which may either validate the intention and transmit an instruction to the action factory (114) in order to implement the corresponding action, or not validate the action and, consequently, not transmit said instruction to the action factory.

[0042] The logical reasoning engine may, for example, be implemented in the form of rules specifying one or more logical formulas as antecedents and one or more logical formulas as conclusions. The logical formulas specified as antecedents may relate to one or more descriptive data from the database (108). Associated rules and logical formulas are detailed later in this document through several specific examples.

[0043] The logical reasoning engine (112) can also implement an action planning engine (116), for example of the PDDL type for “Planning Domain Definition Language” in English.

[0044] The actions from the action factory and concerning a given piece of equipment are transmitted to a command interface (118) responsible for controlling the piece of equipment concerned.

[0045] The system may be configured to transmit feedback to the user after processing of the command by the system. The feedback may be a simple acknowledgment following receipt of the command by the human-machine interface (102). The feedback may also contain an indication relating to the user's intention and following interpretation of the command by the interpreter (104). For example, when the user's underlying intention relates to a request for information, the command interpreter may be configured to provide a response to this request for information. For example, when the underlying intention If the user's action concerns a change in the state of a managed device, the logic reasoning engine can be configured to send a message to the user to indicate whether an action corresponding to this change of state is validated or not.

[0046] Reference is now made to [Fig.3] which describes in detail the process followed by the command interpreter when it receives a command expressed in natural language and interacts with the language model (LLM) to generate a logical instruction intended for the logical reasoning engine.

[0047] When a command is received by the human-machine interface, it is transmitted (302) to the command interpreter. As already indicated, the command can be of various natures, ranging from a request for action on managed equipment to a general question.

[0048] The interpreter begins by obtaining (304) a set of interpretation rules. These rules may be stored in the database and include both predefined NLU rules and potentially rules generated by previous interaction with the language model.

[0049] In parallel, the interpreter can retrieve (306) the descriptive data concerning the equipment mentioned in the command from the database. For example, if the command concerns the state of a lamp in the living room, the interpreter can retrieve the descriptive data(s) of the current state of this lamp from the database.

[0050] The interpreter transforms the received command into a structured request to the language model. This structured request, also called a "prompt" in English, is composed of a series of instructions and information that help the language model understand and process the command.

[0051] The structured query includes the user's command as well as a specific set of one or more interpretation rules, which we will call here the "first interpretation rules". These first rules are directly provided to the language model to guide its understanding and interpretation of the command. The query may also include one or more descriptive data, deemed useful to help the language model understand the context, but this is not strictly necessary.

[0052] In one embodiment, the language model may process multiple successive requests, thereby creating a discussion thread with the user. In this case, it may be sufficient to transmit only the command as a request to obtain a satisfactory response, provided that a complete structured request has been transmitted previously in the same discussion thread.

[0053] The crucial interaction with the LLM occurs when the interpreter requests it (308) by submitting the query, thus exploiting its power to generate a logical representation of the user's intention. The language model responds by generating a logical formula that represents this intention.

[0054] If an interpretation rule present in the query corresponds directly to the command, the logical formula generated is identical to the result obtained by directly applying this rule to the command.

[0055] However, it is possible that the command requires the application of a specific interpretation rule, different from the first rules provided, which we will call "second interpretation rule". It is not necessary for this second interpretation rule to be explicitly present in the set of rules initially provided to the language model. It is sufficient that it can be logically derived or inferred from these first interpretation rules, for the language model to have the ability to correctly understand the command and generate an appropriate response. In other words, even if the exact rule necessary to interpret the command is not directly provided in the request, the language model can provide a result corresponding to the application of this rule from the information it has received, thus ensuring a correct and accurate interpretation of the user's intention.

[0056] A logical formula is thus obtained (310) and transmitted (312) to the logical reasoning engine. If the intention concerns an action, the engine can check whether this action is possible or relevant given the current state of the equipment, as described in the database. If the intention concerns a request for information, the engine can directly respond by consulting the database and, for example, inform the user that the lamp is already on.

[0057] The process thus described therefore makes it possible to convert a command expressed in natural language into a clear and precise logical instruction, facilitating decision-making and action by the system.

[0058] Several examples of processing particular commands are now described. In each of these examples, the conversion process presented above is implemented to obtain a logical formula, which is then processed by the logical reasoning engine,

[0059] In a first example, the user requests to turn on the living room lamp.

[0060] A simplified example of a query to the language model, presented arbitrarily as a series of JSON objects, is as follows: { role: System, content: You are an NLU system capable of understanding natural language Instructions: ... Awareness : ... Actions: - to turn on or off a device uses the intent (is-switched?device ?State) Entities: - to designate the living room lamp use the symbol living-room-light { role: user, content: color rectangle 1 in blue}, { role: assistant, content: (I ?user (is-colored rectangle-1 blue))}, { role: user, content: what is the current temperature in the room}, { role: assistant, content: (I ?user (B ?user (current-temperature bedroom ?temperature))}, • • • 9 { role: user, content: turns on the living room lamp}

[0061] The dotted lines refer to information that may be contained in the query but is not repeated here because it is not relevant for the purposes of the explanation.

[0062] Specifically, descriptive data may be provided in the section titled "Knowledge." A set of initial interpretation rules is also provided in the sections titled "Actions" and "Entities," and the command is provided as its own JSON object at the end of the query.

[0063] In this example, the set of first interpretation rules is conceptually divided into two parts, one relating to the designation of specific states of any equipment, and the other relating to the designation of specific equipment.

[0064] In this example, the request contains other information, including a header, possibly specific instructions, as well as several previous commands, one relating to a request for action, the other relating to a request for information, and the interpretation result associated with each. This various additional information can supplement or replace certain interpretation rules.

[0065] Based on the provided query, the language model provides an interpretation result of the command to turn on the living room lamp. The expected result is the following logical formula: (I ?user (is-switched living-room-light on))

[0066] This logical formula represents the user's intention and is of the form "(I ?user ?phi)"; it indicates that the user ?user intends the formula ?phi to be true. More specifically, it indicates that the user ?user intends the lamp designated by "living-room-lamp" to be turned on.

[0067] The logical reasoning engine is configured to receive and process the logical formula by applying a limited number of generic predetermined rules, which include each take one or more logical formulas as input and one or more logical formulas as output.

[0068] The predetermined rules are generic in that the logic formulas contained therein do not require any reference to any particular equipment or state, which helps to make them compatible with the processing of a large number of specific logic formulas that may be generated by the language model and provided as input.

[0069] Two predetermined rules are provided here as examples.

[0070] The first predetermined rule provides for signaling to the user, following receipt of an instruction to turn on equipment, a situation where the instruction cannot be applied because the equipment concerned is already turned on.

[0071] The first predetermined rule is as follows: (I ?user (is-switched ?dev on)) & (is-switched ?dev on) -> (I self (B ?user (is-switched ?dev on))

[0072] It contains two logical formulas as antecedents, the first in the form "(I ?user ?phi)" and obtained at the output of the command interpreter and the second in the form "?phi" and corresponding to a descriptive data stored in the database.

[0073] It also contains an output logical formula in the form "(I self (B ?user ?phi))" expressing the intention of the system (self) to communicate a specific descriptive data corresponding to "?phi" to the user.

[0074] The second predetermined rule provides for turning on the equipment if it is turned off.

[0075] The second predetermined rule is as follows: (I ?user (is-switched ?dev on) & (is-switched ?dev off) -> (I self (is-switched ?dev on)

[0076] Just like the first predetermined rule, it contains two logical formulas as antecedents, the first in the form "(I ?user ?phi)" and obtained as output from the command interpreter and the second in the form "?phi" and corresponding to a descriptive data stored in the database. It also contains a logical formula as output in the form "(I self ?phi)", expressing the intention of the system (self) to carry out the action ?phi. For example, if there is an action to satisfy the intention ("is switched device on?"), this action (which in this case turns on the device) will be automatically executed.

[0077] The two predetermined rules thus described enable the logic reasoning engine to adequately process an instruction to turn on the living room lamp. If the living room lamp is already on, the reasoning engine The logic engine knows this by looking at the available descriptive data. It can then decide to send a message to the user to tell them that the lamp in question is already on. If the living room lamp is initially off, then the logic engine can proceed to issue an instruction to turn it on according to the user's intention.

[0078] In a second example, the user asks if the living room lamp is on.

[0079] The corresponding query can be generated by the command interpreter so as to contain the same information as that contained in the query indicated in the first example, with the exception of the command, which is formulated as: { role: user, content: is the living room lamp on}.

[0080] Based on the provided query, the language model provides an interpretation result of the command to indicate whether the living room lamp is on. The expected result is: “(I?user (B?user (is-switched living-room-lamp on))”.

[0081] This is a logical formula of the form "(I ?user (B ?user ?phi))", which means that the user ?user intends to know a specific descriptive data corresponding to "?phi".

[0082] A logical formula with the same structure "(I ?user (B ?user (current-temperature bedroom ?temperature))" is provided in the query: it transcribes an intention of the user to know the temperature in a room.

[0083] In the second example, the expected logical formula "(I ?user (B ?user (is-switched living-room-lamp on))" means that the user intends to know if the living-room lamp is on.

[0084] The reasoning engine can be configured to process this logical formula by applying a third predetermined rule providing that if the user wants to know a specific descriptive data corresponding to "?phi" then the system intends to communicate this descriptive data to him.

[0085] The third predetermined rule is as follows: (I ?user (B ?user ?phi)) -> (I self (B ?user ?phi))

[0086] In a third example, the user requests to put the living room of his home in cinema mode.

[0087] It is assumed that such a mode involves several devices simultaneously. For example, the cinema mode may correspond to a situation where the shutters are closed and the lamps are adjusted so as to obtain low brightness.

[0088] To the extent that several actions must potentially be implemented, it may be planned to calculate a corresponding action plan using a planner, for example of the PDDL type.

[0089] A possible result of interpretation of the command by the language model is: (I ?user (movie-mode living-room on))

[0090] To achieve such a result, one possibility is that the set of predetermined rules provided in the query contains one or more predetermined rules specifying to translate the reference to the cinema mode in the command by the expression "movie-mode" in the logical formula, as well as to translate the reference to the living room in the command by the expression "living-room" in the logical formula.

[0091] Even if such specific rules are absent, a sufficient number of sufficiently varied predetermined rules can enable the language model to provide this result.

[0092] The reasoning engine may be configured to process this logic formula by applying a fourth predetermined rule providing that if the user intends to achieve a goal "?phi" and if there is a plan capable of satisfying the goal "?phi" then the system intends to execute this plan.

[0093] The fourth predetermined rule is as follows: (I ?user ?phi) & (exists ?plan ?phi)) -> (I self (done ?plan))

[0094] The proposed technique offers multiple technical advantages compared to known systems and methods based exclusively on the application of predetermined NLU rules.

[0095] In particular, the proposed technique allows the user to use natural language more freely to express his commands.

[0096] The proposed technique makes it possible to simplify the development of equipment control systems by avoiding the need to provide a large number of NLU rules.

[0097] The proposed technique also makes it possible to maintain control over the commands to be implemented via the logical reasoning engine, since the latter can rely on data stored in the database to authorize or not the execution of an action. Thus, for example, if the database has descriptive data relating to the user's age, he can decide not to perform the action corresponding to his intention to turn on the oven. Another advantage of the proposed technique is that it does not require communicating to the language model the content, in terms of code, of the actions that can be performed. This advantage and that expressed previously offer solid security guarantees.

[0098] Another advantage of the proposed technique is that it allows, thanks to the logical reasoning engine, by combining the commands expressed in natural language by the user with other types of interactions (tactile, haptic, etc.), to implement multimodal interactions. Thus, for example, if the user says "turn on this lamp" and if the system also has knowledge about the object designated by the user, then it can send this knowledge to the language model, which then produces a logical formula faithfully reflecting the corresponding intention.

[0099] Another advantage of the proposed technique is the possibility of implementing via the reasoning engine a planning engine in order to execute not an action but a plan of actions to satisfy the user's intention.

Claims

Claims

1. Method for converting a command, expressed at least in part in natural language and concerning a state of equipment, into a logical instruction, the method comprising obtaining (310) the logical instruction via an interpreter (104) of said command requesting (308) a language model (106) with a query comprising the command and a set of interpretation rules comprising only first interpretation rules, the logical instruction being in accordance with what would be produced by an interpretation of the command by application of a second interpretation rule derivable from the set of first rules, the second interpretation rule being distinct from the first interpretation rules.

2. The method of claim 1, wherein the logic instruction is adapted to cause a change in state of the equipment from a current state to a desired state when the logic instruction is executed by a logic reasoning engine (112).

3. The method of claim 1 or 2, wherein the logic instruction is adapted to cause transmission of a signal specifying the current state when the logic instruction is executed by a logic reasoning engine (112).

4. Method according to one of claims 1 to 3, in which the set of first rules contains a rule relating to a designation of the equipment and a rule relating to a designation of the desired state.

5. Method according to one of claims 1 to 4, in which the request further comprises data descriptive of the current state of the equipment.

6. The method of one of claims 1 to 5, further comprising transmitting (312) the logic instruction to a logic reasoning engine (112) for execution of the logic instruction.

7. A computer program comprising instructions which, when the program is implemented by a processor (202), lead to implementing the method according to one of claims 1 to 6.

8. A non-transitory computer-readable recording medium (204) having recorded thereon the computer program of claim 7.

9. Interpreter (104) of a command, expressed at least in part in natural language and concerning a state of a piece of equipment, into a logical instruction, the interpreter being configured to obtain (310) the logical instruction by requesting (308) a language model (106) with a query comprising the command and a set of interpretation rules comprising only first interpretation rules, the logical instruction being consistent with what would be produced by an interpretation of the command by application of a second interpretation rule derivable from the set of first rules, the second interpretation rule being distinct from the first interpretation rules.