Method for converting a command

Abstract

The invention proposes a method for converting a command, and also a corresponding computer program, recording medium and interpreter. The command is expressed at least partially in natural language and concerning a state of a piece of equipment, into a logic instruction. The proposed technique comprises obtaining the logic instruction via an interpreter for interpreting the command, invoking a language model with a request comprising the command, a set of interpretation rules comprising only first interpretation rules, and a datum describing the current state of the piece of equipment, the logic instruction being in accordance with what would be produced by interpreting the command by applying a second interpretation rule able to be derived from the set of first rules, the second interpretation rule being distinct from the first interpretation rules.

Description

Order conversion process

[0001] The present disclosure relates to a method for converting a command and to a corresponding computer program, recording medium and interpreter.

[0002] Currently, to control electronic equipment such as lamps, roller shutters, heating systems, robots, computers, etc. (there are a wide variety of these) using natural language commands, whether oral or written, it is necessary to have a natural language interpretation mechanism, called NLU (Natural Language Understanding), specifically designed to take into account all the commands that can be formulated. However, such a system generally poses at least two problems. On the one hand, it is tedious to build, because it requires anticipating all possible formulations for each command. On the other hand, even with a good level of anticipation, such a system generally proves insufficient, because many variants of a formulation will have been forgotten.Furthermore, the larger the number of NLU rules, to accommodate a wide variety of formulations, the more complex the system becomes to maintain. Summary

[0003] This disclosure improves the situation.

[0004] According to one aspect, there is provided a method for converting a command, expressed at least in part in natural language and relating to a state of a piece of equipment, into a logical instruction, the method comprising obtaining the logical instruction via an interpreter of said command requesting a language model with a query comprising the command, a set of interpretation rules comprising only first interpretation rules, and a data item descriptive of the current state of the equipment, the logical instruction being consistent with what would be produced by an interpretation of the command by application of a second interpretation rule derivable from the set of first rules, the second interpretation rule being distinct from the first interpretation rules.

[0005] According to another aspect, there is provided a computer program comprising instructions which, when the program is implemented by a processor, result in implementing said method.

[0006] According to another aspect, there is provided a non-transitory computer-readable recording medium on which said computer program is recorded.

[0007] According to another aspect, there is provided an interpreter of a command, expressed at least in part in natural language and concerning a state of a piece of equipment, into a logical instruction, the interpreter being configured to obtain the logical instruction by requesting a language model with a query comprising the command, a set of interpretation rules comprising only first interpretation rules, and a descriptive data item of the current state of the equipment, the logical instruction being in accordance with what would be produced by an interpretation of the command by application of a second interpretation rule derivable from the set of first rules, the second interpretation rule being distinct from the first interpretation rules.

[0008] This disclosure makes it possible, in each of its aspects, to optimize communication between users and any equipment controlled by computer means. It facilitates interaction in natural language while offering the possibility of precise and reliable execution of commands through transformation into logical instructions.

[0009] Taking the current state of the equipment into account when interpreting commands strengthens the robustness and reliability of the proposed technique. This ensures that the actions taken are always relevant and appropriate to the current situation.

[0010] In one example, the logic instruction is adapted to cause a change in state of the equipment from a current state to a desired state when the logic instruction is executed by a logic reasoning engine.

[0011] This has the advantage of making interactions with automated systems more accessible and intuitive for users, while maintaining high accuracy in command execution. It also reduces the risk of errors due to misunderstandings or incorrect interpretations of natural language, thus improving the reliability of the system.

[0012] In one example, the logic instruction is adapted to cause a transmission of a signal specifying the current state when the logic instruction is executed by a logic reasoning engine.

[0013] This makes it possible, for example, to provide immediate feedback to the user on the current status of the equipment, which can be crucial in situations where knowledge of the current status of the equipment is necessary to make informed decisions or to ensure safety.

[0014] In one example, the first rule set contains a rule for a designation of the equipment and a rule for a designation of the desired state.

[0015] This has the advantage of facilitating system configuration and customization, as users can specifically designate the desired equipment and state using their own language. This makes the proposed technique more flexible and easier to use, even for those unfamiliar with the technology.

[0016] In one example, the method further comprises or the system further provides for transmitting the logic instruction to a logic reasoning engine for execution of the logic instruction.

[0017] The logical reasoning engine participates in the execution of commands and contributes to a smooth interaction between the user and the system responsible for processing the commands.

[0018] Other features, details and advantages will become apparent upon reading the detailed description below, and upon analyzing the attached drawings, in which: Fig. 1

[0019] represents a system for processing an order in an exemplary embodiment. Fig. 2

[0020] represents a processing circuit adapted to implement a method of converting a command, expressed at least in part in natural language, into a logical instruction. Fig. 3

[0021] represents, in the form of a flowchart, an algorithm allowing the processing circuit to implement the aforementioned conversion method.

[0022] In the following description, like reference numerals designate like parts or parts having similar functions.

[0023] represents the main elements of a system (100) suitable for processing a command expressed at least partly in natural language and for managing a set of equipment.

[0024] The elements of the system are implemented by computer means, for example by one or more processing circuits (200) such as that shown in, which comprises a processor (202) having a data processing function, connected to a memory (204) having a data storage function, and to a communication interface (206) having a data reception and / or transmission function.

[0025] The system can have a home automation function, the managed equipment being installed in a home and including for example lamps, a heating system, shutters, a security system, multimedia equipment, etc.

[0026] Alternatively, the system may be intended for the management of industrial equipment, such as robots installed at one or more sites. Its functions may be to control such machines or to obtain information on their status, particularly for the purposes of safety and efficiency.

[0027] Another possible area of ​​application is customer service, with the aim of enabling agents to provide more precise and relevant answers in real time to their customers' requests.

[0028] The system can also find application in the fields of medical assistance and / or accessibility, allowing people with reduced mobility to control different medical or home assistance devices, in order to improve their quality of life.

[0029] A human-machine interface (102) allows the system to receive a command spoken in natural language by a user, for example a resident in the case of a home automation system or an operator in the case of an industrial equipment management system. The interaction can be provided in any format appropriate in the case at hand, in particular text or voice. The command represents an underlying intention of the user.

[0030] The user's intention may concern the control of a managed device, in particular a change in the state of the managed device to a desired state. For example, in the case of a home automation system, this may involve turning on or off the living room lamp.

[0031] Alternatively, the user's intention may relate to a request for information, which may or may not be related to the status of a managed device. For example, the user's intention, as a resident, may be to know whether, at a current time, the living room lamp is on. Alternatively, the user's intention may be to obtain an answer to a general knowledge question unrelated to the set of managed devices.

[0032] A command interpreter (104) is provided to enable the command thus received to be interpreted. For this purpose, the interpreter is likely to request one or more modules of the system, in particular a language model (106), LLM in English for “Large Language Model”, a database (108) and / or a natural language generator (110) and / or a natural language processing processor or “NLU processor” (not shown). The result of the interpretation of the command by the command interpreter (104) is a logical formula representing the underlying intention of the user.

[0033] A language model (106) is a service whose use is now democratized. Its simplified operating principle is based on the generation of an output sequence of words, carried out by successive word-by-word prediction. This prediction is based on a sequence of words provided as input. Modern language models, particularly those based on transformer-type architectures, integrate many other elements into their operation, such as attention, lexical embeddings and neural layers.

[0034] The database (108) contains descriptive information relating to the equipment managed by the system. Taking the example of the living room lamp in a home automation system, the database may be designed to store descriptive information indicating the current state of the lamp, i.e. whether it is, for example, on or off, and in the case of a lamp having adjustable parameters such as intensity or color, the current value of these parameters. A suitable mechanism may be provided for updating the descriptive information in the database each time the state of the corresponding equipment changes.

[0035] The database (108) also contains predetermined natural language understanding rules, or NLU rules, and the natural language generator (110) is configured to access and apply these NLU rules to interpret received commands.

[0036] The command interpreter can be expected to first call the natural language generator or NLU processor to attempt to interpret the command and, only if this fails, then call the language model. This sequential approach makes it possible to take advantage of already provided NLU rules, in particular NLG (Natural Language Generation) rules that explain how to go from a logical formula to a natural language sentence, and can have advantages in terms of speed and resource savings. Alternatively, the command interpreter can always call both the natural language generator or NLU processor to obtain a first interpretation result and the language model to obtain a second interpretation result. This parallel approach allows for cross-validation, which increases the robustness of the interpretation.It also allows the implementation, by appropriate artificial intelligence means, of an algorithm for the continuous improvement of the NLU rules applicable by the natural language generator. The choice between one of these two approaches may depend on the specific needs of the system, the types of commands expected and the available resources.

[0037] The command interpreter can be configured to access all or part of the database content, in particular descriptive data and NLU rules or NLG rules, with a view to passing all or part of this content to the language model. Filtering or masking mechanisms can be provided for various reasons. One reason may be to prevent, for example, sensitive information contained in the database from being passed to the language model. Another reason may be to avoid passing information to the language model that is deemed unnecessary for interpreting the command.

[0038] Examples of exchanges between the command interpreter and the aforementioned modules are detailed later in this document through several specific examples.

[0039] The system is also configured to implement a logical reasoning engine (112) and an action factory (114) to ultimately execute actions when they are deemed relevant.

[0040] In particular, the command interpreter may be configured to transmit the logic formula to the logic reasoning engine (112), which may either validate the intent and transmit an instruction to the action factory (114) to implement the corresponding action, or not validate the action and, consequently, not transmit said instruction to the action factory.

[0041] The logical reasoning engine may, for example, be implemented in the form of rules specifying one or more logical formulas as antecedents and one or more logical formulas as conclusions. The logical formulas specified as antecedents may relate to one or more descriptive data from the database (108). Rules and associated logical formulas are detailed later in this document through several specific examples.

[0042] The logical reasoning engine (112) can also implement an action planning engine (116), for example of the PDDL type for “Planning Domain Definition Language” in English.

[0043] Actions from the action factory and concerning a given piece of equipment are transmitted to a command interface (118) responsible for controlling the piece of equipment concerned.

[0044] The system may be configured to transmit feedback to the user after processing of the command by the system. The feedback may be a simple acknowledgment following receipt of the command by the human-machine interface (102). The feedback may also contain an indication relating to the user's intention and following interpretation of the command by the interpreter (104). For example, when the user's underlying intention concerns a request for information, the command interpreter may be configured to provide a response to this request for information. For example, when the user's underlying intention concerns a change in state of a managed device, the logical reasoning engine may be configured to send a message to the user to indicate whether an action corresponding to this change in state is validated or not.

[0045] It is now referred to which describes in detail the process followed by the command interpreter when it receives a command expressed in natural language and interacts with the language model (LLM) to generate a logical instruction intended for the logic reasoning engine.

[0046] When a command is received by the human-machine interface, it is transmitted (302) to the command interpreter. As already indicated, the command can be of various natures, ranging from a request for action on managed equipment to a general question.

[0047] The interpreter begins by obtaining (304) a set of interpretation rules. These rules may be stored in the database and include both predefined NLU rules and potentially rules generated by previous interaction with the language model.

[0048] In parallel, the interpreter can retrieve (306) the descriptive data concerning the equipment mentioned in the command from the database. For example, if the command concerns the state of a lamp in the living room, the interpreter can retrieve the descriptive data(s) of the current state of this lamp from the database.

[0049] The interpreter transforms the received command into a structured request to the language model. This structured request, also called a "prompt," is composed of a series of instructions and information that help the language model understand and process the command.

[0050] The structured query includes the user's command and a specific set of one or more interpretation rules, which we will refer to here as the "first interpretation rules." These first rules are provided directly to the language model to guide its understanding and interpretation of the command. The query also includes one or more descriptive data, deemed useful to help the language model understand the context, but this is not strictly necessary.

[0051] In one embodiment, the language model may process multiple successive requests, thereby creating a thread with the user. In this case, it may be sufficient to transmit only the command as a request to obtain a satisfactory response, provided that a complete structured request has been transmitted previously in the same thread.

[0052] The crucial interaction with the LLM occurs when the interpreter requests it (308) by submitting the query, thus exploiting its power to generate a logical representation of the user's intention. The language model responds by generating a logical formula that represents this intention.

[0053] If an interpretation rule present in the query directly matches the command, the generated logical formula is identical to the result obtained by directly applying this rule to the command.

[0054] However, it is possible that the command requires the application of a specific interpretation rule, different from the first rules provided, which we will call the "second interpretation rule." This second interpretation rule does not need to be explicitly present in the set of rules initially provided to the language model. It is sufficient that it can be logically derived or inferred from these first interpretation rules, for the language model to have the ability to correctly understand the command and generate an appropriate response. In other words, even if the exact rule needed to interpret the command is not directly provided in the request, the language model can provide a result corresponding to the application of this rule from the information it has received, thus ensuring a correct and accurate interpretation of the user's intention.

[0055] A logical formula is thus obtained (310) and transmitted (312) to the logical reasoning engine. If the intention concerns an action, the engine can check whether this action is possible or relevant given the current state of the equipment, as described in the database. If the intention concerns a request for information, the engine can directly respond by consulting the database and, for example, inform the user that the lamp is already on.

[0056] The process thus described therefore makes it possible to convert a command expressed in natural language into a clear and precise logical instruction, facilitating decision-making and action by the system.

[0057] Several examples of processing particular commands are now described. In each of these examples, the conversion process presented above is implemented to obtain a logical formula, which is then processed by the logical reasoning engine,

[0058] In a first example, the user asks to turn on the living room lamp.

[0059] A simplified example of a query to the language model, presented arbitrarily as a series of JSON objects, is:{ role: system, content: You are an NLU system capable of understanding natural language. Instructions: ...Knowledge: ...Actions: - to turn a device on or off, use the intent (is-switched ?device ?state) - ...Entities: - to designate the living room lamp, use the symbol living-room-light- ...},{ role: user, content: color rectangle 1 blue},{ role: assistant, content: (I ?user (is-colored rectangle-1 blue))},{ role: user, content: what is the current temperature in the bedroom},{ role: assistant, content: (I ?user (B ?user (current-temperature bedroom ?temperature))},…,{ role: user, content: turn on the living room lamp}

[0060] The dotted lines refer to information that may be contained in the query but is not repeated here because it is not relevant to the explanation.

[0061] Specifically, descriptive data can be provided in the topic titled "Knowledge." A set of initial interpretation rules is also provided in the topics titled "Actions" and "Entities," and the command is provided as its own JSON object at the end of the query.

[0062] In this example, the set of first interpretation rules is conceptually divided into two parts, one relating to the designation of specific states of any equipment, and the other relating to the designation of specific equipment.

[0063] In this example, the request contains other information, including a header, possibly specific instructions, as well as several previous commands, one relating to an action request, the other relating to an information request, and the interpretation result associated with each. This various additional information can supplement or replace certain interpretation rules.

[0064] Based on the provided query, the language model provides an interpretation result for the command to turn on the living room light. The expected result is the following logical formula: (I ?user (is-switched-living-room-light-on))

[0065] This logical formula represents the user's intention and is of the form "(I ?user ?phi)"; it indicates that the user ?user intends the formula ?phi to be true. More specifically, it indicates that the user ?user intends the lamp designated by "living-room-lamp" to be turned on.

[0066] The logic reasoning engine is configured to receive and process the logic formula by applying a limited number of generic predetermined rules, each of which includes one or more input logic formulas and one or more output logic formulas.

[0067] The predetermined rules are generic in that the logic formulas contained in them do not require any reference to any particular device or state, which helps make them compatible with the processing of a large number of specific logic formulas that may be generated by the language model and provided as input.

[0068] Two predetermined rules are provided here as examples.

[0069] The first predetermined rule provides for signaling to the user, following receipt of an instruction to turn on equipment, a situation where the instruction cannot be applied because the equipment concerned is already turned on.

[0070] The first predetermined rule is: (I ?user (is-switched ?dev on)) & (is-switched ?dev on) -> (I self (B ?user (is-switched ?dev on))

[0071] It contains two logical formulas as antecedents, the first in the form "(I ?user ?phi)" and obtained as output from the command interpreter and the second in the form "?phi" and corresponding to a descriptive data stored in the database.

[0072] It also contains a logical output formula in the form "(I self (B ?user ?phi))" expressing the intention of the system (self) to communicate a specific descriptive data corresponding to "?phi" to the user.

[0073] The second predetermined rule is to turn on the equipment if it is turned off.

[0074] The second predetermined rule is: (I ?user (is-switched ?dev on) & (is-switched ?dev off) -> (I self (is-switched ?dev on)

[0075] Just like the first predetermined rule, it contains two logical formulas as antecedents, the first in the form "(I ?user ?phi)" and obtained as output from the command interpreter and the second in the form "?phi" and corresponding to a descriptive data stored in the database. It also contains an output logical formula in the form "(I self ?phi)", expressing the intention of the system (self) to carry out the action ?phi. For example, if there is an action to satisfy the intention ("is switched device on?"), this action (which in this case turns on the device) will be automatically executed.

[0076] The two predetermined rules thus described allow the logic reasoning engine to properly process an instruction to turn on the living room lamp. If the living room lamp is already on, the logic reasoning engine knows this by consulting the available descriptive data. It can then decide to send a message to the user to indicate that the lamp in question is already on. If the living room lamp is initially off, then the logic reasoning engine can proceed to issue an instruction to turn it on in accordance with the user's intention.

[0077] In a second example, the user asks if the living room lamp is on.

[0078] The corresponding query can be generated by the command interpreter to contain the same information as in the query shown in the first example, except for the command, which is formulated as: { role: user, content: is the living room lamp on}.

[0079] Based on the provided query, the language model provides an interpretation result for the command to indicate whether the living room lamp is on. The expected result is: "(I ?user (B ?user (is-switched living-room-lamp on))".

[0080] This is a logical formula of the form "(I ?user (B ?user ?phi))", which means that the user ?user intends to know a specific descriptive data corresponding to "?phi".

[0081] A logical formula with the same structure "(I ?user (B ?user (current-temperature bedroom ?temperature))" is provided in the query: it transcribes a user's intention to know the temperature in a room.

[0082] In the second example, the expected logical formula "(I ?user (B ?user (is-switched living-room-lamp on))" means that the user intends to know if the living-room lamp is switched on.

[0083] The reasoning engine can be configured to process this logical formula by applying a third predetermined rule providing that if the user wants to know a specific descriptive data corresponding to "?phi" then the system intends to communicate this descriptive data to him.

[0084] The third predetermined rule is: (I ?user (B ?user ?phi)) -> (I self (B ?user ?phi))

[0085] In a third example, the user asks to put the living room of his home in cinema mode.

[0086] It is assumed that such a mode involves several devices simultaneously. For example, cinema mode might correspond to a situation where the shutters are closed and the lamps are set to low brightness.

[0087] Since several actions potentially need to be implemented, it may be possible to calculate a corresponding action plan using a planner, for example of the PDDL type.

[0088] A possible result of interpreting the command by the language model is: (I ?user (movie-mode living-room on))

[0089] To achieve such a result, one possibility is that the set of predetermined rules provided in the query contains one or more predetermined rules specifying to translate the reference to the cinema mode in the command by the expression "movie-mode" in the logical formula, as well as to translate the reference to the living room in the command by the expression "living-room" in the logical formula.

[0090] Even if such specific rules are absent, a sufficient number of sufficiently varied predetermined rules can enable the language model to provide this result.

[0091] The reasoning engine can be configured to process this logic formula by applying a fourth predetermined rule providing that if the user intends to achieve a goal "?phi" and if there is a plan capable of satisfying the goal "?phi" then the system intends to execute this plan.

[0092] The fourth predetermined rule is: (I ?user ?phi) & (exists ?plan ?phi)) -> (I self (done ?plan))

[0093] The proposed technique offers multiple technical advantages compared to known systems and methods based exclusively on the application of predetermined NLU rules.

[0094] In particular, the proposed technique allows the user to use natural language more freely to express his commands.

[0095] The proposed technique simplifies the development of equipment control systems by eliminating the need to provide a large number of NLU rules.

[0096] The proposed technique also allows for maintaining control over the commands to be implemented through the logical reasoning engine, as the latter can rely on data stored in the database to authorize or not the execution of an action. Thus, for example, if the database has descriptive data relating to the user's age, the user can decide not to perform the action corresponding to their intention to turn on the oven. Another advantage of the proposed technique is that it does not require communicating to the language model the content, in terms of code, of the actions that can be performed. This advantage and the one expressed previously offer strong security guarantees.

[0097] Another advantage of the proposed technique is that it allows, thanks to the logical reasoning engine, by combining the commands expressed in natural language by the user with other types of interactions (tactile, haptic, etc.), to implement multimodal interactions. Thus, for example, if the user says "turn on this lamp" and if the system also has knowledge about the object designated by the user, then it can send this knowledge to the language model, which then produces a logical formula faithfully reflecting the corresponding intention.

[0098] Another advantage of the proposed technique is the possibility of implementing a planning engine via the reasoning engine in order to execute not an action but a plan of actions to satisfy the user's intention.

Claims

A method of converting a command, expressed at least in part in natural language and relating to a state of a piece of equipment, into a logical instruction, the method comprising obtaining (310) the logical instruction via an interpreter (104) of said command requesting (308) a language model (106) with a request comprising the command, a set of interpretation rules comprising only first interpretation rules, and data descriptive of the current state of the equipment, the logical instruction being consistent with what would be produced by an interpretation of the command by application of a second interpretation rule derivable from the set of first rules, the second interpretation rule being distinct from the first interpretation rules. The method of claim 1, wherein the logic instruction is adapted to cause a change in state of the equipment from a current state to a desired state when the logic instruction is executed by a logic reasoning engine (112). The method of claim 1 or 2, wherein the logic instruction is adapted to cause transmission of a signal specifying the current state when the logic instruction is executed by a logic reasoning engine (112). Method according to one of claims 1 to 3, in which the set of first rules contains a rule relating to a designation of the equipment and a rule relating to a designation of the desired state. The method of one of claims 1 to 4, further comprising transmitting (312) the logic instruction to a logic reasoning engine (112) for executing the logic instruction. Computer program comprising instructions which, when the program is implemented by a processor (202), lead to implementing the method according to one of claims 1 to 5. A non-transitory computer-readable recording medium (204) having recorded thereon the computer program according to claim 6. Interpreter (104) of a command, expressed at least in part in natural language and concerning a state of a piece of equipment, into a logical instruction, the interpreter being configured to obtain (310) the logical instruction by requesting (308) a language model (106) with a query comprising the command, a set of interpretation rules comprising only first interpretation rules, and a descriptive data item of the current state of the equipment, the logical instruction being consistent with what would be produced by an interpretation of the command by application of a second interpretation rule derivable from the set of first rules, the second interpretation rule being distinct from the first interpretation rules.

Images