Computer implemented method for obtaining an output response to a natural language input prompt
By determining complexity and time-criticality scores for input prompts, the method optimizes LLM selection on IoT edge devices, addressing resource constraints and ensuring high-quality responses, enhancing user comfort and device usability.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- LUXOTTICA SRL
- Filing Date
- 2025-12-18
- Publication Date
- 2026-06-25
AI Technical Summary
Deploying large language models (LLMs) on IoT edge devices like smart glasses requires significant computational and memory resources, making the devices impractical and uncomfortable to wear, while using simpler LLMs compromises performance, and randomly selecting cloud-based LLMs provides suboptimal responses.
A method to select a language model that best fits user needs by determining complexity and time-criticality scores of input prompts, using transformer and early-exit neural networks, to minimize computational and memory resources, ensuring optimal responses while balancing quality and latency.
The method reduces device weight and dimensions, making IoT edge devices comfortable to wear and easy to use, while providing high-quality and timely responses by selecting appropriate LLMs based on prompt analysis.
Smart Images
Figure EP2025087939_25062026_PF_FP_ABST
Abstract
Description
[0001] COMPUTER IMPLEMENTED METHOD FOR OBTAINING AN OUTPUT RESPONSE TO A NATURAL LANGUAGE INPUT PROMPT
[0002] The present invention refers to a computer implemented method for obtaining an output response generated by a language model on the basis of a natural language input prompt, and to a wearable device comprising computing means configured to execute such a method.
[0003] In the field of natural language processing, language models, and in particular large language models (also known as LLMs), have been developed and used for a wide variety of tasks, including speech recognition, machine translation, natural language generation and Al assistance.
[0004] With particular reference to Al assistants, several LLMs have been proposed to generate a swift output response on the basis of a natural language input prompt formulated by a user, and this variety enables users to choose the LLM that each time best fits their specific use-case requirements. However, the deployment of these LLMs as Al assistants in Internet of Things (loT) edge devices, such as wearable devices and in particular smart glasses, presents significant challenges.
[0005] In fact, running even a single high-performance LLM directly on an edge device requires large computational and memory resources, inevitably increasing the weight and the dimensions of the device. In the case of smart glasses, this would make the device impractical and even painful to wear. On the other hand, the use of a simpler LLM would compromise the overall performance of the Al assistant, providing the user with low quality and / or late output responses. As an alternative, the edge device might take advantage of cloud-based LLMs, to which the edge device might forward the input prompt using an Internet connection in order to receive an output response. Considering the wide variety of available cloud-based LLMs, it would be desirable to forward the input prompt to the cloud-based LLM that best fits the user needs, instead of randomly selecting a cloud-based LLM between the ones available: the random selection, in fact, provides the user with a suboptimal average quality of the output responses.
[0006] An object of the present invention is to overcome the above-mentioned drawbacks and in particular to set up a computer implemented method for obtaining an output response to a natural language input prompt, generated by the LLM that best fits the user needs in order to provide the user with the optimal output response, that requires limited computational and memory resources. Another object of the present invention is to realize a wearable device comprising computing means configured to execute the above-mentioned method, such device being comfortable to wear and easy to use.
[0007] These and other results are achieved according to the present invention setting up a computer implemented method according to claim 1, and a wearable device according to claim 12.
[0008] Further preferred characteristics are the object of the dependent claims.
[0009] The present invention will be now described, in an illustrative and non-limiting way, referring to the attached schematic drawings, in which:
[0010] - figure 1 is a sketch illustrating a particular implementation of a computer implemented method according to the present invention;
[0011] - figure 2 is a schematic drawing of the operational logic of the step of determining a complexity score and a time-criticality score of the input prompt.
[0012] With reference to figure 1, the computer implemented method according to the present invention allows to obtain an output response
[0013]
[0014] to a natural language input prompt x formulated by a user U, the output response Mi(x) being generated by a language model M, in particular a large language model.
[0015] The input prompt x could be any question formulated in natural language by the user U, such as " What is the height of the Eiffel tower?", or any other request of information such as " Tell me how to set up this desk.". On the other hand, the output response Mi(x) will be the answer to the formulated question (" The Eiffel tower is 312 metres high."), or the information requested (" Take the piece A and connect it with the piece B, then take the piece C and..."). The output response Mi(x) is generated by a language model
[0016]
[0017] selected from a language model group G of N possible language models M, and the method comprises a selection procedure S set up to select a selected language model
[0018]
[0019] that is used to generate the output response Mi(x) to the input prompt x.
[0020] Determination of the complexity score and the timecriticality score
[0021] First of all, the input prompt x is received, and a complexity score Ot(x) and a time-criticality score / 3 (x) of the input prompt x are determined. Preferably both the complexity score C
[0022]
[0023] t(x) and the time-criticality score / 3 (x^ of the input prompt x are comprised between 0 and 1. For example, the input prompt " Tell me how to heat milk." has a complexity score Ot(x^ close to 0, while the input prompt " Tell me how to design an aircraft." has a complexity score C
[0024]
[0025] t(x) close to 1. On the other hand, the input prompt " Give me some game suggestions." has a timecriticality score?(x) close to 0, while the input prompt " How do I treat my wasp sting immediately?" has a time-criticality score / 3 (x) close to 1.
[0026] With reference to figure 2, preferably both the complexity score C
[0027]
[0028] t(x) and the time-criticality score /
[0029]
[0030] 3 (x) of the input prompt x are determined on the basis of a text embedding 6(x^ of the input prompt x, and not directly from the input prompt x. In fact, it has been noted that text embedding neural networks 1 (such as Sentence-Bert and Jina-Bert) provide text embeddings e(x) that capture the semantic meaning of the input prompt x well enough to determine its complexity score a(x) and time-criticality score / 3 (x), limiting the overall computational and memory resources necessary to determine the complexity score C
[0031]
[0032] t(x) and the timecriticality score / 3 (x) of the input prompt x.
[0033] Preferably, the complexity score C
[0034]
[0035] t(x) of the input prompt x is determined using a transformer neural network 2. In fact, the self-attention mechanism in transformers allows the model to weigh the importance of different parts of the input prompt x dynamically. This is particularly useful for understanding the context and nuances in natural language, which is crucial for accurately assessing the complexity of the input prompt x. Further, transformers excel at capturing long-range dependencies and contextual relationships within the input prompt x. Complexity in language often arises from such dependencies, where the meaning and difficulty of an input prompt x can depend on the interplay between distant words or phrases. Transformers are well-suited to handle these dependencies effectively.
[0036] More in particular, the transformer neural network 2 comprises a first feed-forward layer 21, a transformer 22 and a second feed-forward layer 23, such that:
[0037] e(1)(x) = FeedForward(e(x))
[0038] e(2)(x) = Transformer
[0039] α(x) = FeedForward(e(2)(x))
[0040]
[0041] In fact, the first feed-forward layer 21 and the second feed-forward layer 23 help reduce the size of the input prompt x and consequently the memory footprint of the model.
[0042] Preferably, the complexity score Ot(x^ of the input prompt x is used to determine the time-criticality score / 3 of the input prompt x.
[0043] In fact, the complexity of an input prompt x is often tied to measurable factors such as the length of the input prompt x, the syntactic and semantic intricacy, and the need for advanced language understanding or generation capabilities. These factors are generally more straightforward to quantify and analyze using established natural language processing techniques. Further, complexity prediction relies more on the intrinsic properties of the input prompt x rather than external factors. This intrinsic focus makes it less context-dependent compared to time-criticality, which can be influenced by external and situational factors. Allowing complexity score Ot(x^ prediction to be used as a guiding prior for time-criticality score / 3 (x^ prediction facilitates in mitigating challenges such as dependence on external factors such as the current state of the user U, the environment, or the specific application requirements. These factors can be dynamic and fluctuate in real-time, adding complexity to the prediction process and requiring long-horizon multi-step reasoning. Finally, unlike time-criticality, there exist well-defined metrics and features in natural language processing to assess the complexity of text, such as syntactic tree depth, number of entities, sentiment analysis, and lexical diversity. These features can be extracted from the input prompt x and used to train models that predict complexity with relatively high accuracy.
[0044] More in particular, the time-criticality score / 3 (x^ of the input prompt x is determined also on the basis of the complexity score Cc x') of the input prompt x using an early-exit neural network 3. The early-exit neural network 3 is designed to provide exit points at different layers of the network, allowing the model to stop processing once a confident prediction is made and ensuring that the system can quickly respond to urgent prompt inputs x without unnecessary computation.
[0045] The early-exit neural network 3 has a number of exits n that preferably is equal to four. The first exit comprises a third feed-forward layer 31 and a fourth feed-forward layer 32, and provides a first binary class output
[0046]
[0047] (x) and a first time-criticality score prediction
[0048]
[0049] such that:
[0050] b(1)(x) = FeedForward(e(x),α(x))
[0051] σ(1)(x) = sigmoid(FeedForward(b(1)(x)))
[0052]
[0053] The second exit comprises a fifth feed-forward layer 33 and a sixth feed-forward layer 34, and provides a second binary class output (J^ (x and a second timecriticality score prediction j?(2)(x), such that:
[0054] r / 2)(x) σ(1)(x) = sigmoid(FeedForward(b(1)(x)))
[0055]
[0056] The third exit comprises a seventh feed-forward layer 35 and an eighth feed-forward layer 36, and provides a third binary class output
[0057]
[0058] and a third time-criticality score prediction
[0059]
[0060] such that:
[0061] σ(3)(x) = sigmoid(FeedForward(b(3)(x)))
[0062]
[0063] The fourth exit comprises a ninth feed-forward layer 37 and a tenth feed-forward layer 38 and provides a fourth time -criticality score prediction
[0064]
[0065] such that:
[0066]
[0067] The binary class output σ(i)(x)
[0068]
[0069] a prediction of whether to exit at the i-th exit while the timecriticality score prediction
[0070]
[0071] the output of the i-th exit.
[0072] The binary class output σ(i)(x) allows to exit early;
[0073]
[0074] particular the time-criticality score / 3 (x) of the input prompt x is determined as the earliest time-criticality score prediction β(i)(x) such that the prediction of the corresponding binary class output σ(i)(x) is positive, and all the previous binary class outputs σ(i)(x) are negative. Specifically:
[0075] β(x) = β(k)(x) | [(σ(k)(x) > 0.5)
[0076]
[0077] A (a^(x) < 0.5 V i < / <)] Training of the transformer neural network and the early-exit neural network
[0078] Preferably, the transformer neural network 2 and the early-exit neural network 3 are trained minimizing an overall loss function L given by a linear combination of a first loss function L1related to the complexity score α(x) and the time-criticality score β(x) of the input prompt x, and a second loss function L2related to the binary class output σ(i)(x) of the early exit neural network. A control hyperparameter X is defined such that:
[0079] L = L1+ λ · L2
[0080] This linear combination allows balancing the trade-off between the accuracy of the scores
[0081]
[0082] α(x), β(x) and the effectiveness of the early-exit mechanism.
[0083] More in particular, the training is based on a training dataset Y comprising a training total m of training prompts
[0084]
[0085] . The training dataset Y is preferably a fraction of a larger dataset such as MT Bench or HH Bench.
[0086] For each training prompt
[0087]
[0088] a ground truth complexity score α(yi) and a ground truth time-criticality score
[0089]
[0090] are determined, preferably using a high-parameter large language model as judge language model, for example GPT-4o.
[0091] The first loss function L1is defined as the mean squared error between the scores α(yi), β(yi) determined by the neural networks 2, 3 on the basis of the training prompts yi, and the ground truth scores
[0092]
[0093] determined on the basis of the same training prompts
[0094]
[0095] . Specifically:
[0096]
[0097] i=l For each training prompt
[0098]
[0099] and for each exit k, a ground
[0100] truth binary class output
[0101]
[0102] (y^) is determined using a discretization threshold t: when the time-criticality score prediction
[0103]
[0104] differs from the ground truth
[0105] time-criticality score
[0106]
[0107] of less than the discretization threshold t, then the ground truth binary class output
[0108]
[0109] is 1, else 0. Specifically:
[0110]
[0111] = id[\^kKyt) - p(yt)\ < t] Preferably the discretization threshold t is equal to 0.5. By setting the discretization threshold t, we ensure that the model only exits early if the predictions are sufficiently accurate. This prevents premature exits that could lead to significant errors in the final predictions.
[0112] For each training prompt
[0113]
[0114] and for each exit k, a binary r (k) f
[0115] cross-entropy function (yt) is determined on the
[0116] basis of the ground truth binary class output
[0117]
[0118] CXi)
[0119] and the binary class output σ(k)(yi). Specifically: = - [<?(k)(yi) log (o-(k)(yi)) +
[0120]
[0121] + (1 - <j(k)(yi))log(1 - σ̂(k)(yi))] The second loss function L2 is defined on the basis of the binary cross-entropy function L2(k)(yi). Specifically:
[0122] m n-1
[0123]
[0124] i=l k=l
[0125] Preferably, the control hyperparameter X is equal to 1. Computation of the quality of service score and selection of the selected language model
[0126] For each language model
[0127]
[0128] of the language model group G, a response quality function
[0129]
[0130] of the language
[0131] model Miand a latency function φ̃(Mi) of the language model Miare stored in memory.
[0132] When the complexity score α(x) and the time-criticality score β(x) of the input prompt x have been determined, a quality of service score QoS x, Mi) related to the combination of each language model M[ and the input prompt x is computed as follows:
[0133] Q
[0134]
[0135] oS(x, Mi) = α(x) · μ̃(Mi) + β(x) · φ̃(Mi) The response quality function μ̃(Mi) of the language model Miis computed on the basis of the quality of the responses generated, i.e. the correctness and the helpfulness. Preferably, the of response quality function μ̃(Mi) of the language model
[0136]
[0137] is computed on the basis the response quality scores
[0138]
[0139] of the responses
[0140]
[0141] generated by the language model M on the basis of the training prompts y, using also a confidence level hyperparameter z. Specifically:
[0142] SDy∈Y(μ(y, Mi)) μ̃(Mi) = Meany∈Y(μ(y, Mi)) - z · —
[0143]
[0144] y / m where the symbol "SD" represents the standard deviation function.
[0145] Preferably the response quality score μ(y, Mi) of a response Mi(y) generated by each language model Mion the basis of the training prompt y is determined using a judge language model. In this case, the judge language model is asked to determine the response quality score considering correctness and helpfulness of the
[0146]
[0147] response
[0148] On the other hand, the latency function
[0149]
[0150] of the language model
[0151]
[0152] is computed on the basis of the time employed by the language model
[0153]
[0154] to generate the
[0155] responses. Preferably, the latency function
[0156]
[0157] of the language model
[0158]
[0159] is computed on the basis of the latency scores
[0160]
[0161] of the responses
[0162]
[0163] generated by the language model
[0164]
[0165] on the basis of the training prompts y. Specifically: φ̃(Mi) = Meany∈Y(φ(y, Mi)) - z · —
[0166]
[0167] m Preferably the latency score φ(y, Mi) of a response Mi(y) generated by the language model Mion the basis of the training prompt y is computed as:
[0168] φ(y, Mi) = 1 - [t(y, Mi) - minM∈G(t(y, Mj))] / [maxM∈G(t(y, Mj)) - minM∈G(t(y, Mj))]
[0169]
[0170] MjEG VV' ' MjEG \V 7wherein t(y, Mi) is the time employed by the language model Mi(selected from the language model group G) to generate the response Mi(y) on the basis of the training prompt y.
[0171] Preferably, the confidence level hyperparameter z is comprised between 1.40 and 1.50, in particular 1.44. Finally, the language model Miwhose quality of service score QoS(x, Mi) computed on the basis of the input prompt x is the highest is chosen as the selected language model Mi. The input prompt x is forwarded to the selected language model
[0172]
[0173] and the output response M((x) generated by the selected language model
[0174]
[0175] on the basis of the input prompt x is received.
[0176] This selection procedure S allows to make informed decisions about which language model Mito deploy for a given task by considering both the quality and latency in a conservative manner.
[0177] The computer implemented method according to the present invention also minimizes the computational and memory resources necessary to obtain the output response
[0178]
[0179] to the natural language input prompt x, since the output response Mi(x) is obtained by slow and high-quality language models
[0180]
[0181] only when the time-criticality score / 3 (x^ of the input prompt x is low, and the output response Mtx) is obtained by fast and low-quality language models M only when the complexity score Ot(x^ of the input prompt x is low. In turn, when the method is run on an IoT edge device, this allows to reduce the weight and the dimensions of the device itself; in the case of smart glasses, this makes the device comfortable to wear and easy to use.
[0182] Wearable device
[0183] Preferably, the computer implemented method is executed by computing means 11 of a wearable device 10 (visible in figure 1), in particular a head-mountable device, more in particular smart glasses, that can be worn by the user U.
[0184] The device 10 comprises also input means configured to collect the natural language input prompt x formulated by the user U, and to forward the input prompt x to the computing means 11; the device 10 comprises also output means configured to receive the output response
[0185]
[0186] obtained by the computing means 11 and to provide the user U with the output response
[0187]
[0188] The input means can comprise a microphone configured to collect a vocal input prompt x formulated by the user. The output means can comprise a speaker configured to provide the user U with an audible output response Mgx). The output means comprise also a display configured to provide the user U with a visible output response
[0189]
[0190] The visible output response Mi(x) could be projected on a lens of the device 10.
[0191] The present invention has been described in an illustrative and non-limiting way according to relative favorite embodiments, but it is clear that the person skilled in the art could perform many variations and modifications, all of which are within the scope of the invention.
Claims
CLAIMS1) Computer implemented method for obtaining an output response (Mi(x)) to a natural language input prompt (x), the output response (Mi(x)) being generated by a language model (Mi) selected from a language model group (G) of language models (Mi), comprising the steps:- receiving the input prompt (x);- determining a complexity score (α(x)) of the input prompt (x);- determining a time-criticality score (β(x)) of the input prompt (x);- for each language model (Mi), computing a quality of service score (QoS(x, Mi)) related to the combination of the language model (Mi) and the input prompt (x), on the basis of the complexity score (α(x)) of the input prompt (x), the time-criticality score (β(x)) of the input prompt (x), a stored response quality function (μ̃(Mi)) of the language model (Mi) computed on the basis of a quality of the responses generated by the language model (Mi), and a stored latency function (φ̃(Mi)) of the language model (Mi) computed on the basis of a time employed by the language model (Mi) to generate responses;- selecting a selected language model (Mi) on the basis of the computed quality of service score (QoS(x, Mi));- forwarding the input prompt (x) to the selected language model (Mi);- receiving the output response) generated by the selected language model (M^) on the basis of the input prompt (x).2) Method according to claim 1, wherein the complexity score (α(x)) and the time-criticality score (β(x)) of the input prompt (x) are determined on the basis of a text embedding (e(x)) of the input prompt (x).3) Method according to any of the preceding claims, wherein the complexity score (α(x)) of the input prompt (x) is determined using a transformer neural network (2).4) Method according to any of the preceding claims, wherein the time-criticality score (β(x)) of the input prompt (x) is determined also on the basis of the complexity score (α(x)) of the input prompt (x).5) Method according to claim 4, wherein the time-criticality score (β(x)) of the input prompt (x) is determined using an early-exit neural network (3).6) Method according to claim 5, when dependent in turn from claim 3, wherein the transformer neural network (2) and the early-exit neural network (3) are trained minimizing an overall loss function (L) given by a linear combination of a first loss function (L^) and a second loss function (L2), wherein the first loss function (L^) is a mean squared error function related to thecomplexity score and the time-criticality score (β(x)), and the second loss function (L2) is a binary cross-entropy loss function related to a binary class output (σ̂(i)(x)) of the early-exit neural network (3).7) Method according to any of the preceding claims, wherein the quality of service score (QoS(x, Mi)) related to the combination of the language model (Mi) and the input prompt (x) is computed on the basis of the complexity score (α(x)) of the input prompt (x), the time-criticality score (β(x)) of the input prompt (x), the response quality function (μ̃(Mi)) of the language model (Mi), and the latency function (φ̃(Mi)) of the language model (Mi), as:QoS(x, Mi) = α(x) · μ̃(Mi) + β(x) · φ̃(Mi) 8) Method according to claim 7, wherein the response quality function (μ̃(Mi)) and the latency function (φ̃(Mi)) of the language model (Mi) are computed on the basis of a training dataset (Y) comprising a training total (m) of training prompts (yi), a response quality score (μ(y, Mi)) of a response (Mi(y)) generated by the language model (Mi) on the basis of the training prompt y, a latency score (φ(y, Mi)) of the response (Mi(y)) generated by the language model (Mi) on the basis of the training prompt (y), and a confidence levelhyperparameter (z), as:SDy∈Y(μ(y, Mi)) μ̃(Mi) = Meany∈Y(μ(y, Mi)) - z · — SDy∈Y(φ(y, Mi)) φ̃(Mi) = Meany∈Y(φ(y, Mi)) - z · —ym 9) Method according to claim 8, wherein the response quality score (μ(y, Mi)) of the response (Mi(y)) generated by each language model (Mi) on the basis of the training prompt (y) is determined using a judge language model.10) Method according to claim 8 or claim 9, wherein the latency score (φ(y, Mi)) of the response (Mi(y)) generated by the language model (Mi) on the basis of the training prompt (y) is computed on the basis of the time (t(y, Mi)) employed by the language model (Mi) to generate the response (Mi(y)) on the basis of the training prompt (y), as:φ(y, Mi) = 1 - [t(y, Mi) - minM∈G(t(y, Mj))] / [maxM∈G(t(y, Mj)) - minM∈G(t(y, Mj))]MjEG VVMjEG VV J11) Method according to any of claims 8 to 10, wherein the confidence level hyperparameter (z) is comprised between 1.40 and 1.50, in particular 1.44.12) Wearable device (10), in particular head-mountable device, that can be worn by a user (U), comprising: - computing means (11) configured to execute a method according to any of claims 1 to 11;- input means configured to collect the input prompt (x) formulated by the user (U) and to forward the input prompt (x) to the computing means (11);- output means configured to receive the output response) obtained by the computing means (11) and to provide the user (U) with the output response13) Device (10) according to claim 12, wherein the input means comprise a microphone configured to collect a vocal input prompt (x) formulated by the user (U).14) Device (10) according to any of claims 12 or 13, wherein the output means comprise a speaker configured to provide the user (U) with an audible output response (Mi(x)).15) Device (10) according to any of claims 12 to 14, wherein the output means comprise a display configured to provide the user (U) with a visible output response (Mi(x)).