Edge artificial intelligence system and context management method thereof
Patent Information
- Application Number
- TW114104698
- Authority / Receiving Office
- TW · TW
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-02-07
- Publication Date
- 2026-08-16
- Estimated Expiration
- 2045-02-06
AI Technical Summary
Large language models like ChatGPT-4, Copilot, and Claude 3.5 face limitations in context management due to lexical limits, short-term memory, and computational inefficiencies, leading to inaccurate responses and loss of dialogue history relevance.
An edge AI system with a mixer module, classifier module, and sorting module that classifies dialogue history into topic categories using a small-scale language model, adding timestamps and sequence IDs to manage context effectively, reducing computational burden and maintaining dialogue coherence.
Enhances context management by retaining all dialogue history, improving computational efficiency, and maintaining response accuracy and coherence by categorizing and ordering prompts based on similarity, thus reducing errors and resource waste.
Smart Images
Figure TWG2TA001072131_001 
Figure TWG2TA001072131_002 
Figure TWG2TA001072131_003
Abstract
Description
[Technical Field]
[0001] The present invention relates to a context management method for edge artificial intelligence systems, and in particular to a method for sorting dialogue history records using edge artificial intelligence systems. [Previous Technology]
[0002] Currently, commonly used large language models (LMs) include Copilot, ChatGPT4, and Claude 3.5. These large language models each have their own unique context management methods.
[0003] Copilot's context management method prioritizes recent interactions as the dialogue history increases to ensure the relevance of responses. Furthermore, Copilot has a limit on the number of tokens (i.e., the smallest unit of text) it can process in each dialogue. This token limit directly affects the context capacity of the dialogue; once the limit is exceeded, earlier tokens are discarded to process the latest content.
[0004] ChatGPT-4's context management method uses a fixed context window to define the maximum amount of text that can be processed. ChatGPT-4 performs operations based on tokens; when the input exceeds the context window, it discards the oldest tokens and retains only the newest ones to focus on processing the most recent content. Furthermore, each token in ChatGPT-4 can be associated with other tokens in the input, determining which tokens are most important for generating the response to the current question based on attention weights. This method processes key content through an attention mechanism, ignoring irrelevant parts even with a large amount of context. ChatGPT-4 essentially only has short-term memory and can only generate responses based on the context provided by the current interaction. As the conversation progresses, short-term memory is constantly updated, gradually forgetting earlier content. Once the context window limit is exceeded, earlier important information may be lost, affecting the accuracy of the response. In addition, errors in early responses may carry over to subsequent conversations, causing errors to accumulate.
[0005] Claude 3.5's context management method analyzes the current conversation's context to determine if the current topic still aligns with the user's interests. When it detects a decrease in user interest in the current topic or a shift in the question's direction, it may suggest switching topics to maintain conversation flow. Furthermore, when it determines a new topic better suits the user's interests, it may proactively introduce that topic. Claude 3.5 can dynamically adjust its conversation strategy based on algorithms, ensuring the content is both relevant and engaging.
[0006] Currently known large language models have the following problems in context management: 1. Limitation on the number of lexical units: ChatGPT-4, Copilot, and some other large language models are all limited by the number of lexical units when processing input. Since large language models require extremely high computational resources to operate, their lexical capacity cannot be easily expanded. Furthermore, these large language models include both the user's questions and the model's responses in the lexical unit count, resulting in a reduction in the number of effective lexical units available to the user and wasting resources. 2. Impact of dialogue history: When the dialogue history is too long, large language models usually suggest that the user switch topics, reminding the user that the model will forget previous dialogue history. Different large language models adopt different strategies when dealing with excessively long dialogue history. For example, ChatGPT-4, due to its detailed responses, quickly exceeds the context window's capacity; while Copilot forces a new dialogue to begin after answering about 30 questions or reaching a certain time.
[0007] Due to the influence of attention mechanisms and the limitation of only having short-term memory, the response accuracy of large language models may be affected. For example, when asked, "How many questions have I asked?", Copilot correctly responds with "3 questions." After asking two clearly different questions, and then asking the same question again, "How many questions have I asked?", Copilot will respond with the previous answer "3 questions" instead of updating to "5 questions." On the one hand, large language models typically process context through attention mechanisms. In some situations, large language models over-rely on previous responses, mistakenly assuming that previous responses are still valid and failing to accurately update the count. On the other hand, the memory of large language models is limited to the context window of the current dialogue and cannot track all the details of the entire dialogue history. Therefore, large language models can only respond based on what is "visible" and cannot accumulate the number of previous questions. Additionally, some large language models, such as ChatGPT-4 and Copilot, may be too eager to forget questions unrelated to the current topic, leading to information loss and disjointed dialogue. Some large language models, such as Claude 3.5, only remember the relevant content of the last dialogue topic, completely ignoring previous discussions. While this strategy can reduce computational burden, it may lead to the loss of context, affecting the accuracy of responses. Therefore, how to more effectively manage the use of lexical units, avoid unnecessary computational burden, and maintain the relevance and coherence of dialogue is one of the key directions for the continuous improvement of large language models. [Summary of the Invention]
[0008] Given that ChatGPT-4 and Copilot rely on the complete dialogue history for attention calculations, while Claude 3.5 relies entirely on the last dialogue topic, a balance should be sought. All dialogue history records should be retained, and the current question should be determined to belong to a specific topic category. Then, the dialogue history records for that topic category should be provided as prompts to the large language model. Other parts of the dialogue history can be stored in local memory instead of being directly used in the current calculation. This approach not only avoids over-reliance on the entire dialogue history but also improves computational efficiency and reduces unnecessary computational burden.
[0009] As the number of input tokens in large language models continues to increase, multiple questions may be calculated as the same sentence, which could lead to a lack of understanding of time series. To avoid this, a timestamp or sequence ID can be added to the text of each question. This helps to effectively distinguish different questions, avoid referencing incorrect information, and prevent errors from accumulating. In addition, this method can also solve the problem that some large language models cannot understand time zones or lack time awareness.
[0010] This invention provides an edge artificial intelligence system, comprising: a mixer module, a classifier module, and a sorting module. The mixer module receives user questions, adds auxiliary information to the user questions to generate the current question, and stores the current question in the dialogue history. The classifier module receives the dialogue history and uses a small-scale large-scale language model (small-scale LLM) to classify the dialogue history into at least one topic category, and determines the topic category to which the current question belongs. The sorting module sorts historical questions and historical responses in the topic category to which the current question belongs based on their similarity to the current question, to generate a prompt output order for the topic category to which the current question belongs, and transmits the prompt output order to an external large-scale language model (external LLM).
[0011] The present invention provides a context management method for an edge artificial intelligence system, comprising: receiving a user question by a mixer module and attaching auxiliary information to the user question to generate a current question and storing the current question in a dialogue history record; receiving the dialogue history record by a classifier module and classifying the dialogue history record into at least one topic category using a small LLM, and determining the topic category to which the current question belongs; and sorting the historical questions and historical responses in the topic category to which the current question belongs according to their similarity to the current question by a sorting module to generate a prompt output order for the topic category to which the current question belongs, and transmitting the prompt output order to an external LLM.
Implementation Method
[0012] FIG1 is a block diagram of a first embodiment of an edge artificial intelligence system 100 according to an embodiment of the present invention, and FIG2 is a schematic diagram of a second embodiment of an edge artificial intelligence system 200 according to an embodiment of the present invention. Please refer to FIG1 and FIG2 simultaneously below.
[0013] In the embodiment shown in FIG1, the edge AI system 100 includes a mixer module 110, a classifier module 120, and a sorting module 130. The mixer module 110 is used to receive a user question 201, attach auxiliary information 202 to the user question 201 to generate a current question (the latest user question 201 containing auxiliary information 202), and store the current question in the dialogue history record 211. The classifier module 120 is used to receive the dialogue history record 211, and classify the dialogue history record 211 using a small LLM 140 to classify the dialogue history record 211 into at least one topic category 221, and determine the topic category 221 to which the current question belongs. The sorting module 130 is used to sort the historical questions and historical responses in the topic category 221 to which the current question belongs according to their similarity to the current question, to generate a prompt output order 231 for the topic category 221 to which the current question belongs, and transmit the prompt output order 231 to the external LLM 150. The dialogue history 211 contains historical questions and responses. Historical questions are questions asked by the user before asking the current question. Historical responses are responses generated by the external LLM 150 based on the corresponding historical questions. Historical questions and responses are stored in memory and become part of the dialogue history 211.
[0014] In one embodiment, the edge AI system 100 can be applied to an edge AI device. The edge AI device can be a smart wearable device, a smart mobile device, a personal computer, a laptop computer, or a server. The edge AI device includes a computing unit (e.g., a central processing unit) and a memory unit (e.g., random access memory or non-volatile memory elements). The edge AI device implements the functions of the mixer module 110, the classifier module 120, and the sorter module 130 in the edge AI system 100 through the computing unit. The memory unit of the edge AI device is used to store dialogue history records 211. In addition, the edge AI system 100 includes a small LLM 140. The small LLM 140 is a lightweight version of a large language model with fewer parameters, designed to run efficiently on a single GPU or low-performance hardware. Compared to large-scale language models (e.g., GPT-4, PaLM), the smaller LLM 140 is more focused on achieving specific application goals while retaining core capabilities in language understanding, generation, and reasoning. Classifier module 120 uses the smaller LLM 140 to determine the topic category 221 to which the current question belongs. Ranking module 130 uses the smaller LLM 140 to sort historical questions and responses within the topic category 221 to which the current question belongs. In one embodiment, the smaller LLM 140 may be housed within classifier module 120.
[0015] In the embodiment shown in FIG2, the edge artificial intelligence system 200 includes a mixer module 110, a classifier module 120, a selector module 210, and a sorter module 130.
[0016] In the embodiment shown in FIG2, the mixer module 110 of the edge AI system 200 is used to receive the user question 201 and attach auxiliary information 202 to the user question 201 to generate the current question (the latest user question 201 containing auxiliary information 202). Additionally, the edge AI system 200 can generate the auxiliary information 202 through the edge AI device, for example: generating clock information 203 using the clock of the edge AI device and sending it to the mixer module 110; and generating global positioning system information 204 using the global positioning system (GPS) of the edge AI device and sending it to the mixer module 110. The mixer module 110 can attach clock information 203 and GPS information 204 to the user question 201. Furthermore, the mixer module 110 can attach a sequence identifier to the user question 201 to generate the current question. Therefore, the auxiliary information 202 can include time information, time zone information, clock information 203, GPS information 204, and / or a sequence identifier. The format of the current question is, for example, "2024 / 9 / 12 14:56 (UTC+8) How many questions have I asked?" In addition, the mixer module 110 stores the current question in the dialogue history record 211 so that the order of each historical question can be identified from the dialogue history record 211.
[0017] In one embodiment, the edge AI system 200 can store the dialogue history record 211 in the random access memory (RAM) of the edge AI device, and allocate a region in the memory to store the dialogue history record 211. In addition to the current question, the dialogue history record 211 also includes historical questions and historical responses. Historical questions are questions that the user asked before the current question arose, and are stored in the memory by the additional auxiliary information 202, becoming part of the dialogue history record 211. Historical responses are responses generated by the external LLM 150 based on the corresponding historical questions, and are also stored in the memory by the additional auxiliary information 202, becoming part of the dialogue history record 211. In one embodiment, the edge AI system 200 can also store the dialogue history record 211 in a non-volatile memory element of the edge AI device, such as a hard drive or flash memory.
[0018] In the embodiment shown in FIG2, the classifier module 120 of the edge AI system 200 is used to receive the dialogue history record 211 and classify the dialogue history record 211 using a small LLM 140 to classify the dialogue history record 211 into at least one topic category 221, and to determine the topic category 221 to which the current question belongs. In one embodiment, the classifier module 120 is provided with a tokenizer. The tokenizer is used to split the current question and historical questions in the dialogue history record 211 into multiple tokens for processing by the small LLM 140. In another embodiment, the classifier module 120 does not provide a tokenizer, but instead sets the tokenizer in the small LLM 140 to preprocess the dialogue history record 211.
[0019] In one embodiment, the classifier module 120 includes a small LLM 140. The classifier module 120 uses the small LLM 140 to determine the topic category 221 to which the current question belongs. The small LLM 140 consists of an embedding layer, a transformer layer, and a classification head layer. The embedding layer is used to convert the multiple tokens split from the current question in the dialogue history record 211 into embedding vectors. The transformer layer is used to process the embedding vectors to generate a contextual semantic representation of the current question. The classification head layer is used to determine the topic category 221 to which the current question belongs based on the contextual semantic representation of the current question.
[0020] In one embodiment, the classifier module 120 calculates the semantic similarity between the current problem and each historical problem one by one. The classifier module 120 uses a small LLM 140 to convert the current problem and all historical problems into vector representations, and measures the semantic similarity between the current problem and each historical problem based on the vector distance between the current problem and each historical problem. Common similarity measurement methods include cosine similarity or Euclidean distance. Cosine similarity measures the similarity between two vectors by calculating the angle between them, with a value range from 1 (representing a small vector distance) to -1 (representing a large vector distance). When the cosine similarity is close to 1, it means that the two problems are very similar semantically; when the cosine similarity is close to 0, it means that the two problems are significantly different semantically; and when the cosine similarity is close to -1, it means that the two problems are almost completely opposite semantically. Euclidean distance is another way to measure the distance between two vectors in space. The smaller the Euclidean distance (representing a smaller vector distance), the smaller the semantic difference between the two problems; and the larger the Euclidean distance (representing a larger vector distance), the greater the semantic difference between the two problems.
[0021] In one embodiment, the classifier module 120 clusters historical questions and uses a clustering algorithm (e.g., K-means or hierarchical clustering) to classify each historical question in the dialogue history record 211 to classify at least one topic category 221 and determine the boundary of each topic category 221. To evaluate the similarity between the current question and each topic category 221, the vector distance between the vector of the current question and the center vector of each topic category 221 is calculated to obtain the similarity between the current question and each topic category 221. The center vector of the topic category 221 is defined as the average vector of all historical questions in the topic category 221. In addition, the classifier module 120 may set a threshold (e.g., 0.8). If the similarity between the current question and a certain topic category 221 exceeds the threshold, the current question is considered similar to that topic category 221 and is classified into that topic category 221. Conversely, if the similarity between the current question and all topic categories 221 is below the threshold, the current question is considered dissimilar to any existing topic category 221. In this scenario, classifier module 120 can create a new topic category 221 and use the current question as the first question in the new topic category 221.
[0022] In the embodiment shown in Figure 2, the selector module 210 of the edge AI system 200 is used to select a topic category 221 containing the current question, so as to output all historical questions and historical responses in the topic category 221 to which the current question belongs. In one embodiment, the selector module 210 determines whether the topic category 221 contains the current question. When the selector module 210 determines that a topic category 221 contains the current question, it outputs all historical questions and historical responses in that topic category 221 to the sorter module 130. When the selector module 210 determines that the topic category 221 does not contain the current question, it will judge the next topic category 221.
[0023] In the embodiment shown in FIG2, the sorting module 130 of the edge AI system 200 is used to receive historical questions and historical responses in the topic category 221 to which the current question belongs, and arrange the order of historical questions and historical responses in the topic category 221 to which the current question belongs according to the similarity between the current question and historical questions and the similarity between the current question and historical responses, so as to generate a prompt output order 231 for the topic category 221 to which the current question belongs, and transmit the prompt output order 231 to the external LLM 150. In one embodiment, the sorting module 130 arranges according to the semantic similarity between the current question and historical questions and the semantic similarity between the current question and historical responses. In response to the sorting module 130 determining that the higher the similarity between the current question and historical questions and the higher the similarity between the current question and historical responses, the historical questions and historical responses are arranged in the prompt output order 231 closer to the current question. In response to the sorting module 130 determining that the similarity between the current question and a historical question is lower, and the similarity between the current question and a historical response is lower, the historical question and historical response are arranged in the prompt output order 231 at positions further away from the current question. In response to the sorting module 130 determining that the similarity between the current question and a historical question is 100% or substantially the same, and determining that the similarity between the current question and a historical response is 100% or substantially the same, the historical question and historical response are not arranged in the prompt output order 231. In one embodiment, in response to the sorting module 130 determining that the similarity between the current question and a historical question is 100% or substantially the same, the historical question and the corresponding historical response are not arranged in the prompt output order 231.
[0024] Please refer to FIG3, which illustrates a sorting method of the sorting module 130 of the edge artificial intelligence system 100 according to an embodiment of the present invention. In the embodiment shown in FIG3, the sorting module 130 may include an input data structure and an output data structure. The input data structure is used to store historical questions (Q1, Q2, ..., QN) and historical responses (A1, A2, ..., AN) in the topic category 221 to which the current question (QNow) belongs, received from the selector module 210. The input data structure is used to store historical questions (Q1, Q2, ..., QN) and historical responses (A1, A2, ..., AN) after being sorted by the sorting module 130. The data in the input data structure that has not been processed by the sorting module 130 may be arranged chronologically or according to other rules based on historical questions (Q1, Q2, ..., QN) and historical responses (A1, A2, ..., AN), for example: Q1, A1, Q2, A2...QN, AN. The sorting module 130 sorts the current question (QNow) based on the vector distance between it and the previous questions (Q1, Q2, ..., QN) and the vector distance between it and the previous responses. The vector distance is calculated using cosine similarity or Euclidean distance. As shown in Figure 3, the vector distance between the current question (QNow) and the previous question (Q1) is d1; the vector distance between the current question (QNow) and the previous response (A1) is d2; and so on, the vector distance between the current question (QNow) and the previous response (AN) is d2N. The sorting module 130 arranges the previous questions (Q1, Q2, ..., QN) and previous responses (A1, A2, ..., AN) within the topic category 221 to which the current question (QNow) belongs, according to the magnitude of the vector distances (d1, d2, ..., d2N) to generate the sorting result. The sorting module 130 stores the sorting results into the output data structure, such as: Q1, A1, QNOW, A5, Q2…Q4, A2.
[0025] In one embodiment, the sorting module 130 normalizes the vector distance to the range [0,1] to obtain the similarity between the current question (QNow) and historical questions (Q1, Q2, ..., QN), and the similarity between the current question (QNow) and historical responses (A1, A2, ..., AN). Based on the similarity between the current question (QNow) and historical questions (Q1, Q2, ..., QN), and based on the similarity between the current question (QNow) and historical responses (A1, A2, ..., AN), the sorting module 130 determines whether the similarity between the current question (QNow) and historical questions (Q1, Q2, ..., QN) is 100% or substantially the same, and also determines whether the similarity between the current question (QNow) and historical responses (A1, A2, ..., AN) is 100% or substantially the same. When the sorting module 130 determines that the similarity between the current question (QNow) and a historical question (e.g., Q1) is 100% or substantially the same, and when the sorting module 130 determines that the similarity between the current question (QNow) and a historical response (e.g., A1) is 100% or substantially the same, the historical question (e.g., Q1) and historical response (e.g., A1) are removed from the output data structure of the sorting module 130, or the historical question (e.g., Q1) and historical response (e.g., A1) are not stored in the output data structure of the sorting module 130. Then, the sorting module 130 outputs the prompt output sequence 231 from the output data structure to the external LLM 150 for transmission. At this time, the data in the prompt output sequence 231 is QNOW, A5, Q2...Q4, A2, excluding Q1 and A1.
[0026] In one embodiment, the sorting module 130 sorts the current problem (QNow) and historical problems (Q1, Q2, ..., QN) according to their similarity, and determines whether the similarity between the current problem (QNow) and historical problems (Q1, Q2, ..., QN) is 100% or substantially the same. When the sorting module 130 determines that the similarity between the current problem (QNow) and historical problems (e.g., Q1 and QN) is 100% or substantially the same, the historical problems (e.g., Q1 and QN) and their corresponding historical responses (e.g., A1 and AN) are removed from the output data structure of the sorting module 130, or the historical problems (e.g., Q1 and QN) and their corresponding historical responses (e.g., A1 and AN) are not stored in the output data structure of the sorting module 130. Then, the sorting module 130 outputs the sorting result from the output data structure to the prompt output sequence 231 for transmission to the external LLM 150. At this time, the data in the prompt output sequence 231 is QNOW, A5, Q2...Q4, A2, excluding Q1, QN, A1 and AN.
[0027] In one embodiment, when the sorting module 130 determines that the similarity between the current question (QNow) and a historical question (e.g., Q2) is higher, and when the sorting module 130 determines that the similarity between the current question (QNow) and a historical response (e.g., A5) is higher, then in the output data structure, the historical question (e.g., Q2) and the historical response (e.g., A5) are arranged closer to the current question (QNow). When the sorting module 130 determines that the similarity between the current question (QNow) and a historical question (e.g., Q4) is lower, and when the sorting module 130 determines that the similarity between the current question (QNow) and a historical response (e.g., A2) is lower, then in the output data structure, the historical question (e.g., Q4) and the historical response (e.g., A2) are arranged further away from the current question (QNow). Then, the sorting module 130 outputs the sorting result from the output data structure to the prompt output order 231 to transmit it to the external LLM 150. At this point, the data in the output sequence 231 is QNOW, A5, Q2...Q4, A2, where A5 and Q2 are closer to QNOW, and Q4 and A2 are farther away from QNOW.
[0028] Please refer to FIG4, which illustrates a flowchart of a context management method for an edge artificial intelligence system 100 according to an embodiment of the present invention. In the embodiment shown in FIG4, the mixer module 110 receives a user question 201 and adds auxiliary information 202 to the user question 201 to generate a current question, and stores the current question in the dialogue history record 211 (step S401); the classifier module 120 receives the dialogue history record 211 and uses a small LLM 140 to classify the dialogue history record 211 into at least one topic category 221, and determines the topic category 221 to which the current question belongs (step S402); and the sorter module 130 sorts at least one historical question and at least one historical response in the topic category 221 to which the current question belongs according to the similarity with the current question, so as to generate a prompt output order 231 for the topic category 221 to which the current question belongs, and transmits the prompt output order 231 to the external LLM 150 (step S403).
[0029] In step S401, the auxiliary information 202 may be clock information 203, global positioning system information 204, or sequence identifier. The dialogue history record 211 includes the current question, historical questions, and historical responses.
[0030] In step S402, the classifier module 120 includes a word segmenter and a small LLM 140. The word segmenter splits the dialogue history record 211 into multiple tokens for processing by the small LLM 140. The small LLM 140 includes an embedding layer, a transformer layer, and a classification head layer. The embedding layer converts the multiple tokens of the current question in the dialogue history record 211 into embedding vectors; the transformer layer processes the embedding vectors to generate a contextual semantic representation of the current question; and the classification head layer determines the topic category 221 to which the current question belongs based on the contextual semantic representation of the current question.
[0031] In step S403, the sorting module 130 obtains the similarity between the current question and historical questions and historical responses based on the vector distance between the current question and historical responses, to determine the similarity between the current question and historical questions and historical responses. If the sorting module 130 determines that the similarity between the current question and historical questions and historical responses is 100% or substantially the same, then the historical questions and historical responses are not arranged in the prompt output order 231. If the sorting module 130 determines that the similarity between the current question and historical questions and historical responses is higher, then the historical questions and historical responses are arranged closer to the current question in the prompt output order 231. If the sorting module 130 determines that the similarity between the current question and historical questions and historical responses is lower, then the historical questions and historical responses are arranged further away from the current question in the prompt output order 231.
[0032] Although the present invention has been disclosed above by way of embodiments, it is not intended to limit the present invention. Anyone skilled in the art can make some modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims. [Simplified Explanation of the Diagram]
[0033] Figure 1 is a block diagram of a first embodiment of an edge artificial intelligence system according to an embodiment of the present invention. Figure 2 is a schematic diagram of a second embodiment of an edge artificial intelligence system according to an embodiment of the present invention. Figure 3 is a schematic diagram of a sorting method of a sorter module in an edge artificial intelligence system according to an embodiment of the present invention. Figure 4 is a flowchart of a context management method in an edge artificial intelligence system according to an embodiment of the present invention.
Claims
1. An edge artificial intelligence system, comprising: a mixer module, for receiving a user question and attaching auxiliary information to the user question to generate a current question and storing the current question in a dialogue history record; a classifier module, for receiving the dialogue history record and classifying the dialogue history record into at least one topic category using a small LLM, and determining the topic category to which the current question belongs; and a sorter module, for sorting at least one historical question and at least one historical response in the topic category to which the current question belongs based on similarity to one of the current questions, to generate a prompt output order for the topic category to which the current question belongs, and transmitting the prompt output order to an external LLM.
2. The edge AI system as described in claim 1, wherein the edge AI system can be applied to an edge AI device.
3. The edge AI system as described in claim 2, wherein the dialogue history is stored in a memory of the edge AI device.
4. The edge artificial intelligence system as described in claim 3, wherein the auxiliary information is a clock information, a GPS information, or a sequence identifier.
5. The edge AI system as described in claim 4, wherein the classifier module includes a word segmenter for splitting the dialogue history record into multiple tokens for processing by the small LLM.
6. The edge AI system as described in claim 5, wherein the small LLM comprises an embedding layer, a transformer layer and a classification head layer, wherein the embedding layer is used to convert the plurality of words of the current question in the dialogue history into an embedding vector; the transformer layer is used to process the embedding vector to generate a contextual semantic representation of the current question; and the classification head layer is used to determine the topic category to which the current question belongs based on the contextual semantic representation of the current question.
7. An edge AI system as described in claim 6, wherein the sorting module obtains the similarity based on a vector distance between the current question and the historical question and between the current question and the historical response.
8. The edge AI system as described in claim 7, wherein if the sorting module determines that the similarity between the current question and the historical question and the current question and the historical response is 100% or substantially the same, then the historical question and the historical response are not arranged in the prompt output order.
9. The edge AI system as described in claim 7, wherein the sorting module determines that the higher the similarity between the current question and the historical question and the current question and the historical response, the closer the historical question and the historical response are to the current question in the prompt output order.
10. A context management method for an edge artificial intelligence system, comprising: (a) receiving a user question by a mixer module and attaching auxiliary information to the user question to generate a current question, and storing the current question in a dialogue history record; (b) receiving the dialogue history record by a classifier module and classifying the dialogue history record into at least one topic category using a small LLM, and determining the topic category to which the current question belongs; and (c) sorting at least one historical question and at least one historical response in the topic category to which the current question belongs based on similarity to one of the current questions, to generate a prompt output order for the topic category to which the current question belongs, and transmitting the prompt output order to an external LLM.