Apparatus and method for selecting fully localized domain-specific language models

Abstract

This invention provides a fully local domain-specific language model selection device that enables the fully local implementation of dialogue systems with practical response quality across multiple domains, even under limited computing resources and power conditions. [Solution] The device comprises: an input acquisition means for receiving natural language input from a user; an intent estimation means for classifying the field or intent to which the input belongs into one of a plurality of field labels based on the acquired natural language input and, if necessary, state tags obtained from sensors; a model storage means for storing a plurality of field-specific language models; a routing means for selecting at least one target model from the field-specific language models based on the field labels obtained by the intent estimation means and the amount of computational resources available in the device, and inputting the natural language input to the target model; and an output means for presenting the response sentence generated by the target model to the user as text display and / or audio.

Description

Technical Field

[0001] The present invention relates to a dialogue system using natural language, and particularly to a plurality of domain-specific language models (small LLMs) that operate only on the terminal side without using the cloud, and a complete local Chatbot platform equipped with a routing mechanism for estimating the intention of user input and distributing it to each domain-specific model.

Background Art

[0002] In recent years, in the field of natural language processing, dialogue systems using large language models (LLMs) have been widely put into practical use. These large language models are composed of neural networks with a parameter scale ranging from billions to trillions, and generally perform inference processing on a cloud environment equipped with high-performance GPU servers. Input from a user terminal is transmitted to the cloud side via a network, inference is performed by the large language model on the cloud side, and the result is returned to the terminal, providing a so-called cloud-based chatbot service. On the other hand, with the increasing need for AI utilization on edge devices and embedded devices, technologies for performing natural language processing on the device using small language models with reduced parameter numbers or quantization models that reduce the model size and computational amount by converting weights to low bit widths have also been proposed. In such technologies, a configuration for performing simple dialogue functions and command interpretation using a single small language model is known even in limited computational resources such as mobile terminals and single-board computers. Also, as a technology for combining multiple models and expert networks, a Mixture-of-Experts type neural network structure that selectively activates multiple internal experts according to the input has been proposed. In such a structure, a gating network determines the weighting of experts based on input feature amounts, and only the selected experts are used for calculation, aiming at efficient utilization of computational resources. Furthermore, for enterprise cloud services, a "switchboard platform" has been proposed that selects the appropriate model from among multiple infrastructure models according to cost, performance, security policies, etc., and routes requests accordingly. Routing frameworks have also been proposed that select one of several large-scale language models based on application, user attributes, ethical constraints, etc. Furthermore, in edge computing environments, methods are being explored to optimize the balance between latency, power consumption, and response quality by coordinating small on-device models with large cloud models and switching between the two models on a token-by-token or query-by-query basis. In this type of approach, lightweight inference is typically performed on the on-device side, and processing is offloaded to the large cloud model as needed. However, general-purpose chatbot platforms that deploy multiple domain-specific, distilled small language models in parallel within a single terminal and select and route to the appropriate model based on user input intent have not yet been fully disclosed. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Patent Document 1: Switchboard for routing multiple foundation models EP 4 647 941 A1 Title: SWITCHBOARD PLATFORM FOR FOUNDATION MODELS Patent Document 2: Mixture-of-Experts (MoE) neural networks US 10,719,761 B2 / JP6853354B2 / WO2018 085643 etc. Title: Mixture of experts neural networks [Non-patent literature]

[0004] [Non-Patent Document 1] Non-Patent Literature 1: Mixture-of-Agents (MoA) Junlin Wang et al., “Mixture-of-Agents Enhances Large Language Model Capabilities”, arXiv:2406.04692 (2024) Non-Patent Literature 2: Token Level Routing Inference System for Edge Devices ACL 2025 Demo, “Token Level Routing Inference System for Edge Devices” Non-Patent Literature 2: Token Level Routing Inference System for Edge Devices ACL 2025 Demo, “Token Level Routing Inference System for Edge Devices” [Overview of the project] [Problems that the invention aims to solve]

[0005] In recent years, dialogue systems using large-scale language models (LLMs) have become widespread. However, most of these systems run models with billions to hundreds of billions of parameters on GPU servers in the cloud, resulting in significant latency and usage costs due to network communication. Furthermore, since user input and dialogue history are sent to the cloud, their use is restricted from the perspective of privacy protection and confidential information. To address this issue, technologies have been developed to run smaller LLMs or quantized models with reduced parameter counts on a single edge device. However, when a single model incorporates knowledge and conversational skills across multiple fields such as general conversation, monitoring, pet consultation, construction / infrastructure, and security, the response quality in each field tends to be insufficient due to model size constraints, and the trade-off between performance and computational resources remains significant. In addition, conventional technologies that combine multiple models or expert networks, and switchboard-like mechanisms that select and route the appropriate model from multiple base models in a cloud environment have been proposed, but most of these rely on large-scale computational resources on the cloud or data center side. A general-purpose chatbot platform that can house multiple domain-specific miniature LLMs (Low-Level Models) in parallel on a single edge device, distilling knowledge from large-scale LLMs (Low-Level Models) that serve as training, and then select and execute the appropriate model entirely locally based on the user's intent, has not yet been fully realized. Furthermore, prior inventions by the applicant, such as a fully local dog emotion estimation device, a monitoring system, and a crime prevention behavior analysis device, propose a configuration in which multimodal information acquired from cameras and microphones is analyzed by edge AI, and the results are input into a single generative AI model to perform natural language responses and voice output. However, these are individual devices optimized for specific applications, and a higher-level foundational technology as a "fully local chatbot platform" consisting of a group of domain-specific, small-scale LLMs and an intent-based routing mechanism that can be used in common across multiple fields has not yet been established. Therefore, the problem that the present invention aims to solve is to provide a chatbot platform that can realize a fully local conversational system with practical response quality across multiple fields, even under limited computing resources and power conditions, by arranging multiple domain-specific, distilled, small-scale LLMs in parallel on a single edge device without relying on large-scale LLMs in the cloud, and selecting and executing an appropriate model from among them based on the intent of the user input. [Means for solving the problem]

[0006] To solve the problem, a fully local domain-specific language model selection device according to one aspect of the present invention includes: an input acquisition means for receiving natural language input from a user; an intent estimation means for classifying the field or intent to which the input belongs into one of a plurality of domain labels based on the natural language input acquired by the input acquisition means and, if necessary, state tags obtained from sensors; a model storage means for storing a plurality of domain-specific language models; a routing means for selecting at least one target model from the domain-specific language models based on the domain labels obtained by the intent estimation means and the amount of computational resources available in the device, and inputting the natural language input to the target model; and an output means for presenting the response sentence generated by the target model to the user as text display and / or audio. The domain-specific language models stored in the model storage means may be distilled miniature language models obtained by training a small base language model using a parameter-efficient fine-tuning method, such as LoRA (Low-Rank Adaptation), based on training data consisting of prompt-response pairs generated by inputting representative prompts for each domain to a large-scale language model that serves as a teacher and obtaining the responses. Furthermore, it is preferable that these domain-specific language models are implemented as quantized models with reduced parameter bit widths through quantization, etc., and stored in the storage within the device. The intent estimation means is configured using a lightweight classification model or a language model with a small number of parameters, and by using words, context, and state tags included in the user's natural language input as features, it can estimate one of several domain labels such as "pet consultation," "monitoring / health," "construction / infrastructure," "crime prevention / safety," and "general small talk." The routing means selects at least one target model from among a plurality of domain-specific language models stored in the model storage means, based on the domain label estimated by the intent estimation means and resource information such as CPU load, memory usage, and inference execution history in the device. The routing means may select a single target model and perform inference using that model, or it may be configured to select two or more domain-specific language models in parallel to obtain multiple candidate responses, and then determine the final response using a response integration means described later. In one embodiment of the present invention, the apparatus may further include response integration means for selecting the optimal response from a plurality of candidate responses obtained from one or more domain-specific language models selected by the routing means, based on the degree of relevance to the intent label, response length, predefined evaluation metrics, etc., or for integrating the plurality of candidate responses to generate a single final response sentence. Another aspect of the present invention relates to a fully local domain-specific language model selection method which includes an input acquisition step of acquiring natural language input from a user; an intent estimation step of classifying the input into one of a plurality of domain labels based on the natural language input and state tags; a model selection step of selecting a target model from a plurality of domain-specific language models based on the domain labels and the computing resource status of the device; an inference step of inputting the natural language input as a prompt to the target model and causing it to generate a response sentence; and a response output step of presenting the response sentence to the user, characterized in that the inference processing by the plurality of domain-specific language models is completed within the device without going through a cloud server. Furthermore, a program according to another aspect of the present invention, when executed by a computer, causes the computer to perform the input acquisition step, intent estimation step, model selection step, inference step, and response output step, and can be provided by being stored on a storage medium mounted on an edge device, or distributed via a network. It should be noted that the present invention is not limited to the above embodiments, and those skilled in the art can make various changes, modifications, and extensions to the types of intent labels, the number and configuration of domain-specific language models, the distillation method and quantization method, and the specific algorithms of the response integration means. These are also included in the scope of the present invention as long as they do not depart from the spirit of the invention. [Effects of the Invention]

[0007] According to the present invention, compared to conventional configurations that pack knowledge and dialogue functions for all domains into a single compact language model, multiple domain-specific compact language models are used, each with knowledge distilled from a large-scale trained language model. This allows for higher expertise and response quality in each domain while reducing the number of parameters in each model. As a result, the trade-off between model size and performance is mitigated, enabling practical dialogue performance even on edge devices with limited computing resources. Furthermore, the device according to the present invention includes a routing mechanism that estimates the intent of user input based on the input and state tags, and selects and executes a domain-specific compact language model according to the result. This means that, unlike conventional methods that always use a single general-purpose model, only the model for the most appropriate domain needs to be activated for each input, and unnecessary calculations are avoided, thereby reducing inference time and power consumption. In particular, even in situations where consultations from multiple domains are mixed, the combination of a lightweight router and a group of domain-specific models enables efficient dialogue processing. Furthermore, in this invention, a series of inference processes, including multiple domain-specific miniature language models and a routing mechanism, are completed entirely locally within a single edge device without relying on a cloud server. Therefore, compared to conventional conversational systems that depend on large-scale cloud-based language models, user input and inference results are not transmitted externally, offering significant advantages in terms of privacy protection and confidentiality. In addition, conversational functionality can be maintained even in situations where cloud connectivity is limited, such as in environments with unstable networks, during disasters, or within closed networks. In addition, the domain-specific miniature language model in this invention is pre-trained by knowledge distillation from a large-scale language model used as training material, and further lightweighting techniques such as quantization are applied to configure it to fit into the storage and memory of an edge device. Therefore, unlike conventional cloud + on-device collaborative methods, there is no need to offload critical processing to the cloud, and a highly functional chatbot can be configured that is completed entirely on the device. Furthermore, the present invention can function as a common platform for the generation AI portion used in the fully local emotion estimation device, monitoring system, and crime prevention behavior analysis device previously proposed by the applicant. That is, by using the field-specific small language model group and intention-based routing mechanism of the present invention, natural language response functions for different applications, such as dog emotion estimation, monitoring dialogue after fall detection, and warning after crime prevention behavior detection, can be integrated on a single chatbot platform. This eliminates the need to individually design and implement the generation AI portion for each application device, improving the overall development efficiency and maintainability of the system. Furthermore, domain-specific miniature language models are easy to add and update. By creating miniature models corresponding to new domains through distillation, registering them in the model storage means, and adding domain labels to the intent estimation means, the scope of application of the chatbot platform can be gradually expanded. Thus, the present invention offers advantages over conventional technologies in that it provides high scalability and reusability for fully local dialogue systems on edge devices. [Brief explanation of the drawing]

[0008] [Figure 1] Prototype Configuration Diagram: 1. Camera 2. Microphone 3. AI Assembler 4. Computing (Raspberry Pi 5) 5. Bluetooth Speaker 6. 3.5-inch Panel Display [Figure 2] Block diagram of the overall configuration of the apparatus of the present invention [Figure 3] Process flowchart of the present invention [Figure 4] Schematic diagram of domain-specific language models and distillation learning procedures [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described in detail below with reference to the drawings. However, the present invention is not limited to these embodiments, and those skilled in the art can implement it with various changes and modifications. (1) Example of hardware configuration Figure 1 is a prototype configuration diagram. Figure 2 is a block diagram showing an example of the overall configuration of the fully local domain-specific language model selection device (hereinafter referred to as "the device") according to the present invention. The device 1 consists of an arithmetic unit 11 consisting of a single-board computer, and various input / output devices and storage devices connected to the arithmetic unit 11. As the arithmetic unit 11, an edge device equipped with a CPU and a GPU represented by Raspberry Pi 5 can be used. Additionally, if necessary, an AI arithmetic unit 12 consisting of an NPU or an external AI accelerator (such as the Hailo series) can be connected, and a part of the domain-specific language model can be executed by the AI arithmetic unit 12. The device 1 includes an input interface 13 for receiving input from the user, such as a keyboard, mouse, touch panel, microphone, etc. Also, as an output interface 14 for presenting the interaction result, it includes a display device (such as a liquid crystal display) and a speaker. The input interface 13 and the output interface 14 are connected to the arithmetic unit 11 either wired or wirelessly. Furthermore, the device 1 includes a storage 15 for storing a plurality of domain-specific language models and routing programs, etc. As the storage 15, a semiconductor memory, SSD, eMMC, etc. can be used. The device 1 may be configured to be portable by including a power supply device 16 such as a mobile battery. (2) Functional block configuration As shown in FIG. 2, the arithmetic unit 11 realizes at least the following functional blocks by software. Input acquisition means (101) Intention estimation means (102) Routing means (103) Domain-specific language model group (104, 104a, 104b, 104c, 104n) Response integration means (105) (optional) Output means (106) The input acquisition means 101 acquires text input or voice input from the input interface 13. If voice input is provided, the local speech recognition engine converts the voice to text. In addition, if the device 1 is connected to a camera or various sensors, it also acquires status information (such as a fall detection flag or a pet detection flag) obtained from them as status tags. The intent estimation means 102 is a functional block that estimates the field or intent to which the input belongs based on the text and state tags obtained by the input acquisition means 101. The intent estimation means 102 is configured using a lightweight classification model or a language model with a small number of parameters. Using the vector representation of the input sentence and the state tags as features, it outputs one of the field labels such as "pet consultation," "monitoring / health," "construction / infrastructure," "crime prevention / safety," or "general small talk." The routing means 103 selects a target model to be used for inference from the domain-specific language model group 104, 104a, 104b, 104c, and 104n stored in the model storage means, based on the domain label output from the intent estimation means 102 and resource information such as CPU usage and memory usage. The routing means 103 may select a single domain-specific language model, or it may be configured to select two or more models simultaneously and perform inference in parallel. The domain-specific language model groups 104, 104a, 104b, 104c, and 104n consist of small language models trained to specialize in different domains. For example, model 104 is specialized for consultations about dogs and pets, model 104a is specialized for monitoring and health consultations, model 104b is specialized for construction and infrastructure-related matters, model 104c is specialized for crime prevention and safety, and model 104n is a general-purpose model that can handle general conversation and a wide range of questions. As will be described later, each of these models is a distilled small language model created by knowledge distillation from a large-scale trained language model, and has undergone lightweight processing such as quantization. The response integration means 105 is a functional block that evaluates candidate responses obtained from each model when multiple domain-specific language models are selected by the routing means 103, and ultimately determines a single response sentence. The response integration means 105 selects candidates based on factors such as the degree of relevance to the domain label, the length of the response, and a predefined scoring function, or it integrates multiple candidates to generate a single response sentence. The response integration means 105 may be omitted when using a configuration with only one model. The output means 106 is a functional block that displays the response sentence output from the response integration means 105 or the domain-specific language model group 104, 104a, 104b, 104c, 104n as text on a display device, and, if necessary, converts it into speech using a local speech synthesis engine and outputs it from a speaker. (3) Processing procedure Figure 3 is a flowchart showing the operation procedure of the fully local domain-specific language model selection device according to the present invention. In step S101, the input acquisition means 101 acquires natural language input from the user. If it is voice input, it is converted to text by local speech recognition. In addition, if necessary, status tags indicating state information from sensors are also acquired. In step S102, the intent estimation means 102 estimates the domain label to which the input belongs based on the natural language input and state tags. Here, the intent estimation means 102 may be configured to convert the input sentence into an embedding vector and input it into a lightweight classifier, or it may be configured to determine the domain label using the output of a small language model. In step S103, the routing means 103 selects a target model to be used for inference from the domain-specific language model group 104, 104a, 104b, 104c, and 104n based on the domain label obtained from the intent estimation means 102, CPU usage, memory usage, past inference time, etc. The target model is based on the model associated with the domain label, but control may be performed to select an alternative model depending on the load situation. In step S104, the selected domain-specific language model receives a prompt containing natural language input and generates a response sentence. If multiple models are selected, each model generates a response sentence independently. In step S105, if there are multiple candidate responses, the response integration means 105 either selects the optimal one from among them or integrates multiple candidates to create a single final response. If only one model is used, the response of that model becomes the final response. In step S106, the output means 106 displays the final response as text on the display device and, if necessary, performs speech synthesis and outputs it from the speaker. The above process is repeated at predetermined intervals as long as the interaction with the user continues. (4) Training (distillation) procedure for domain-specific language models Figure 4 is a block diagram illustrating the domain-specific language model group and distillation learning procedure according to the present invention. First, we prepare 200 large-scale language models (general-purpose language models with billions to hundreds of billions of parameters) as training models. These training models are deployed either in the cloud or in an offline environment. The distilled data generation unit 201 inputs a set of prompts designed for each domain into the training model 200 and obtains its responses. The prompt-response pairs are used as training data that reflects the training model's knowledge and dialogue style. The obtained prompt-response pairs may be saved as files in JSONL format. The learning unit trains a small-scale base model 202 (for example, a general-purpose language model with approximately 1 billion parameters, or a model with approximately 700 million parameters) using the JSONL format training data and a parameter-efficient fine-tuning method. Specifically, it uses LoRA based on the PEFT framework to add low-rank matrices to some parameters of the small-scale base model to acquire domain-specific behavior. The resulting domain-specific miniature language models 203a, 203b, 203c, and 203n are distilled models specialized for specific fields (such as dog emotions, monitoring, construction, crime prevention, and general conversation). Each model's parameters are converted to a low bit width through quantization processing and then stored in storage 15, where they are used as the domain-specific language model group 104, 104a, 104b, 104c, and 104n of this device 1. The training of the domain-specific language model may be performed in advance in a high-performance computing environment separate from this device 1, and only the resulting model file may be distributed to the storage 15 of this device 1. Alternatively, a feedback learning mechanism may be provided, and a portion of the user interaction log may be used for retraining in an anonymized and summarized form to continuously improve the performance of the domain-specific model. (5) Examples of applications and other embodiments The fully local domain-specific language model selection device according to the present invention can be used not only as a standalone chatbot, but can also be incorporated as a common platform for the generation AI portion of prior inventions by the applicant, such as the fully local emotion estimation device, the fully local monitoring system, and the fully local crime prevention behavior analysis device. For example, in the dog emotion estimation device, the emotion estimation result (anxiety, happiness, etc.) can be passed as additional context to the domain-specific language model, and the chatbot platform of the present invention can provide natural language explanations and care method suggestions to the owner. In the monitoring system, when events such as fall detection or wandering detection occur, the intent estimation means 102 of this device 1 selects the "monitoring / health" category label and uses a category-specific language model specialized for that category to generate verbal communication with the person being cared for or notification messages for the caregiver. Similarly, in the crime prevention behavior analysis device, when suspicious behavior is detected, a model in the "crime prevention / safety" category is selected and warning messages and response guidelines are generated. The device of the present invention can flexibly change the number and size of domain-specific language models depending on the type and performance of the edge device. Devices with ample computing resources can have a large number of domain-specific models permanently installed, while devices with limited resources can selectively install only models corresponding to high-priority domains. In the above embodiment, an example was shown in which intent estimation and model selection are separated into two stages. However, these can be integrated, and a single small router model can directly output the "identifier of the domain-specific language model to be used." Furthermore, various modifications can be made to the types and hierarchical structures of domain labels, evaluation metrics, and routing algorithms depending on the application, and these are also included within the scope of the present invention. As described above, the present invention enables the realization of a multi-domain, high-performance dialogue system even with limited computing resources by arranging multiple domain-specific, distilled, compact language models in parallel on a single edge device and selecting and executing the appropriate model from among these models entirely locally based on the intent of the user input. [Examples]

[0010] The following describes a specific embodiment of the fully local domain-specific language model selection device according to the present invention. The description is an example and is not intended to limit the technical scope of the present invention. Example 1: A fully local chatbot using a Raspberry Pi 5 and a small LLM array. (1) Hardware configuration The apparatus in this embodiment uses a Raspberry Pi 5 (16GB main memory) as the computing unit. A Hailo-8L is connected as the AI ​​computing unit, and the configuration allows some preprocessing and simplified models to be executed on the AI ​​accelerator as needed. The input interfaces include a USB microphone, keyboard, and touchscreen display. Output interfaces include a display and a Bluetooth speaker. It stores multiple domain-specific language model files and routing programs (described later). A mobile battery is used for power, making it portable for field use. (2) Configuration of domain-specific language models In this example, we prepare the following five domain-specific miniature language models. 1) Pet / Dog Emotional Counseling Model (Model A) 2) Monitoring and health consultation model (Model B) 3) Construction and Infrastructure Consultation Model (Model C) 4) Crime prevention and safety consultation model (Model D) 5) General Conversation / General Consultation Model (Model E) As the foundation for each model, a small language model with approximately 700 million parameters is employed. As a training model, a large-scale general-purpose language model with billions of parameters is prepared on a workstation in a separate environment, and prompt-response pairs are generated as distilled data for each domain. For the pet / dog emotion consultation model A, numerous example questions from pet owners ("My dog ​​is trembling," "My dog ​​won't eat," etc.) are prepared, and simple explanations and advice in Japanese are obtained from the training model and used as training data in JSONL format. Similarly, for the monitoring / health consultation model B, example conversations expected in elderly monitoring settings are prepared, for the construction / infrastructure consultation model C, Q&A related to construction sites and infrastructure maintenance is prepared, for the crime prevention / safety consultation model D, example dialogues related to dealing with suspicious persons and evacuation actions is prepared, and for the general small talk model E, data on everyday conversations and small talk is prepared, and responses are obtained from the training model. Training of the small-scale base model will be performed using the PEFT framework, and adapter weights for each domain will be learned using LoRA. After training is complete, the domain-specific adapters will be integrated into the base model, or the adapters will be saved as individual files. Subsequently, each model will be converted to an integer low-bit width representation using a quantization tool, and then converted to a format executable on Raspberry Pi 5 using llama.cpp, etc. (such as GGUF format). After quantization, each model size is approximately several hundred MB, allowing the five domain-specific models and the lightweight router model to be stored on an SSD. This enables a configuration where all domain-specific compact language models can reside in a single Raspberry Pi 5 environment. (3) Intent estimation and routing processing This embodiment uses a small classification model as a means of estimating intent. The input sentence is fed into a pre-trained encoder for Japanese, and a softmax output is obtained that classifies it into five classes: "pet," "monitoring," "construction," "crime prevention," and "general." The number of parameters in the classification model can range from several million to tens of millions, and it can also be configured to run on Hailo-8L. The routing mechanism determines which domain-specific language model to use by referring to the domain label obtained as an intent estimation result, along with the CPU usage and memory usage of the Raspberry Pi 5. For example, if the intent estimation result is "pet" and other models are idle, only model A is launched. If the intent estimation result spans multiple domains near the decision boundary, models A and E are launched simultaneously, response candidates from each are obtained, and the response integration mechanism selects the final response. The routing mechanism also references user-specific configuration information. For example, a profile can be set for a particular user, such as "prioritize inquiries about construction" or "prefer concise answers to pet-related inquiries," and the model's priority and response length can be controlled accordingly. This allows devices with the same hardware configuration to be operated as customized conversational terminals according to their purpose and user. (4) Fully local dialogue flow When a user speaks into the microphone, the voice input is converted into text by the device's speech recognition engine. The speech recognition result, along with status tags supplied from monitoring systems, security cameras, etc., as needed, are input into the intent estimation means. The intent estimation method outputs a domain label based on the input sentence and state tags. For example, if the input sentence is something like "My dog ​​has been lethargic since yesterday," the domain label selected will be "pets." The routing means selects domain-specific language model A, which corresponds to the label "pet," as the target model and inputs a prompt to model A. The prompt can include supplementary information such as the user utterance text, the dog's emotion label estimated by the device (output from the prior art ROCAL1), and the living environment. Model A operates on a Raspberry Pi 5 as a quantized miniature language model and generates response sentences in Japanese to user questions. If the response integration means is a single model configuration, the generated response sentence is adopted as the final response. The output means displays the response sentence on a display and generates speech using local speech synthesis, which is output through a Bluetooth speaker. Even when the network is disconnected, the entire series of processes described above are completed within the Raspberry Pi 5 and the connected device. The large-scale teacher language model is used only during the training phase, and there is no process to call the large-scale model on the cloud side while the device in this embodiment is running. (5) Examples of collaboration with monitoring and crime prevention As an example of integration with a monitoring system, consider the case where a fall detection module operating in a separate process sends a "fall detection flag = true" to this device. Immediately after the fall detection module issues a status tag, this device sets a high priority for the "monitoring / health" field in its intent estimation means and controls the system to always use monitoring / health consultation model B as the initial communication with the user. As an example of integration with a security behavior analysis device, when suspicious behavior is detected, a status tag is sent to this device. The intent estimation means includes this status tag in its input features, thereby outputting a "security / safety" category label with high probability. The routing means generates a warning message using the security / safety consultation model D to alert on-site workers. Thus, the device of this embodiment can be used not only as a standalone chatbot, but also as a general-purpose platform that provides a common dialogue interface to existing fully local AI devices (e.g., emotion estimation, monitoring, security). The configurations, number of fields, model size, quantization method, routing algorithm, etc., described in the above embodiments are examples only and can be appropriately modified depending on the application, hardware performance, and target field. For example, configurations that increase or decrease the number of fields, configurations that use a different type of small-scale base model, or configurations that perform model inference using only an AI accelerator are also included within the technical scope of the present invention. [Industrial applicability]

[0011] The fully localized, domain-specific language model selection device and method according to the present invention can be used in a wide range of industrial fields as an interactive information provision platform that does not depend on the cloud. In particular, in fields where personal information protection and real-time performance are important, it is possible to realize chatbot functions that support multiple fields on a small edge device, making it highly likely to be used in information services, software contract development, edge AI equipment manufacturing, construction, welfare and nursing care, security and disaster prevention, and pet-related services. In the construction and infrastructure sector, the device of this invention can be integrated into on-site terminals to provide fully localized answers to inquiries regarding construction procedures, quality control, safety measures, and equipment specifications, even in on-site environments with unstable network connectivity. By linking with on-site photographs and status tags from sensors, it can contribute to improving the productivity of construction management as an interactive support terminal combined with structural diagnostic devices and differential analysis devices. In the welfare and caregiving fields, the device of the present invention can be incorporated into monitoring systems and home care terminals to provide services such as verbal guidance after fall detection, reminders for medication and hydration, and responses to lifestyle consultations, without relying on communication infrastructure. By introducing specialized models tailored to individual needs for care facilities and elderly people living at home, it is possible to reduce the burden on caregivers and improve the sense of security for those receiving care. In the crime prevention and disaster prevention fields, it can be used as an interactive warning terminal when suspicious behavior is detected, in conjunction with security cameras and behavioral analysis devices. When suspicious behavior is detected, it uses a specialized model for the crime prevention and safety field to provide real-time warnings about dangerous behavior, evacuation guidance, and instructions on reporting procedures, which can be used to maintain public order and prevent accidents. Because it operates completely locally even when communication is cut off or during disasters, it is suitable for crisis management purposes. In the pet-related industry, this technology can be used as an interactive advice terminal for pet owners by combining it with dog emotion estimation devices and pet monitoring cameras. The emotion estimation results are input as context into the chatbot platform of this invention, enabling services that provide natural conversations with pet owners regarding training methods, health management, stress reduction measures, and more. It can also be installed as a consultation terminal in pet shops and veterinary hospitals. The production method of this invention first generates distillation data using a large-scale trained language model in a cloud or on-premise environment, and performs parameter-efficient fine-tuning learning on a small-scale base model, thereby generating a group of domain-specific small-scale language models as the output. Next, each model is quantized and converted into a format for edge devices, and distributed in the form of firmware, container images, or application packages. Equipment manufacturers and system integrators can combine models and configure routing according to the target field and assemble them as terminals for field use. In terms of usage, the end user inputs questions or inquiries to the device of the present invention via voice or text, and the intent estimation means and routing means automatically select a domain-specific language model, presenting the response generated within the terminal via screen display and audio output. System administrators can keep up with changes in target business or service content by checking log information and adding or replacing domain-specific models as needed. The present invention can be provided in various forms, including as a finished terminal for edge AI devices, as an embedded module to be incorporated into existing monitoring and security devices, and as a software platform licensed to other companies' devices. Therefore, the industrial applicability of the present invention is high. [Explanation of Symbols]

[0012] Prototype Configuration Diagram: 1. Camera 2. Microphone 3. AI Assembler 4. Computing (Raspberry Pi 5) 5. Bluetooth Speaker 6. 3.5-inch Panel Display

[0013] In this specification, the names "Raspberry Pi," "Hailo," "Gemma," "Ollama," "Python," "TensorFlow," "TensorFlow Lite," "scikit-learn," "GStreamer," "YOLO," "Bluetooth," "PEFT," "LoRA," "GGUF," and "llama.cpp" are registered trademarks or trademarks of their respective companies unless otherwise specified. The inclusion of these names is for illustrative purposes only and does not imply that the applicant has received any endorsement, warranty, or affiliation from the respective rights holders.

Claims

1. A fully local domain-specific language model selection device that engages in dialogue in response to natural language input from a user, (A) Input means for acquiring the natural language input, (B) Based on the words and context included in the natural language input, Classify the field or intent to which the input belongs into one of several field labels. Means for estimating intent, (C) Each person has studied in a specialized field. A model storage means for storing multiple domain-specific language models, (D) The field label obtained by the intent estimation means, Based on the amount of computing resources available in the device, Select at least one target model from the aforementioned domain-specific language models, The aforementioned natural language input is input to the target model. Routing methods, (E) The response statement generated by the target model is Present to the user as text display and / or audio output. Output means and Equipped with, The inference process of the aforementioned domain-specific language model, Without using a cloud server or external network The device is characterized by being self-contained within the device itself. A fully localized, domain-specific language model selection device.

2. In the fully localized domain-specific language model selection device according to claim 1, Each domain-specific language model stored in the aforementioned model storage means is: (i) For a large-scale language model that will serve as a teacher, Enter typical prompts for each field, The process of obtaining teacher responses, (ii) Pairing the prompt with the teacher's response The process of generating training data, (iii) For a small base language model, Based on the aforementioned training data The process of training using a parameter-efficient fine-tuning method, It is characterized by being obtained through distillation learning that includes A fully localized, domain-specific language model selection device.

3. A fully local domain-specific language model selection method that performs dialogue in response to natural language input from a user, (a) A process of obtaining natural language input from the user, (b) An intent estimation step of classifying the natural language input into one of a plurality of domain labels, (c) Based on the field label and the computing resource status of the device, The process of selecting a target model from among multiple domain-specific language models, (d) A step of inputting the natural language input as a prompt to the target model and generating a response sentence, (e) The process of presenting the generated response to the user, Includes, The aforementioned domain-specific language model, Pre-trained through knowledge distillation from large-scale language models that serve as teachers, The generation of the aforementioned response statement does not go through the cloud server. The device is characterized by being self-contained within the device itself. A method for selecting a completely localized, domain-specific language model.

Images