Method for providing user query answers using manufacturing specific complex language models

Abstract

A method for providing a user query response using a manufacturing-specific composite language model according to an embodiment of the present invention comprises: a) a first language model building unit building a manufacturing-specific base model including a first small language model and a second small language model, a second language model building unit building a plurality of specialized models for each company and manufacturing area through fine-tuning of the manufacturing-specific base model using specialized data for each company and manufacturing area, and a third language model building unit building a manufacturing-specific composite language model for providing a response to a user query; b) a user inputting a user query related to manufacturing into a routing manager; c) the routing manager inferring a company and manufacturing area from the user query using the first small language model and selecting a specialized model for each company and manufacturing area specialized among the plurality of specialized models for each company and manufacturing area that is specialized for the inferred company and manufacturing area; d) after the manufacturing-specific base model and the specialized model for each company and manufacturing area generate a response to the user query, converting the meaning of the text constituting the response to the user query into a numeric vector through text embedding; and e) a step in which the manufacturing-specialized complex language model merges the numeric vector of the manufacturing-specialized base model and the numeric vector of the enterprise and manufacturing area-specific specialized model to provide an answer to the user query to the user.

Description

Technology Field

[0001] The present invention relates to a method for providing user query answers using a manufacturing-specific complex language model, wherein the manufacturing-specific complex language model can provide answers to user queries related to the user's manufacturing industry. Background Technology

[0002] Recently, Natural Language Processing (NLP) technology has advanced rapidly in the field of artificial intelligence, and pre-trained language models are being applied across various fields using fine-tuning.

[0003] However, existing large-scale language models have problems such as generating unnatural and inaccurate responses because they fail to understand user queries due to a lack of specialization in non-English languages ​​like Korean, or failing to provide professional and detailed answers because they are not specialized for specific domains.

[0004] Furthermore, as the size of pre-trained language models becomes very large, full fine-tuning consumes significant computing resources and time, and there is a risk of losing existing knowledge when fine-tuning existing language models with new data.

[0005] Meanwhile, the manufacturing industry is vast and requires multiple language models for each sector and company; however, fully fine-tuning and managing all of them requires significant computing resources and time. Prior art literature

[0006] Korean Patent Publication No. 10-2674954 (Registered June 10, 2024) The problem to be solved

[0007] Accordingly, the present invention has been devised to solve the above-mentioned problems, and the objective of the present invention is to provide a method for providing answers to user queries using a manufacturing-specialized complex language model, wherein the manufacturing-specialized complex language model is configured in manufacturing areas such as injection molding, welding, forging, pressing, rolling, and precision machining to provide answers to users regarding user queries related to manufacturing industries that produce products.

[0008] Furthermore, the present invention provides a method for providing user query answers using a manufacturing-specialized complex language model that can reduce computing resources of the manufacturing-specialized model and prevent the risk of losing existing knowledge by utilizing at least one Parameter Efficient Fine-tuning (PEFT) technique among Low-Rank Adaptation (LoRA), Quantized LoRA (QLoRA), Adapter, and Prefix Tuning when constructing specialized models for enterprises and manufacturing regions through fine-tuning of the manufacturing-specialized model.

[0009] Furthermore, the present invention provides a method for providing user query answers using a manufacturing-specific complex language model, which can provide answers applicable to the enterprise and manufacturing domains to the user by utilizing answers to user queries generated by specialized models specific to the enterprise and manufacturing domains.

[0010] However, the technical problems to be solved by the present invention are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which the present invention belongs from the description below. means of solving the problem

[0011] A method for providing a user query response using a manufacturing-specific composite language model according to an embodiment of the present invention for achieving the above-mentioned purpose comprises: a) a first language model building unit building a manufacturing-specific base model including a first small language model and a second small language model, a second language model building unit building a plurality of specialized models for each company and manufacturing area through fine-tuning of the manufacturing-specific base model using specialized data for each company and manufacturing area, and a third language model building unit building a manufacturing-specific composite language model for providing a response to a user query; b) a user inputting a user query related to manufacturing into a routing manager; c) the routing manager inferring a company and manufacturing area from the user query using the first small language model and selecting a specialized model for each company and manufacturing area specialized among the plurality of specialized models for each company and manufacturing area that is specialized for the inferred company and manufacturing area; d) after the manufacturing-specific base model and the specialized model for each company and manufacturing area generate a response to the user query, converting the meaning of the text constituting the response to the user query into a numerical vector through text embedding; and e) a step in which the manufacturing-specialized complex language model merges the numeric vector of the manufacturing-specialized base model and the numeric vector of the enterprise and manufacturing area-specific specialized model to provide an answer to the user query to the user.

[0012] In addition, the above Hangul manufacturing dataset may be a dataset composed of Korean-based text data collected from manufacturing areas of the manufacturing industry, such as injection molding, welding, forging, pressing, rolling, and precision machining.

[0013] And the above-mentioned first small language model may be a natural language understanding (NLU) open source pre-trained small language model fully fine-tuned with the Hangeul manufacturing dataset.

[0014] Additionally, the above step a) may include the step of the first language model building unit constructing the first small language model by full fine-tuning a natural language understanding open source model of the BERT family among open source models with the Hangul manufacturing dataset.

[0015] And the above-mentioned first small language model can generate answers to user queries related to Korean-based manufacturing through full fine tuning based on the above-mentioned Korean manufacturing dataset.

[0016] In addition, the second small language model may be a second small language model that is a natural language processing (NLP) open source pre-trained small language model fully fine-tuned with a Hangeul manufacturing dataset.

[0017] And the above step a) may include the step of the first language model building unit constructing the second small language model by full fine-tuning a natural language understanding open source model of the GPT family among open source models with the Hangul manufacturing dataset.

[0018] In addition, the second small language model can generate answers to user queries related to Korean-based manufacturing through full fine tuning based on the Korean manufacturing dataset.

[0019] And the second language model building unit can build the plurality of specialized models for each company and manufacturing area by fine-tuning the manufacturing-specialized based model using the specialized data for each company and manufacturing area, based on at least one PEFT (Parameter Efficient Fine-tuning) technique among LoRA (Low-Rank Adaptation), QLoRA (Quantized LoRA), Adapter, and Prefix Tuning.

[0020] In addition, the above manufacturing area may include areas of the manufacturing industry such as injection molding, welding, forging, pressing, rolling, and precision machining.

[0021] And the above-mentioned company may be a plurality of companies capable of performing manufacturing operations to produce products in the above-mentioned manufacturing area.

[0022] Additionally, in step c), the routing manager infers the company to which the user belongs from the user query based on user information including the affiliation of the user who entered the user query, and can infer the manufacturing area from the text constituting the user query.

[0023] And the answer to the above user query may be a final answer consisting of manufacturing-related text containing information about the above company and manufacturing area. Effects of the invention

[0024] The present invention provides a manufacturing-specialized complex language model that can provide answers to user queries related to manufacturing industries that produce products, wherein manufacturing areas such as injection molding, welding, forging, pressing, rolling, and precision machining are configured.

[0025] In addition, when constructing specialized models for enterprises and manufacturing regions through the fine-tuning of manufacturing-specialized models, the present invention utilizes at least one Parameter Efficient Fine-tuning (PEFT) technique among Low-Rank Adaptation (LoRA), Quantized LoRA (QLoRA), Adapter, and Prefix Tuning, thereby reducing computing resources for manufacturing-specialized models and preventing the risk of losing existing knowledge.

[0026] Furthermore, the present invention can provide users with answers applicable to corporate and manufacturing areas by utilizing answers to user queries generated from specialized models specific to corporate and manufacturing areas.

[0027] However, the effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0028] FIG. 1 is a diagram illustrating the components of a language model construction system according to one embodiment of the present invention. FIG. 2 is a diagram illustrating a method for constructing a manufacturing-specialized base model according to an embodiment of the present invention. FIG. 3 is a diagram illustrating the relationship between a manufacturing-specialized base model, a specialized model by enterprise and manufacturing area, and a manufacturing-specialized complex language model according to one embodiment of the present invention. FIGS. 4 and 5 are drawings illustrating the process of a method for providing user query answers using a manufacturing-specific complex language model according to an embodiment of the present invention. Specific details for implementing the invention

[0029] Hereinafter, embodiments of the present invention are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention. However, since the description of the present invention is merely an example for structural or functional explanation, the scope of the present invention should not be interpreted as being limited by the embodiments described in the text. That is, since the embodiments are subject to various modifications and may take various forms, the scope of the present invention should be understood to include equivalents capable of realizing the technical concept. Furthermore, the objectives or effects presented in the present invention do not imply that a specific embodiment must include all of them or only such effects; therefore, the scope of the present invention should not be understood as being limited by them.

[0030] The meaning of the terms described in this invention should be understood as follows.

[0031] Terms such as "first" and "second" are intended to distinguish one component from another, and the scope of rights shall not be limited by these terms. For example, the first component may be named the second component, and similarly, the second component may be named the first component. When a component is referred to as being "connected" to another component, it should be understood that it may be directly connected to that other component, or that there may be other components in between. Conversely, when a component is referred to as being "directly connected" to another component, it should be understood that there are no other components in between. Meanwhile, other expressions describing the relationship between components, such as "between" and "exactly between," or "adjacent to" and "directly adjacent to," shall be interpreted in the same manner.

[0032] A singular expression should be understood to include a plural expression unless the context clearly indicates otherwise, and terms such as "include" or "have" are intended to specify the existence of the set-up features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood not to preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0033] Unless otherwise defined, all terms used herein have the same meaning as generally understood by those skilled in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having meanings consistent with the context of the relevant technology and should not be interpreted as having an ideal or overly formal meaning unless explicitly defined in this invention.

[0035] Language Model Building System

[0036] Hereinafter, a language model building system (10) for performing a method (S10) for providing a user query answer using a manufacturing-specific complex language model according to an embodiment of the present invention will be described in detail with reference to the attached drawings.

[0037] FIG. 1 is a diagram illustrating the components of a language model construction system according to one embodiment of the present invention.

[0038] Referring to FIG. 1, a language model building system (10) according to one embodiment of the present invention includes a first language model building unit (11), a second language model building unit (12), a routing manager (13), and a third language model building unit (14).

[0039] In one embodiment, the first language model building unit (11) is fully fine-tuned with a Korean manufacturing dataset to build a manufacturing-specific base model (110) capable of converting the meaning of text (words, sentences, etc.) that constitutes routing and answers for user queries related to manufacturing into numeric vectors.

[0040] In one embodiment, the manufacturing-specific base model (110) includes a first small language model (111) which is a natural language understanding (NLU) open source pre-trained small language model fully fine-tuned with a Korean manufacturing dataset, and a second small language model (112) which is a natural language processing (NLP) open source pre-trained small language model.

[0041] At this time, the first small language model may be a BERT-family language model, and the second small language model may be a GPT-family language model.

[0042] In addition, the Hangeul manufacturing dataset refers to a Korean language dataset built based on data related to the manufacturing industry.

[0043] As a more specific example, the Hangeul manufacturing dataset may be a dataset composed of Korean-based text data collected from manufacturing areas such as injection molding, welding, forging, pressing, rolling, and precision machining.

[0044] In one embodiment, the second language model building unit (12) can build a specialized model (120) for each company and manufacturing area that is capable of converting the meaning of text constituting the answer to a user query related to manufacturing into a numeric vector through fine-tuning of a manufacturing specialized base model (110) using specialized data for each company and manufacturing area.

[0045] In this case, the company refers to a business entity capable of performing manufacturing operations that produce products by establishing a manufacturing area, and in one embodiment, it may be composed of multiple entities.

[0046] In addition, the manufacturing area may include areas such as injection molding, welding, forging, pressing, rolling, and precision machining.

[0047] In one embodiment, the specialized model (120) for each enterprise and manufacturing area may be a manufacturing-specialized base model (110), which is a small language model, and may be a language model specialized for each enterprise and manufacturing area, and it is preferable to construct multiple models so as to be specialized for each enterprise and manufacturing area.

[0048] In one embodiment, the routing manager (13) receives a user query related to manufacturing and infers a company and manufacturing area from the user query using a first small language model (110), and can select a company and manufacturing area specialized model (120) specialized for the inferred company and manufacturing area among a plurality of company and manufacturing area specialized models (120).

[0049] In one embodiment, the third language model building unit (14) can build a manufacturing-specific complex language model (14) capable of providing answers to user queries related to manufacturing based on numeric vectors converted from a manufacturing-specific base model (110) and numeric vectors converted from a specialized model (120) selected by a routing manager (13).

[0051] 1) Manufacturing specialized base model (110)

[0052] Hereinafter, a manufacturing-specialized base model (110) according to one embodiment of the present invention will be described in detail with reference to the attached drawings.

[0053] FIG. 2 is a diagram illustrating a method for constructing a manufacturing-specialized base model according to an embodiment of the present invention.

[0054] Referring to FIG. 2, the manufacturing-specific base model (110) may be a language model for generating an answer to a user query input into a routing manager (130) and then converting the meaning of the text (words, sentences, etc.) constituting the answer to the user query into a numerical vector through text embedding, so as to provide an answer to a user query related to manufacturing in the manufacturing-specific complex language model (140), and includes a first small language model (111) and a second small language model (112).

[0055] In one embodiment, the first small language model (111) may be a natural language understanding (NLU) open source pre-trained small language model that has been fully fine-tuned with a Hangul manufacturing dataset.

[0056] This first small language model (111) is used in the retrieval of the RAG (Retrieval Augmented Generation) methodology and can be used to improve the performance of the manufacturing-specific base model (110) by understanding the context of user queries related to manufacturing, and can also be used in the Dialog Coordination technique to identify the intent and type of user queries.

[0057] Referring to FIG. 2, the first language model building unit (11) can build the first small language model (111) by training a BERT-family natural language understanding open source model (NLU) among pre-trained open source models (Open Source Pretrained sLM, English main) that can separate intent and type from English-language-based user queries, based on manufacturing-specific Korean fine-tuning and Korean fine-tuning (Hugging Face).

[0058] At this time, manufacturing-specific Korean fine tuning and Korean fine tuning refer to the process of training the BERT-family small language model through full-fine tuning based on the Korean manufacturing dataset so that the BERT-family small language model can generate answers to user queries related to manufacturing based on the Korean language (Korean language).

[0059] That is, the first small language model (111) may be a small language model of the BERT family trained through full fine tuning based on a Korean language manufacturing dataset from the first language model building unit (11).

[0060] In one embodiment, the second small language model (112) may be a natural language processing (NLP) open source pre-trained small language model that has been fully fine-tuned with a Korean language manufacturing dataset.

[0061] This second small language model (112) can act as a generator to generate answers to user queries based on the RAG (Retrieval Augmented Generation) methodology.

[0062] Referring to FIG. 2, the first language model building unit (11) can build a second small language model (112) by training a GPT-family natural language processing open source model (NLP) among pre-trained open source models (Open Source Pretrained sLM, English main) that can separate intent and type from English-speaking language-based user queries, based on manufacturing-specific Korean fine-tuning and Korean fine-tuning (Hugging Face).

[0063] At this time, manufacturing-specific Korean fine tuning and Korean fine tuning refer to the process of training the GPT-family small language model through full-fine tuning based on the Korean manufacturing dataset so that the GPT-family small language model can generate answers to user queries related to manufacturing based on the Korean language (Korean).

[0064] That is, the second small language model (112) may be a GPT-family small language model trained through full fine tuning based on the Hangul manufacturing dataset from the first language model building unit (11).

[0065] In this way, a manufacturing-specific base model (110) including a first small language model (111) and a second small language model (112) can transmit numeric vectors to a manufacturing-specific complex language model (140).

[0067] 2) Specialized models by enterprise and manufacturing sector (120)

[0068] Hereinafter, a specialized model (120) for each enterprise and manufacturing area according to one embodiment of the present invention will be described in detail with reference to the attached drawings.

[0069] FIG. 3 is a diagram illustrating the relationship between a manufacturing-specialized base model, a specialized model by enterprise and manufacturing area, and a manufacturing-specialized complex language model according to one embodiment of the present invention.

[0070] Referring to FIG. 3, the specialized model (120) for each enterprise and manufacturing area may be a language model that converts the meaning of text constituting the answer to a user query related to manufacturing into a numeric vector through fine-tuning of the manufacturing specialized base model (110) using specialized data for each enterprise and manufacturing area.

[0071] At this time, the fine-tuning of the manufacturing-specific base model (110) may be a process of fine-tuning the manufacturing-specific base model (110) based on at least one PEFT (Parameter Efficient Fine-tuning) technique among LoRA (Low-Rank Adaptation), QLoRA (Quantized LoRA), Adapter, and Prefix Tuning.

[0072] In one embodiment, the Low-Rank Adaptation (LoRA) technique is a method for learning by decomposing model parameters into low-dimensional matrices. By freezing the existing model parameters and training only the small-dimensional parameters, it is possible to efficiently fine-tune the model, thereby reducing the training cost and time of the model.

[0073] In addition, the QLoRA (Quantized Low-Rank Adaptation) technique is a method that combines the aforementioned LoRA technique with a quantization technique to represent model weights using a smaller number of bits.

[0074] Furthermore, the Prefix Tuning technique is a method that trains only a very small set of parameters by freezing the parameters of the existing model and training only the parameters of the prefix added before the input token.

[0075] Here, the PEFT technique refers to a lightweighting method that tunes a model by fine-tuning only some parameters, rather than learning all parameters, similar to LoRA, QLoRA, and Prefix Tuning.

[0076] Below, we will describe in detail the fine-tuning of the manufacturing-specialized base model (110) based on the LoRA technique.

[0077] In one embodiment, the second language model building unit (12) can freeze the weights (parameters) of the manufacturing specialized base model (111).

[0078] At this time, the weight of the manufacturing-specialized base model (111) refers to the parameter (knowledge) obtained by the manufacturing-specialized base model (111) through full fine-tuning of the Korean manufacturing dataset.

[0079] In one embodiment, the second language model building unit (12) can decompose parameters that the manufacturing-specialized-based model (11) can additionally learn into low-dimensional matrices (Rank decomposition matrices) to reduce computing resources and prevent the risk of losing existing knowledge during the fine-tuning process of the manufacturing-specialized-based model (110) based on the LoRA technique.

[0080] At this time, the manufacturing-specialized base model (111) learns only the data specific to the company and manufacturing specialization, which is a new parameter (low-dimensional matrix) with the weights frozen, so the number of parameters to be learned is reduced compared to full fine-tuning, and thus the time for fine-tuning can be shortened compared to full fine-tuning.

[0081] In this way, a lightweight enterprise and manufacturing area-specific specialized model (120) can transmit numeric vectors to a manufacturing-specific complex language model (140) through at least one PEFT (Parameter Efficient Fine-tuning) technique among LoRA (Low-Rank Adaptation), QLoRA (Quantized LoRA), Adapter, and Prefix Tuning.

[0083] 3) Routing Manager (13)

[0084] Hereinafter, a routing manager (13) according to an embodiment of the present invention will be described in detail with reference to the attached drawings.

[0085] Referring to FIG. 1, the routing manager (13) is an input device capable of receiving user queries related to manufacturing from a user, and at the same time, may be a control device that infers the enterprise and manufacturing area from the user queries.

[0086] Additionally, the routing manager (13) can infer the company to which the user belongs from the user query based on user information including the affiliation of the user who entered the user query, and can infer the manufacturing area from the text constituting the user query.

[0087] And the routing manager (13) can select a specialized model (120) specialized for a specific corporate and manufacturing area that is inferred from among the multiple specialized models (120) specialized for a specific corporate and manufacturing area built from the second language model building unit (12).

[0088] As a specific example, the routing manager (13) can select a specialized model (120) specialized for injection molding of the first company among multiple specialized models (120) specialized for injection molding of the first company, when the user is an injection molding manager of the first company among multiple companies and the user query is a text-based user question related to injection molding.

[0090] 4) Manufacturing-specific complex language model (140)

[0091] Hereinafter, a manufacturing-specific composite language model (140) according to one embodiment of the present invention will be described in detail with reference to the attached drawings.

[0092] Referring to FIG. 3, the manufacturing-specific complex language model (140) can provide answers to user queries related to manufacturing by merging the numeric vector of answers to user queries converted from the manufacturing-specific base model (110) and the numeric vector of answers to user queries converted from the enterprise and manufacturing area-specific model (120) selected from the routing manager (13).

[0093] At this time, it is preferable that the answer to the user query related to manufacturing provided by the manufacturing-specialized complex language model (140) to the user be a final answer consisting of manufacturing-related text (words, sentences, etc.) containing information about the enterprise and manufacturing area.

[0094] In one embodiment, the manufacturing-specific composite language model (140) can convert numeric vectors received from the manufacturing-specific base model (110) and the enterprise and manufacturing area-specific model (120) into text-based answers based on Natural Language Generation (NLG) techniques.

[0095] As a specific example, the manufacturing-specific compound language model (140) may be a decoder model that generates a natural language sentence from a numeric vector and then provides the natural language sentence to the user as an answer to a user query.

[0096] As another specific example, the manufacturing-specific compound language model (140) may be a decoder model that searches for documents related to manufacturing stored in a separate database (not shown) using numeric vectors based on the RAG technique, calculates similarity from the searched documents, and provides a natural language sentence generated based on the text contained in the document with the highest similarity as an answer to a user query to the user.

[0098] How to provide answers to user queries

[0099] Hereinafter, the process of a method (S10) for providing a user query answer using a manufacturing-specific complex language model according to an embodiment of the present invention will be described in detail with reference to the attached drawings.

[0100] FIGS. 4 and 5 are drawings illustrating the process of a method for providing user query answers using a manufacturing-specific complex language model according to an embodiment of the present invention.

[0101] Referring to FIGS. 4 and FIGS. 5, a user query answer providing method (S10) according to one embodiment of the present invention may proceed in the order of an input step (S11), a selection step (S12, S13), an answer generation and numeric vector conversion step (S14), and an answer providing step (S15).

[0102] In addition, in the method for providing user query answers (S10) according to one embodiment of the present invention, it is preferable to construct a manufacturing-specialized base model (110) in the first language model construction unit (11), a specialized model by enterprise and manufacturing area in the second language model construction unit (12), and a manufacturing-specialized complex language model (140) in the third language model construction unit (14) before the steps (S11 to S15) described above are performed.

[0103] In the input step (S11), the user can input a user query related to manufacturing into the routing manager (13).

[0104] In the selection step (S12, S13), the routing manager (13) can infer a company and manufacturing area from an input user query using a first small language model (111) that constitutes a manufacturing-specialized base model (110), and select a company and manufacturing area-specialized model (120) that is specialized for the inferred company and manufacturing area among a plurality of company and manufacturing area-specific specialized models (120).

[0105] In the answer generation and numeric vector conversion step (S14), the enterprise and manufacturing area specialized model (120) selected from the manufacturing specialized base model (110) and the routing manager (13) can generate an answer to a user query input into the routing manager (130) and then convert the meaning of the text (words, sentences, etc.) constituting the answer to the user query into a numeric vector through text embedding.

[0106] In the answer provision step (S15), the manufacturing-specific complex language model (140) can merge numeric vectors received from the manufacturing-specific base model (110) and the enterprise and manufacturing area-specific specialized model (120) to provide an answer to the user for the user query entered into the routing manager (13).

[0108] Effects according to the present invention

[0109] The method for providing user query answers (S10) of the present invention can provide answers to user queries related to manufacturing industries that produce products, wherein a manufacturing-specific complex language model (140) is configured in manufacturing areas such as injection molding, welding, forging, pressing, rolling, and precision machining.

[0110] In addition, the user query answer providing method (S10) of the present invention can reduce computing resources of the manufacturing-specialized base model (110) and prevent the risk of losing existing knowledge by using at least one PEFT (Parameter Efficient Fine-tuning) technique among LoRA (Low-Rank Adaptation), QLoRA (Quantized LoRA), Adapter, and Prefix Tuning when building a specialized model (120) for each enterprise and manufacturing area through fine-tuning of the manufacturing-specialized base model (110).

[0111] And the user query answer providing method (S10) of the present invention can provide an answer applicable to the enterprise and manufacturing area to the user by using an answer to a user query generated from a specialized model (120) specialized for the enterprise and manufacturing area.

[0113] As described above, the detailed description of the preferred embodiments of the present invention disclosed is provided to enable those skilled in the art to implement and practice the present invention. Although the present invention has been described with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the present invention without departing from the scope of the invention. For example, those skilled in the art may utilize each configuration described in the embodiments described above in combination with one another. Accordingly, the present invention is not intended to be limited to the embodiments shown herein, but to be given the broadest scope consistent with the principles and novel features disclosed herein.

[0114] The present invention may be embodied in other specific forms without departing from the technical spirit and essential features of the invention. Accordingly, the above detailed description should not be interpreted restrictively in all respects but should be considered exemplary. The scope of the invention shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention. The invention is not intended to be limited to the embodiments shown herein, but to be given the broadest possible scope consistent with the principles and novel features disclosed herein. Furthermore, embodiments may be constructed by combining claims that are not explicitly related in the claims, or included as new claims through amendments made after filing. Explanation of the symbols

[0115] 10: Language model construction system, 11: First language model construction unit, 12: Second Language Model Construction Department, 13: Routing Manager, 14: Third Language Model Construction Department, 110: Manufacturing-Specific Base Model, 111: 1st Small Language Model, 112: 2nd Small Language Model, 120: Specialized models by enterprise and manufacturing sector, 140: Manufacturing-specialized complex language model.

Claims

Claim 1 a) A first language model building unit, which is fully fine-tuned with a Hangul manufacturing dataset, builds a manufacturing-specific base model including a first small language model and a second small language model; a second language model building unit builds multiple specialized models for each company and manufacturing area through fine-tuning of the manufacturing-specific base model using specialized data for each company and manufacturing area; and a third language model building unit builds a manufacturing-specific complex language model to provide an answer to a user query; b) A user inputs a user query related to manufacturing into a routing manager; c) The routing manager infers a company and manufacturing area from the user query using the first small language model, and selects a specialized model for each company and manufacturing area that is specialized for the inferred company and manufacturing area among the multiple specialized models for each company and manufacturing area; d) After the manufacturing-specific base model and the specialized model for each company and manufacturing area generate an answer to the user query, the meaning of the text constituting the answer to the user query is converted into a numeric vector through text embedding; and e) a step in which the manufacturing-specialized complex language model merges the numeric vector of the manufacturing-specialized base model and the numeric vector of the enterprise and manufacturing area-specific specialized model to provide an answer to the user query to the user; wherein the second language model building unit builds the plurality of enterprise and manufacturing area-specific specialized models through fine-tuning of the manufacturing-specialized base model using the enterprise and manufacturing area-specific specialized data based on at least one PEFT (Parameter Efficient Fine-tuning) technique among LoRA (Low-Rank Adaptation), QLoRA (Quantized LoRA), Adapter, and Prefix Tuning. Claim 2 A method for providing user query answers using a manufacturing-specialized complex language model, characterized in that, in claim 1, the above-mentioned Hangul manufacturing dataset is a dataset composed of Korean-based text data collected from manufacturing areas of the manufacturing industry, such as injection molding, welding, forging, pressing, rolling, and precision machining. Claim 3 A method for providing user query answers using a manufacturing-specific complex language model, characterized in that, in claim 2, the first small language model is a natural language understanding (NLU) open source pre-trained small language model fully fine-tuned with a Hangul manufacturing dataset. Claim 4 A method for providing user query answers using a manufacturing-specific composite language model, characterized in that, in claim 3, step a) comprises the step of the first language model building unit constructing the first small language model by full fine-tuning a natural language understanding open source model of the BERT family among open source models with the Korean manufacturing dataset. Claim 5 A method for providing user query answers using a manufacturing-specialized complex language model, wherein, in claim 4, the first small language model generates answers to user queries related to manufacturing based on Hangul through full fine tuning based on the Hangul manufacturing dataset. Claim 6 A method for providing user query answers using a manufacturing-specific complex language model, characterized in that, in claim 2, the second small language model is a second small language model that is a natural language processing (NLP) open source pre-trained small language model fully fine-tuned with a Hangul manufacturing dataset. Claim 7 A method for providing user query answers using a manufacturing-specific composite language model, characterized in that, in claim 1, step a) comprises the step of the first language model building unit constructing the second small language model by full fine-tuning a natural language understanding open source model of the GPT family among open source models with the Korean manufacturing dataset. Claim 8 A method for providing user query answers using a manufacturing-specialized complex language model, wherein, in claim 7, the second small language model generates answers to user queries related to manufacturing based on Hangul through full fine tuning based on the Hangul manufacturing dataset. Claim 9 delete Claim 10 A method for providing user query answers using a manufacturing-specialized complex language model, characterized in that, in claim 1, the manufacturing area includes areas of the manufacturing industry such as injection molding, welding, forging, pressing, rolling, and precision machining. Claim 11 A method for providing user query answers using a manufacturing-specialized complex language model, characterized in that, in claim 10, the above-mentioned company is a plurality of companies capable of performing manufacturing that produces products in the above-mentioned manufacturing area. Claim 12 a) A first language model building unit, which is fully fine-tuned with a Hangul manufacturing dataset, builds a manufacturing-specific base model including a first small language model and a second small language model; a second language model building unit builds multiple specialized models for each company and manufacturing area through fine-tuning of the manufacturing-specific base model using specialized data for each company and manufacturing area; and a third language model building unit builds a manufacturing-specific complex language model to provide an answer to a user query; b) A user inputs a user query related to manufacturing into a routing manager; c) The routing manager infers a company and manufacturing area from the user query using the first small language model, and selects a specialized model for each company and manufacturing area that is specialized for the inferred company and manufacturing area among the multiple specialized models for each company and manufacturing area; d) After the manufacturing-specific base model and the specialized model for each company and manufacturing area generate an answer to the user query, the meaning of the text constituting the answer to the user query is converted into a numeric vector through text embedding; and e) a step in which the manufacturing-specialized complex language model merges the numeric vector of the manufacturing-specialized base model and the numeric vector of the enterprise and manufacturing area-specific specialized model to provide an answer to the user query to the user; wherein step c) is characterized in that the routing manager infers the enterprise to which the user belongs from the user query based on user information including the affiliation of the user who entered the user query, and infers the manufacturing area from the text constituting the user query. Claim 13 delete

Images