Large language model system and operating method and application method thereof

Abstract

The invention discloses a large-scale language model system and an operation method and an application method thereof. The large-scale language model system comprises a personal artificial intelligence agent, an enterprise artificial intelligence agent and a public artificial intelligence agent. The operation method of the large language model system comprises the following steps: executing a first action plan by a personal artificial intelligence agent according to a first question to obtain a first integration answer and a first integration score; and executing, by the enterprise artificial intelligence agent, a second action plan according to the second question to obtain a second integration answer and a second integration score. An application method of a large language model system comprises the steps that a lattice diagram of a block diagram is converted into a scalable vector diagram, and semantic tags are integrated into the scalable vector diagram; training a large language model by integrating the scalable vector diagram of the semantic tag and the function specification file; and utilizing the trained large language model to generate a block graph described by structured characters.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to the technology of large-scale language model systems, methods for operating large-scale language model systems, and methods for applying large-scale language model systems. Background Technology

[0002] Large Language Models (LLMs) are artificial intelligence trained on massive amounts of data that can generate responses based on input data, such as GPT-3 or GPT-4. However, they lack the ability to verify the authenticity or accuracy of the generated information. LLMs generate responses based on patterns, relevance, and probabilities learned from training data. While they often provide responses that offer useful information, they can also generate inaccurate or misleading information. This could be due to biases in the training data, a lack of context or immediate information, or simply because large language models are unable to engage in genuine understanding or critical thinking.

[0003] Traditionally, to obtain answers to questions covering multiple domains, the following steps are typically followed: First, ask the question to the first large language model, obtaining response 1. Then, using the context of the prompt, ask the same question to the second large language model, obtaining response 2. Similarly, using the context of the prompt, ask the same question to the third large language model, obtaining response 3. By analyzing the responses from multiple large language models across different domains, valuable insights can be gained from each domain's large language model, allowing for the refinement of the answer accordingly. However, if we use multiple large language models to assist in generating content covering multiple domains, it is necessary to validate the key facts in the generated content to ensure that the responses are not biased. Summary of the Invention

[0004] In view of this, the present invention provides a large-scale language model system, a method for operating the large-scale language model system, and a method for applying the large-scale language model system to improve the problems of the prior art. Instead of relying solely on a large-scale language model to generate content, we employ the following methods to ensure the accuracy of key facts in the generated content without requiring extensive manual verification. The large-scale language model acquires and integrates data from different pipelines, including: placing factual data in a vector database for querying; generating rich information about the factual data through the large-scale language model; providing supplementary responses from a larger AI agent; and an integrator within the AI ​​agent responsible for integrating the factual data from the vector database, the inference results based on the factual data, the rich information from the large-scale language model, and the supplementary responses from the larger AI agent.

[0005] This invention provides a large-scale language model system comprising a personal artificial intelligence (AI) computer and an enterprise AI server. The AI ​​computer executes a personal AI agent, performs a first action plan based on a first question to obtain a first integrated answer and a first integrated score. In response to the first integrated score being lower than a threshold, the personal AI agent poses a second question to the enterprise AI agent. The enterprise AI server executes an enterprise AI agent, performs a second action plan based on the second question to obtain a second integrated answer and a second integrated score. In response to the second integrated score being higher than or equal to a threshold, the enterprise AI agent returns a second integrated answer to the personal AI agent.

[0006] The present invention provides a method for operating a large-scale language model system, comprising: a personal AI agent executing a first action plan based on a first question to obtain a first integrated answer and a first integrated score; in response to the first integrated score being lower than a threshold, the personal AI agent proposing a second question to an enterprise AI agent; and the enterprise AI agent executing a second action plan based on the second question to obtain a second integrated answer and a second integrated score; in response to the second integrated score being higher than or equal to a threshold, the enterprise AI agent returning a second integrated answer to the personal AI agent.

[0007] This invention provides a method for applying a large-scale language model system, comprising: converting a block diagram into a raster graph and integrating semantic tags into the scalable vector graph; training a large-scale language model by integrating the scalable vector graph with semantic tags and a functional specification file; and generating a block diagram with structured text descriptions using the trained large-scale language model. Attached Figure Description

[0008] Figure 1 This is a block diagram illustrating a large-scale language model system according to an embodiment of the present invention;

[0009] Figure 2 This is a schematic diagram illustrating the integration of information by an artificial intelligence agent according to an embodiment of the present invention;

[0010] Figure 3 This is a flowchart illustrating an artificial intelligence agent's action plan execution according to an embodiment of the present invention;

[0011] Figure 4 A flowchart illustrating the operation method of a large-scale language model system according to an embodiment of the present invention;

[0012] Figure 5 A flowchart illustrating the operation method of a large-scale language model system according to an embodiment of the present invention;

[0013] Figure 6 A flowchart illustrating the operation method of a large-scale language model system according to an embodiment of the present invention;

[0014] Figure 7 A flowchart illustrating the operation method of a large-scale language model system according to an embodiment of the present invention;

[0015] Figure 8 This is a flowchart illustrating the operation method of a large-scale language model system according to an embodiment of the present invention.

[0016] Figure 9 A flowchart illustrating an application method of a large-scale language model system according to an embodiment of the present invention;

[0017] Figure 10 This is a flowchart illustrating a method for extracting text in the application of a large-scale language model system according to an embodiment of the present invention.

[0018] Figure 11 This is a flowchart illustrating the execution of an artificial intelligence agent to extract data in an application method of a large-scale language model system according to an embodiment of the present invention.

[0019] Symbol Explanation

[0020] 100: Personal Artificial Intelligence Computer

[0021] 102: Personal AI Agent

[0022] 104: Personal Vector Database

[0023] 106: Personal Large-Scale Language Model

[0024] 110: First question

[0025] 112: First integrated answer

[0026] 114: First integrated score

[0027] 120: Second question

[0028] 122: Second integrated answer

[0029] 124: Second integrated score

[0030] 130: The Third Question

[0031] 132: Third integrated answer

[0032] 134: Third integrated score

[0033] 138: Null value

[0034] 200: Enterprise Artificial Intelligence Server

[0035] 202: Enterprise AI Agent

[0036] 204: Enterprise Vector Database

[0037] 206: Enterprise Large-Scale Language Model

[0038] 300: Public Artificial Intelligence Server

[0039] 302: Public AI Agent

[0040] 304: Public Vector Database

[0041] 306: Public Large-Scale Language Model

[0042] 402: Artificial Intelligence Agent

[0043] 4021: Internal Integrator

[0044] 4022: Internal Inference Engine

[0045] 404: External Vector Database

[0046] 406: External Large Language Model

[0047] 600: User

[0048] 602: Larger AI agent

[0049] S101~S103, S111, S113, S115, S117, S119, S121, S123, S125, S127, S129, S131, S133, S135, S137, S139, S141~S144, S201~S203, S211~S215, S221~S224: Steps Detailed Implementation

[0050] Figure 1 This is a block diagram of a large-scale language model system according to an embodiment of the present invention. Please refer to... Figure 1 ,exist Figure 1 The illustrated embodiments include three types of computers. The first is a personal AI computer 100, which includes a personal AI agent 102, a personal vector database 104, and a personal large language model 106. The personal AI computer 100 is, for example, an employee's personal computer. The second is an enterprise AI server 200, which includes an enterprise AI agent 202, an enterprise vector database 204, and an enterprise large language model 206. The enterprise AI server 200 is, for example, an enterprise server on a private cloud. The third is a public AI server 300, which includes a public AI agent 302, a public vector database 304, and a public large language model 306. The public AI server 300 is, for example, a server running on a public cloud service provider's server.

[0051] The large language model system of this invention has a multi-layered structure, employing large language models of different domains or scales. This allows it to meet varying processing power and training data requirements, including: 1. Scalability: A suitable large language model can be selected based on the complexity of the problem. Simpler problems can utilize a less resource-intensive personal large language model 106 on a personal AI computer 100, while more complex problems can leverage a more powerful enterprise large language model 206 on an enterprise AI server 200. The most complex problems can utilize the most powerful public large language model 306 on a public AI server 300. Here, "complexity" refers to the domain involved. 2. Efficiency: The inference engine in the AI ​​agent (102, 202, or 302) can potentially avoid unnecessary calls to the large language model by handling simpler tasks independently. 3. Flexibility: Action plans allow the AI ​​agent (102, 202, or 302) to adjust its data collection strategy based on the initial response from the large language model.

[0052] Figure 2 This is a schematic diagram illustrating the integration of information by an artificial intelligence agent according to an embodiment of the present invention. Please refer to [link / reference]. Figure 2 ,exist Figure 2 The illustrated embodiment's AI agent 402 includes an internal integrator 4021 and an internal inference engine 4022. The internal integrator 4021 connects the internal inference engine 4022, an external vector database 404, an external large language model 406, and a larger AI agent 602. The internal integrator 4021 is responsible for integrating data from the internal inference engine 4022, the external vector database 404, the external large language model 406, and the larger AI agent 602. The larger AI agent 602 (e.g., enterprise AI agent 202 and public AI agent 302) refers to an AI agent 402 with the ability to handle more complex problems compared to the AI ​​agent 402 (e.g., personal AI agent 102). Therefore, the larger AI agent 602 can obtain answers to problems that the AI ​​agent 402 cannot handle and return the answers obtained by the larger AI agent 602 as supplementary responses to the AI ​​agent 402. The internal inference engine 4022, such as Hermit, obtains inference results by inferring factual data from the external vector database 404. Figure 2 The architecture of the illustrated AI agent 402 is also applicable to personal AI agent 102, enterprise AI agent 202, and public AI agent 302.

[0053] The internal integrator 4021 of the AI ​​agent 402 performs the following tasks: 1. Data preprocessing: Converting vector data from an external vector database 404 into a format suitable for integration. In one embodiment, the vector data may be converted into a general vector representation or the numerical values ​​may be standardized. 2. Rich information extraction: Extracting relevant rich information from the text data using a pre-trained external large-scale language model 406. In one embodiment, natural language processing techniques may be applied. 3. Data fusion: Combining factual data, inference results, and rich information using appropriate fusion techniques. In one embodiment, weighted averaging or more advanced algorithms, such as recurrent neural networks (RNNs) or transformers, may be used. 4. Auxiliary response generation: Generating an auxiliary response using the integrated data. In one embodiment, this may be achieved by utilizing natural language generation techniques, such as template-based generation, rule-based systems, or deep learning methods for sequence-to-sequence models.

[0054] Figure 3 This is a flowchart illustrating an action plan executed by an artificial intelligence agent according to an embodiment of the present invention. Please also refer to... Figure 2 and Figure 3 ,exist Figure 3 In the illustrated embodiment, the so-called generated action plan refers to a plan generated or created by the external large language model 406 to achieve a specific goal or result. The action plan is a series of instructions, steps, tasks, or assignments generated by the external large language model 406 based on its understanding of the problem or situation. The generated action plan may involve various tasks and sub-tasks that need to be performed in a specific order. The action plan includes execution steps S141–S144. In step S141, the external vector database 404 is searched to obtain factual data. In step S142, the factual data is inferred through the internal inference engine 4022 of the artificial intelligence agent 402 to obtain a reasoning result. In step S143, the external large language model 406 is queried to obtain rich information related to the factual data. In step S144, the factual data, reasoning result, and rich information are integrated through the internal integrator 4021 of the artificial intelligence agent 402 to obtain an integrated answer.

[0055] "Enriched information" refers to additional or supplementary information that enhances the understanding, relevance, or value of specific content or data. Enriched information provides additional information, explanations, or diagrams to enrich and deepen the overall understanding and value of the content. It can include various elements such as: 1. Background or historical context; 2. Definitions or explanations; 3. Examples or diagrams; 4. Relevant references or sources; 5. Statistical or quantitative data; 6. Relevant facts or figures; 7. Expert opinions or citations; 8. Visual aids or images; 9. Historical events or anecdotes; 10. Real-world examples or case studies.

[0056] Figure 4 This is a flowchart illustrating the operation method of a large-scale language model system according to an embodiment of the present invention. Please refer to... Figure 4 ,exist Figure 4 In the illustrated embodiment, the operation method of the large language model system includes execution steps S101 and S102. In step S101, a personal AI agent 102 executes a first action plan based on a first question 110 to obtain a first integrated answer 112 and a first integrated score 114. In response to the first integrated score 114 being lower than a threshold, the personal AI agent 102 raises a second question 120 to the enterprise AI agent 202. In step S102, the enterprise AI agent 202 executes a second action plan based on the second question 120 to obtain a second integrated answer 122 and a second integrated score 124. In response to the second integrated score 124 being higher than or equal to a threshold, the enterprise AI agent 202 returns the second integrated answer 122 to the personal AI agent 102.

[0057] Figure 5 This is a flowchart illustrating the operation method of a large-scale language model system according to an embodiment of the present invention. Please also refer to... Figure 1 , Figure 4 and Figure 5 ,exist Figure 5In the illustrated embodiment, step S101 of the operation method of the large language model system further includes steps S111, S113, S115, S117, and S119 executed by a personal artificial intelligence agent 102. A first question 110 from the user 600 is received by executing a program of the personal artificial intelligence agent 102 in the personal artificial intelligence computer 100. In step S111, a first action plan is executed based on the first question 110 to obtain a first integrated answer 112. In one embodiment, a personal AI agent 102 requests a personal large language model 106 to return a first action plan for obtaining a first integrated answer 112. The personal large language model 106 returns the first action plan to the personal AI agent 102, and then the personal AI agent 102 executes each instruction of the first action plan step by step, including: searching a personal vector database 104 to obtain first factual data; reasoning about the first factual data through the inference engine of the personal AI agent 102 to obtain a first inference result; querying the personal large language model 106 to obtain first rich information related to the first factual data; and integrating the first factual data, the first inference result, and the first rich information through the integrator of the personal AI agent 102 to obtain the first integrated answer 112.

[0058] In step S113, a first integrated score 114 is obtained based on the scoring of the first integrated answer 112. In one embodiment, the personal AI agent 102 scores the first integrated answer 112 based on the various information contents contained in the first integrated answer 112 to obtain the first integrated score 114.

[0059] In step S115, it is determined whether the first integration score 114 is greater than a threshold. In one embodiment, for example, the threshold is set to 60 points. In response to the personal AI agent 102 determining that the first integration score 114 is lower than the threshold, step S117 is executed. In response to the personal AI agent 102 determining that the first integration score 114 is higher than or equal to the threshold, step S119 is executed.

[0060] In step S117, a second question 120 is generated for the enterprise AI agent 202. In one embodiment, the second question 120 generated by the personal AI agent 102 includes a first integrated answer 112.

[0061] In step S119, the first integrated answer 112 is returned to the user 600. In one embodiment, in response to the personal AI agent 102 determining that the first integrated score 114 is higher than or equal to a threshold, the personal AI agent 102 returns the first integrated answer 112 to the user 600, indicating that the first integrated answer 112 has met the standard.

[0062] In one embodiment, the personal AI agent 102 may receive a second integrated answer 122 returned by the enterprise AI agent 202, integrate the first integrated answer 112 and the second integrated answer 122 into a first response answer, and score the first response answer to obtain a first response score. In response to a first response score below a threshold, the personal AI agent 102 returns a null value (N / A) to the user 600. In response to a first response score above or equal to the threshold, the personal AI agent 102 returns the first response answer to the user 600.

[0063] Figure 6 This is a flowchart illustrating the operation method of a large-scale language model system according to an embodiment of the present invention. Please also refer to... Figure 1 , Figure 4 and Figure 6 ,exist Figure 6 In the illustrated embodiment, step S102 of the operation method of the large language model system further includes steps S121, S123, S125, S127, and S129 executed by the enterprise AI agent 202. A second question 120 posed by the personal AI agent 102 is received by executing the program of the enterprise AI agent 202 of the enterprise AI server 200. In step S121, a second action plan is executed based on the second question 120 to obtain a second integrated answer 122. In one embodiment, the enterprise AI agent 202 requests the enterprise large language model 206 to return a second action plan for obtaining a second integrated answer 122. The enterprise large language model 206 returns the second action plan to the enterprise AI agent 202, and then the enterprise AI agent 202 executes each instruction of the second action plan step by step, including: searching the enterprise vector database 204 to obtain second fact data; reasoning about the second fact data through the inference engine of the enterprise AI agent 202 to obtain a second inference result; querying the enterprise large language model 206 to obtain second rich information related to the second fact data; and integrating the second fact data, the second inference result, and the second rich information through the integrator of the enterprise AI agent 202 to obtain the second integrated answer 122.

[0064] In step S123, a second integrated score 124 is obtained by scoring based on the second integrated answer 122. In one embodiment, the enterprise AI agent 202 scores based on the various information contents contained in the second integrated answer 122 to obtain the second integrated score 124.

[0065] In step S125, it is determined whether the second integration score 124 is greater than a threshold. In one embodiment, for example, the threshold is set to 60 points. In response to the enterprise AI agent 202 determining that the second integration score 124 is lower than the threshold, step S127 is executed. In response to the enterprise AI agent 202 determining that the second integration score 124 is higher than or equal to the threshold, step S129 is executed.

[0066] In step S127, a third question 130 is generated for the enterprise AI agent 302. In one embodiment, the third question 130 generated by the enterprise AI agent 202 includes a first integrated answer 112 and a second integrated answer 122.

[0067] In step S129, a second integrated answer 122 is returned to the personal AI agent 102. In one embodiment, in response to the enterprise AI agent 202 determining that the second integrated score 124 is higher than or equal to a threshold, the enterprise AI agent 202 returns the second integrated answer 122 to the personal AI agent 102, indicating that the second integrated answer 122 has met the standard.

[0068] In one embodiment, the enterprise AI agent 202 receives a third integrated answer 132 from the public AI agent 302, integrates the third integrated answer 132 with the second integrated answer 122 to form a second response answer, and scores the second response answer to obtain a second response score. If the second response score is lower than a threshold, the enterprise AI agent 202 returns a null value (N / A) to the personal AI agent 102. If the second response score is higher than or equal to the threshold, the enterprise AI agent 202 returns a second response answer to the personal AI agent 102.

[0069] Figure 7 This is a flowchart illustrating the operation method of a large-scale language model system according to an embodiment of the present invention. Please also refer to... Figure 4 and Figure 7 ,exist Figure 7 In the illustrated embodiment, the operation method of the large-scale language model system includes execution steps S101 to S103. Steps S101 to S102 have been described above. Figure 4 The embodiments described herein will not be repeated here. In step S103, the public AI agent 302 receives the third question 130 from the enterprise AI agent 202 and executes the third action plan according to the third question 130 to obtain the third integrated answer 132 and the third integrated score 134. In response to the third integrated score 134 being lower than the threshold, the public AI agent 302 returns a null value 138 to the enterprise AI agent 202, or in response to the third integrated score 134 being higher than or equal to the threshold, the public AI agent 302 returns the third integrated answer 132 to the enterprise AI agent 202.

[0070] Figure 8 This is a flowchart illustrating the operation method of a large-scale language model system according to an embodiment of the present invention. Please refer to... Figure 1 , Figure 7 and Figure 8 ,exist Figure 8 In the illustrated embodiment, step S103 of the operation method of the large language model system further includes steps S131, S133, S135, S137, and S139 executed by the public AI agent 302. A third question 130 posed by the enterprise AI agent 202 is received by executing the program of the public AI agent 302 of the public AI server 300. In step S131, a third action plan is executed based on the third question 130 to obtain a third integrated answer 132. In one embodiment, a public AI agent 302 requests a public large language model 306 to return a third action plan for obtaining a third integrated answer 132. The public large language model 306 returns the third action plan to the public AI agent 302, and then the public AI agent 302 executes each instruction of the third action plan step by step, including: searching a public vector database 304 to obtain third fact data; reasoning about the third fact data through the reasoning engine of the public AI agent 302 to obtain a third reasoning result; querying the public large language model 306 to obtain third rich information related to the third fact data; and integrating the third fact data, the third reasoning result, and the third rich information through the integrator of the public AI agent 302 to obtain the third integrated answer 132.

[0071] In step S133, a third integrated score 134 is obtained based on the scoring of the third integrated answer 132. In one embodiment, the public AI agent 302 scores based on the various information contents contained in the third integrated answer 132 to obtain the third integrated score 134.

[0072] In step S135, it is determined whether the third integration score 134 is greater than a threshold. In one embodiment, for example, the threshold is set to 60 points. In response to the public AI agent 302 determining that the third integration score 134 is lower than the threshold, step S137 is executed. In response to the public AI agent 302 determining that the third integration score 134 is higher than or equal to the threshold, step S139 is executed.

[0073] In step S137, a null value 138 is returned to the enterprise AI agent 202. In one embodiment, in response to the public AI agent 302 determining that the third integration score 134 is below a threshold, the public AI agent 302 returns a null value to the enterprise AI agent 202. This indicates that the public AI agent 302 cannot integrate an answer that meets the standard.

[0074] In step S139, the third integrated score 134 is returned to the enterprise AI agent 202. In one embodiment, in response to the public AI agent 302 determining that the third integrated score 134 is higher than or equal to a threshold, the public AI agent 302 returns a third integrated answer 132 to the enterprise AI agent 202, indicating that the third integrated answer 132 has met the standard.

[0075] In one embodiment, when the personal AI agent 102 poses a second question 120 to the enterprise AI agent 202, it incorporates the first integrated answer 112 of the personal AI agent 102 as context into the second question 120. This helps the enterprise AI agent 202 to understand the background of the question more broadly, enabling it to focus on areas that the personal AI agent 102 cannot handle and provide a more comprehensive and accurate response. In another embodiment, when the enterprise AI agent 202 poses a third question 130 to the public AI agent 302, it incorporates the first integrated answer 112 and the second integrated answer 122 obtained by both the personal AI agent 102 and the enterprise AI agent 202 as context into the third question 130. This helps the public AI agent 302 to understand the background of the question more broadly, enabling it to focus on areas that neither the personal AI agent 102 nor the enterprise AI agent 202 can handle and provide a more comprehensive and accurate response. This saves time and resources and avoids redundant interpretation of already covered information.

[0076] In one embodiment, a personal AI agent 102, an enterprise AI agent 202, and a public AI agent 302 can respectively score the obtained first integrated answer 112, second integrated answer 122, and third integrated answer 132. For example, weights or confidence scores can be assigned to each piece of information content (factual data, enrichment information, supplementary responses) based on their reliability, relevance, or importance. These weights can be determined using various techniques, such as: 1. Rule-based scoring: Assigning scores according to predefined rules or heuristics. For example, assigning higher scores to factual data from reliable sources, or giving higher weights to rich information that matches the user's query. 2. Machine learning-based scoring: Training a machine learning model using labeled training data to predict the relevance or accuracy of each piece of information content. Features such as data source reliability, language style, or contextual similarity can be used to train the model. 3. Integration approach: Combining multiple scoring techniques or models to obtain more robust and accurate scores. This may involve averaging or weighting scores from different models or algorithms. 4. Feedback-based scoring: Integrating user feedback to continuously update and refine scores. For example, if users consistently agree that certain information is helpful or relevant, their scores can be adjusted accordingly.

[0077] Large language models can assist users (e.g., engineers) in quickly prototyping, creating design concepts, and exploring various design possibilities to produce new product proposals. Hardware requirements are typically described in functional specification documents. Block diagrams are drawn as bitmaps. Engineers should ensure that all block diagrams required in the product proposal are interconnected at the interface level. To do this, engineers need to collect small block diagrams from numerous PDF files, attempting to combine them into large block diagrams or system diagrams to complete the new product proposal. Automating tasks using large language models (e.g., Meta's Llama 2) can mitigate the workload and cost involved as much as possible. However, large language models excel primarily in natural language processing, not visual understanding. To accurately generate diagrams, it is also necessary to obtain the text of regions containing text from the block diagrams to understand the relationships and interactions between components described in the text. In this embodiment, we convert all available block diagram bitmaps into Scalable Vector Graphics (SVG). Semantic tags are used to label objects in the Scalable Vector Graphics. Large language models are trained and used for inference through these scalable vector graphs with added semantic labels. In this case, the trained large language model can generate a block graph interconnected at the interface level based on the functional specification file.

[0078] Figure 9This is a flowchart illustrating an application method of a large-scale language model system according to an embodiment of the present invention. Please refer to... Figure 9 ,exist Figure 9 In the illustrated embodiments, the application method of the large-scale language model system of the present invention can be executed on a personal artificial intelligence computer 100, an enterprise artificial intelligence server 200, or / and a public artificial intelligence server 300. The application method of the large-scale language model system of the present invention includes: converting the bitmap of the block diagram into a scalable vector diagram and integrating semantic tags into the scalable vector diagram (step S201); training the large-scale language model by integrating the scalable vector diagram with the semantic tags and the functional specification document (step S202); and generating a block diagram of structured text descriptions using the trained large-scale language model (step S203). In one embodiment, the large-scale language model can combine each block diagram of structured text descriptions to create a system diagram according to the system specification requirements proposed by the user. In one embodiment, the large-scale language model can automatically generate system documents based on the system diagram.

[0079] In an embodiment of step S201, the step of integrating semantic tags is also included. The block graph is analyzed to determine appropriate semantic tags for the scalable vector graph; and the semantic tags are either embedded into the scalable vector graph or maintained as a separate tag structure. In other words, semantic tags can be directly embedded in the scalable vector graph or recorded as an associated data file. Semantic tags provide additional information about the meaning and function associated with each element in the scalable vector graph, as illustrated below:

[0080] <rect class="cpu"> Identify the rectangles representing the central processing unit blocks.

[0081] <circle class="camera"> Identify the circles that represent camera blocks.

[0082] <line class="connection"> Identify the lines that connect the blocks.

[0083] Step S201 further includes the step of identifying semantic labels. In one embodiment, different elements in the scalable vector graph, such as components, connections, interfaces, and functions, are identified and defined by analyzing the block graph, and appropriate semantic labels are assigned. By examining the structure and content of the block graph, key attributes and features that need to be labeled can be determined for semantic understanding.

[0084] Step S201 further includes integrating semantic tags with the scalable vector graph. In one embodiment, semantic tags are embedded in the block graph represented by the scalable vector graph to provide additional context and information to large language models. Engineers can enhance the elements of the scalable vector graph by adding attributes or annotations, specifying assigned semantic tags to make the visual representation more informative and meaningful. Alternatively, engineers can maintain a separate tag structure, such as a JSON file, linking the elements of the scalable vector graph with their corresponding semantic tags for use in large language model input processing.

[0085] Step S201 further includes a step of developing a custom tagging scheme. In one embodiment, a custom tagging scheme is developed that combines established semantic tagging principles with domain-specific elements relevant to the specific system architecture and document requirements. By creating a hybrid semantic tagging approach, engineers can leverage existing vocabulary and structures while integrating domain-specific unique elements and relationships. This custom tagging scheme enhances the understanding of block diagrams by large language models, contributing to more accurate analysis and documentation.

[0086] Step S201 further includes steps for verifying and testing the labels. In one embodiment, the integration of semantic labels with the elements of the scalable vector graph is verified by testing whether the large language model can effectively interpret and process the labeled scalable vector graph. Engineers can perform verification tests to ensure that the semantic labels are correctly embedded in the scalable vector graph and that the large language model can accurately interpret the elements of the labels. Testing the integration of the labels helps to identify any inconsistencies or errors that may affect the performance of the large language model in understanding the system architecture.

[0087] Scalable vector graphics (VectorGraphs) are a vector graphics format, meaning they use mathematical shapes and paths to define images. This makes Scalable VectorGraphs resolution-independent and allow for lossless scaling. When we convert a bitmap image of a blocky image to a Scalable VectorGraph, converter software (such as ImageTracer) attempts to identify and extract the different shapes and elements in the image. This can be done using various techniques such as edge detection, color segmentation, and pattern recognition. However, converter software cannot directly convert bitmap images to text strings. Its primary function is to convert bitmap images to vector graphics formats like Scalable VectorGraphs, not to perform Optical Character Recognition (OCR). To convert a bitmap image containing text to text, we need to use OCR software or tools (such as Tesseract).

[0088] Figure 10 This is a flowchart illustrating the text extraction method of a large-scale language model system according to an embodiment of the present invention. Please refer to... Figure 10 ,exist Figure 10 In the illustrated embodiment, step S201 further includes the step of extracting text, comprising: converting the bitmap into a scalable vector map using conversion software (step S211); analyzing the elements of the scalable vector map to identify regions containing text (step S212); rendering the regions containing text into images (step S213); extracting the text content from the images using optical character recognition software (step S214); and combining the text content of each region containing text to form the complete text content of the bitmap (step S215).

[0089] In the embodiment of step S211, conversion software is used to trace the bitmap to create a scalable vector map file. This scalable vector map file will contain vector representations of shapes and elements in the original image, including text boxes (as shapes).

[0090] In an embodiment of step S212, the scalable vector graph file is analyzed to identify elements that may represent blocks of text. This may involve examining their shape, size, position, and any similar textual attributes.

[0091] In an embodiment of step S213, each region containing text is re-rendered as a bitmap (e.g., PNG), which essentially creates a “screenshot” of the text block in the scalable vector map.

[0092] In the embodiment of step S214, each rendered text raster image is transmitted to OCR software or tools to extract the text content from these raster images.

[0093] In the embodiment of step S215, the text content of each region containing text can be combined into a meaningful text content through artificial intelligence or large language model technology.

[0094] Step S212 further includes analyzing the elements of the scalable vector graph to identify regions containing text. In one embodiment, this includes: searching the scalable vector graph for elements with rectangular or square shapes; searching the scalable vector graph for elements on the same horizontal plane with similar vertical spacing; and searching the scalable vector graph for elements with text attribute labels.

[0095] Step S212 further includes a step of shape and size analysis, comprising:

[0096] 1. Look for elements with rectangular or square shapes: In scalable vector graphics, text is often contained within rectangles or squares.

[0097] 2. Identify elements with consistent or similar dimensions: Text in the same file often has a consistent height and width. Look for groups of elements with similar dimensions.

[0098] 3. Filter out elements that are too small: Very small elements are unlikely to represent text, so a reasonable size threshold can be defined for filtering.

[0099] Step S212 further includes a step of position analysis, comprising:

[0100] 1. Find horizontally arranged elements: Text is usually arranged horizontally, so look for groups of elements that are on the same horizontal plane and have similar vertical spacing.

[0101] 2. Identify elements with consistent spacing: Text usually has consistent horizontal spacing between itself, so look for elements with similar horizontal spacing.

[0102] Step S212 further includes a step of attribute analysis, which includes:

[0103] 1. Check for the presence of a "text" element: Scalable vector graphs have a specific "text" element type used to represent text. Searching has... <tag> text< / tag> The element with the tag.

[0104] 2. Look for text-related attributes: Elements representing text may have the following attributes: (1) text-anchor: This attribute defines how the text is aligned within its bounding box. (2) font-size: This attribute specifies the font size used by the text. (3) font-family: This attribute specifies the font family used by the text. (4) Consider the "fill" attribute: Although not a definitive standard, text elements often have the "fill" attribute to define the color of the text.

[0105] In the embodiment of step S212, by combining the above steps, the accuracy of identifying regions containing text in a scalable vector graphics file can be improved. For example, elements with a rectangular shape, horizontal arrangement, and consistent spacing, and labeled "text," are highly likely to be text.

[0106] In the embodiment of step S201, a block diagram typically contains multiple components, some of which may represent interfaces for communication and interaction between other components within the system. An interface can consist of multiple components. Hardware interfaces allow physical devices to communicate with each other or with a computer. Examples include USB ports, HDMI connectors, and audio jacks. Each component in a hardware interface plays a specific role in transmitting or receiving data or power. The main elements of the semantic tags include component identification information, interface type information, compatibility information, spatial relationship information (optional), and additional contextual information (optional), which ensure that the block diagrams can be interconnected at the interface level.

[0107] In the embodiment of step S201, the component identification information of the semantic tag includes:

[0108] 1. Unique Identifier: Assign a unique identifier to each component (e.g., "Power Supply_1", "Amplifier_4").

[0109] 2. Component Type: Specifies the main function of the component (e.g., "Power Supply", "Filter", "Amplifier", "Controller").

[0110] 3. Manufacturer and Model (Optional): For a specific hardware implementation, include manufacturer and model information.

[0111] In the embodiment of step S201, the interface type information of the semantic tag includes:

[0112] 1. Interface Type: Specifies the type of interface (e.g., "Digital", "Analog", "Power", "Communication").

[0113] 2. Signal direction: Indicates the direction of signal flow (e.g., "input", "output", "bidirectional").

[0114] 3. Signal type: Specifies the type of signal being transmitted (e.g., "voltage", "current", "data", "control").

[0115] 4. Connector Type: Identify the physical connector used (e.g., "USB", "Ethernet", "HDMI", "Screw Terminal").

[0116] 5. Polarity (if applicable): Specifies the positive and negative terminals for power or signal connections.

[0117] In the embodiment of step S201, the compatibility information of the semantic tag includes:

[0118] 1. Compatible Interface Type: List the compatible interface type for each component (e.g., "Power Supply_1" can be connected to "DC_Input" or "AC_Input").

[0119] 2. Signal matching requirements: Specify any signal matching requirements, such as voltage level, impedance, or data protocol.

[0120] 3. Limitations: Indicate any limitations on the connection, such as maximum power consumption or signal timing.

[0121] In the embodiment of step S201, the spatial relationship information of the semantic tags (optionally) includes:

[0122] 1. Relative position: Describes the spatial arrangement of components (e.g., "Power Supply_1" is above "Amplifier_4").

[0123] 2. Connection path: Indicates the physical path of the connection between components.

[0124] In the embodiment of step S201, the additional contextual information (optional) of the semantic tag includes:

[0125] 1. Functional role: Describes the role of a component in the system (e.g., "Power Supply_1" provides power to the entire system).

[0126] 2. Interconnection rules: Specify any general interconnection rules or guidelines applicable to this field.

[0127] In the embodiment of step S201, an example of representing the semantic tags of the component using Extensible Markup Language (XML) is as follows:

[0128] <block id="power_supply_1"type="power_supply">

[0129] <interface type="DC_input"direction="input"signal_type="voltage"connector="screw_terminal"polarity="positive" / >

[0130] <interface type="DC_output"direction="output"signal_type="voltage"connector="screw_terminal"polarity="positive" / >

[0131] <compatibility compatible_interface_types="DC_input,DC_output"voltage_level="12V" / >.

[0132] Figure 11 This is a flowchart illustrating the execution of an artificial intelligence agent 402 extracting data in an application method of a large-scale language model system according to an embodiment of the present invention. Please refer to... Figure 11 ,exist Figure 11 In the illustrated embodiment, when the extracted text content contains annotations or references, the information on the annotations or references can be integrated by the artificial intelligence agent 402 when performing a natural language understanding task on the extracted text content. Step S201 includes the step of performing data extraction by the artificial intelligence agent 402, including: distinguishing the scalable vector graph to identify regions containing text and annotations or references (step 221); extracting text content from regions containing text and combining the text content of each region containing text into complete text content (step 222); extracting relevant contextual information from annotations or references (step 223); and integrating the complete text content with the scalable vector graph and relevant contextual information as input data for the large language model (step 224). Annotations or references provide textual explanations, summaries, or additional details of specific elements in the scalable vector graph. Annotations refer to additional labeled textual information for a block in the block graph, which may contain a description of its function or purpose. References refer to references to external files, which may include address links in the application, and the external files may contain detailed descriptions of a component in the scalable vector graph.

[0133] Step S202 further includes training a large language model using question-and-answer pairs. In one embodiment, while using scalable vector graphs of semantic labels and their corresponding functional specification files as training data is a primary learning method for large language models, this method may not necessarily endow large language models with true reasoning abilities like humans. To improve the reasoning accuracy of large language models, training data using "question-and-answer pairs" can be used. In one embodiment, factual questions test the large language model's understanding of specific facts and relationships in the functional specification and corresponding block graphs. Multiple-choice questions allow the large language model to select the correct answer from a set of predefined options. This format is very helpful for factual questions because the answers are usually explicit. In another embodiment, open-ended questions allow the large language model to not only purely retrieve factual information but also engage in more complex reasoning. The large language model needs to generate a textual answer that responds to the question in a comprehensive and informative way. This format is suitable for open-ended questions, requiring the large language model to demonstrate its reasoning ability and understanding of context. During training, the large language model can learn: 1. Visual elements of the scalable vector graph: identified by semantic labels. 2. Functional description in the file: Describes the purpose and behavior of the blocks and their connections.

[0134] Step S202 further includes an inference step. In one embodiment, a trained large language model is used to infer from an unseen scalable vector graph with semantic labels. The large language model receives the new, unseen scalable vector graph with semantic labels as input and, based on relations learned from the training data, generates a text description or a new scalable vector graph describing the interconnections at the interface level, as specified in the functional specification document.

[0135] Step S203 further includes the step of automatically assembling a block diagram from a knowledge base. In one embodiment, a trained large language model is used to automatically assemble the block diagram based on input data and semantic labels. The large language model, trained on a labeled scalable vector graph and a functional specification document, understands the relationships between elements in the scalable vector graph, enabling it to generate an accurate representation of the system architecture. By leveraging the ability of the large language model to interpret semantic labels and functional descriptions, the process of creating complex block diagrams can be simplified.

[0136] Step S203 further includes generating a block diagram based on the input data and semantic tags. In one embodiment, large language model input data, such as system requirements or component specifications, is provided, along with corresponding semantic tags defining the elements and connections within the diagram. The large language model uses this information to construct a block diagram reflecting the system architecture, including identified components and their interconnections. By automating the assembly of the diagram, engineers can save time and effort in creating visual representations of complex systems. In this invention, the block diagram generated by the large language model refers to generating a structured textual description in markup language to represent the block diagram. The structured textual description in markup language can be converted into a visual or graphical representation using specific graphics conversion software.

[0137] Step S203 further includes a step of verifying interface-level connections. In one embodiment, the connections between blocks at the interface level are verified by comparing the generated block diagram with the functional specification document. A large language model is able to analyze the connections depicted in the block diagram and cross-reference them with the interfaces and interactions specified in the functional specification. This verification process ensures that the block diagram accurately reflects the intended system architecture and meets the documentation requirements.

[0138] Step S203 further includes an alignment step with the functional specification document. In one embodiment, it is ensured that the generated block diagram is consistent with the functional specification document, and the connections and interfaces depicted in the block diagram are verified to conform to system requirements and design specifications. By verifying connections at the interface level, engineers can confirm that the block diagram accurately reflects the expected functions and interactions within the system, facilitating effective communication and collaboration among team members.

[0139] The application method of the large language model system of the present invention further includes the step of generating a system architecture diagram. In one embodiment, a detailed system architecture diagram can be generated based on multiple block diagrams generated from the large language model. The generated system diagram should provide an overall view of the system architecture, including all components, connections, and interfaces identified through an automated diagram assembly process. By visualizing the system architecture in the system diagram, engineers can gain a deeper understanding of the overall structure and internal relationships of the system.

[0140] The application method of the large-scale language model system of the present invention also includes a step of automatically generating system files: using the large-scale language model to automatically generate comprehensive system files based on the generated block diagram, system diagram, and related information. The large-scale language model can analyze system architecture diagrams, component specifications, and connections to create detailed files that capture the system's design, functionality, and interactions. By automating the file generation process, engineers can ensure consistency, accuracy, and efficiency when documenting complex systems.

Claims

1. A large-scale language model system, comprising: Personal AI computer, including a personal AI agent; as well as Enterprise AI server, including enterprise AI agent; Specifically, the personal AI agent executes a first action plan based on a first question to obtain a first integrated answer and a first integrated score; in response to the first integrated score falling below a threshold, the personal AI agent presents a second question to the enterprise's AI server; and Specifically, the enterprise's AI agent executes a second action plan based on the second question to obtain a second integrated answer and a second integrated score. In response to the second integrated score being higher than or equal to the threshold, the enterprise's AI server returns the second integrated answer to the individual AI agent.

2. The large-scale language model system as described in claim 1, wherein the personal artificial intelligence computer further includes a personal vector database and a personal large-scale language model; and the enterprise artificial intelligence server further includes an enterprise vector database and an enterprise large-scale language model.

3. The large-scale language model system as described in claim 2, wherein the first action plan comprises: Search this personal vector database to obtain first-fact data; The first factual data is inferred through the reasoning engine of the personal artificial intelligence agent to obtain the first reasoning result; Query the individual's large language model to obtain the first rich information related to the first factual data; as well as The first integrated answer is obtained by integrating the first factual data, the first reasoning result, and the first rich information through the integrator of the personal artificial intelligence agent.

4. The large-scale language model system of claim 3, wherein the second action plan comprises: Search the company's vector database to obtain second-fact data; The second factual data is inferred through the reasoning engine of the enterprise's artificial intelligence agent to obtain a second reasoning result; Query the enterprise's large language model to obtain second rich information related to the second factual data; as well as The integrator, through the enterprise's AI agent, integrates the second factual data, the second reasoning result, and the second rich information to obtain the second integrated answer.

5. The large language model system of claim 1, wherein the personal AI agent obtains the first integrated score based on the first integrated answer; and the enterprise AI agent obtains the second integrated score based on the second integrated answer.

6. The large language model system of claim 1, wherein the second question includes the first integrated answer.

7. The large-scale language model system as described in claim 1, wherein the large-scale language model system further includes a public artificial intelligence server, the public artificial intelligence server including a public artificial intelligence agent; in, The public AI agent receives a third question from the enterprise AI agent and executes a third action plan based on the third question to obtain a third integrated answer and a third integrated score. In response to the third integrated score being lower than a threshold, the public AI agent returns a null value to the enterprise AI agent, or in response to the third integrated score being higher than or equal to the threshold, the public AI agent returns the third integrated answer to the enterprise AI agent.

8. A method for operating a large-scale language model system, comprising: A personal AI agent executes a first action plan based on a first question to obtain a first integrated answer and a first integrated score. In response to the first integrated score falling below a threshold, the personal AI agent poses a second question to the enterprise AI agent. The enterprise AI agent executes a second action plan based on the second question to obtain a second integrated answer and a second integrated score. In response to the second integrated score being higher than or equal to the threshold, the enterprise AI agent returns the second integrated answer to the individual AI agent.

9. The method of operating a large-scale language model system as described in claim 8, wherein the personal AI agent and the personal vector database are connected to the personal large-scale language model, and executing the first action plan comprises: Search this personal vector database to obtain first-fact data; The first factual data is inferred through the reasoning engine of the personal artificial intelligence agent to obtain the first reasoning result; Query the individual's large language model to obtain the first rich information related to the first factual data; as well as The first integrated answer is obtained by integrating the first factual data, the first reasoning result, and the first rich information through the integrator of the personal artificial intelligence agent.

10. The method of operating a large-scale language model system as described in claim 9, wherein the enterprise AI agent and the enterprise vector database are connected to the enterprise large-scale language model, and the execution of the second action plan comprises: Search the company's vector database to obtain second-fact data; The second factual data is inferred through the reasoning engine of the enterprise's artificial intelligence agent to obtain a second reasoning result; Query the enterprise's large language model to obtain second rich information related to the second factual data; as well as The integrator, through the enterprise's AI agent, integrates the second factual data, the second reasoning result, and the second rich information to obtain the second integrated answer.

11. The method of operating a large language model system as claimed in claim 8, wherein the personal AI agent obtains the first integrated score based on the first integrated answer; and the enterprise AI agent obtains the second integrated score based on the second integrated answer.

12. The method of operating a large language model system as described in claim 8, wherein the second question includes the first integrated answer.

13. The method of operating a large-scale language model system as described in claim 8, wherein the method of operating the large-scale language model system further comprises: The public AI agent receives the third question from the enterprise AI agent and executes the third action plan to obtain the third integrated answer and the third integrated score. In response to the third integrated score being lower than the threshold, the public AI agent returns a null value to the enterprise AI agent, or in response to the third integrated score being higher than or equal to the threshold, the public AI agent returns the third integrated answer to the enterprise AI agent.

14. An application method for a large-scale language model system, comprising: (a) Convert the block map into a raster map and integrate the semantic tags into the scalable vector map; (b) Training a large language model by integrating the scalable vector graph of the semantic tags with the functional specification file; and (c) Generate block diagrams of structured text descriptions using the trained large language model.

15. The application method of the large-scale language model system as described in claim 14, wherein step (a) further comprises the step of integrating semantic tags, including: Analyzing the block graph determines the appropriate semantic label for the scalable vector graph; and Embed the semantic tag into the scalable vector graph or maintain a separate tag structure.

16. The application method of the large-scale language model system as described in claim 14, wherein step (a) further comprises a step of extracting text, including: Use conversion software to convert the raster image into the scalable vector image. Analyze the elements of the scalable vector graph to identify regions containing text; Render the region containing the text as an image; The text content of the image was extracted using optical character recognition software; and The text content of each region containing text is combined to form the complete text content of the bitmap.

17. The method of applying a large-scale language model system as described in claim 16, wherein the step of analyzing the element of the scalable vector graph to identify the region containing text comprises: Find the element with a rectangular shape in the scalable vector graph; In this scalable vector graph, find the element that is on the same horizontal plane and has similar vertical spacing; and Find the element with a text attribute label in the scalable vector graph.

18. The application method of the large-scale language model system as described in claim 14, wherein the semantic label comprises: Component identification information, interface type information, and compatibility information.

19. The application method of the large-scale language model system as described in claim 14, wherein step (a) further comprises the step of performing an artificial intelligence agent to extract data, the data extraction step comprising: Distinguish the scalable vector graph to identify regions containing text and annotations or references; Extract the text content from the region containing the text, and combine the text content of each region containing the text into complete text content; Extract relevant contextual information from the annotation or reference; and The complete text content, along with the scalable vector graph and the relevant contextual information, is integrated as input data for the large language model.

20. The method for applying a large-scale language model system as described in claim 17, wherein step (b) comprises: The large language model was trained using question-and-answer pairs.

21. The method for applying a large-scale language model system as described in claim 17, wherein step (c) comprises: Verify that the block diagram of the structured text description is interconnected at the interface level according to the functional specification document.

22. The application method of the large-scale language model system as described in claim 17, wherein the application method of the large-scale language model system further comprises: Combine the block diagrams of each structured text description to generate a system diagram; and System files are generated based on this system diagram.

Images