Semantic recommendation-based log modeling method
By adopting a semantic recommendation-based method in log modeling, combining deep learning and semantic inference technology, accurate modeling query statements are automatically parsed and generated, and the results are displayed through visual components, the problem of insufficient adaptability and generalization capabilities of log modeling methods in the vertical field in the existing technology is solved, and efficient and accurate log data processing and intuitive user interaction are achieved.
Patent Information
- Application Number
- CN202411579676.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-07
- Publication Date
- 2025-02-28
- Estimated Expiration
- 2044-11-07
AI Technical Summary
When processing complex log data in vertical fields, existing log modeling methods lack generalization ability and adaptability, resulting in limited model accuracy, requiring a large amount of training data and computing resources, which is costly, and lack of intuitiveness and visual interaction, which increases the threshold for learning and use.
The log modeling method based on semantic recommendation is adopted, and deep learning technology combined with semantic inference algorithms is used to deeply analyze the log data, automatically identify and understand key information in the log, generate accurate modeling query statements, and use visual components to visualize the data analysis results.
It quickly adapts to business data in different vertical fields, ensures the accuracy and efficiency of data processing, reduces the cost and threshold of model application, and provides a more intuitive and friendly user interaction experience.
Smart Images

Figure CN119537170A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, in particular to deep learning and natural language processing technology, especially to large model technology, and specifically to a log modeling method based on semantic recommendation. Background Art
[0002] Log modeling is a key network security and data analysis technology that focuses on understanding and predicting the behavior of the system while detecting possible abnormal activities. The process begins with the collection of log data generated by the system, which can come from different sources such as servers, network devices, or applications. Then, features that reflect the state or behavior of the system are extracted or constructed from the raw log data. This may include time series data, IP addresses, operation types, response times, etc. The selection and construction of features directly affects the performance of the model. Use statistical-based models, clustering models, or machine learning models to analyze and discover anomalies.
[0003] In the field of log modeling, existing technical solutions mainly revolve around machine learning, natural language processing (NLP) and deep learning models, aiming to extract valuable information from massive log data for anomaly detection, trend analysis and prediction.
[0004] Traditional machine learning methods, such as support vector machines (SVM) and random forests (RF), preprocess log data through feature engineering, convert it into computable feature vectors, and then use the model for classification or regression analysis to identify abnormal behavior or predict system status. However, when processing unstructured or semi-structured log data, these methods often require a lot of manual participation in feature selection, and the model generalization ability is limited.
[0005] The introduction of NLP technology, especially deep learning-based sequence models such as long short-term memory networks (LSTM) and Transformer, can better understand and process the semantic information in logs. By learning the sequence characteristics of log texts, it can automatically capture the correlation between logs and improve the accuracy of anomaly detection. At the same time, by calculating the semantic similarity of texts, it can assist in log clustering and anomaly detection and reduce manual intervention. However, when facing log data in vertical fields, this method lacks a deep understanding of business data, the accuracy and generalization ability of the model are limited, and it often requires a large amount of training data and computing resources, which is costly.
[0006] Although the modeling generation technology based on the general large model can automate the generation of modeling statements to a certain extent and reduce the workload of data analysts, when this technology is applied in vertical fields, due to the lack of understanding of specific business scenarios, the generated modeling statements may not meet specific needs, and lack intuitiveness and visual interaction, which increases the threshold for learning and use. To solve this problem, it is necessary to combine domain knowledge, fine-tune the model or develop a specialized domain model, but this also increases the complexity and cost of implementation.
[0007] In summary, although models based on traditional machine learning and NLP can calculate text semantic similarity, they often need to rely on manual intervention for confidence judgment when processing complex log data in vertical fields, or the accuracy of the model is limited by the depth of knowledge in a specific field, resulting in insufficient generalization and adaptability. In addition, although the modeling generation technology based on general large models aims to automate the generation of modeling statements and reduce the burden of manual modeling, due to the lack of in-depth understanding of specific business scenarios, the generated modeling statements may not accurately match actual needs, and its non-intuitiveness and high learning threshold, as well as the lack of visual interaction, affect the popularity of technology applications and user experience.
[0008] It can be seen that the development of log modeling technology is facing the challenge of how to improve the adaptability to specific business scenarios while maintaining the generalization ability of the model, and how to reduce the cost and threshold of model application. Future technology trends may focus more on the adaptability of the model, visual interaction and the integration of domain knowledge to achieve more efficient and intelligent log analysis and processing. In view of this, the scheme of the present invention proposes a log modeling method based on semantic recommendation. Summary of the invention
[0009] The present invention aims to solve the problems that the log modeling methods in the prior art have insufficient model generalization ability and adaptability, cannot accurately match actual needs, and affect the popularity of technical applications and user experience. A log modeling method based on semantic recommendation is proposed. Through deep learning technology and combined with semantic reasoning algorithm, the log data is deeply analyzed, which can quickly adapt to business data in different vertical fields and ensure the accuracy and efficiency of data processing.
[0010] In order to achieve the above-mentioned object of the invention, the technical solution of the present invention is as follows: A log modeling method based on semantic recommendation, characterized by comprising the following steps: Step a, extracting text content including scene description text and log modeling statements from conventional modeling work; Step b: classify and sort the above text content according to the modeling goal, and construct a question-answer pair with a special structure based on the classified text content; Step c: Integrate the text content required for modeling and the constructed question-answer pairs into a knowledge base; Step d: When the user raises a specific modeling question, the large model and prompt word technology are used to understand the question and infer the text content involved; Step e: using deep learning and semantic reasoning technology, obtain question-answer pairs related to the text content from the knowledge base, and recommend question-answer pairs that are similar or close to the question; Step f: For the recommended question-answer pairs, the system automatically parses the answers and reads their associated attributes, and renders them in conjunction with the visualization component; or uses large models and prompt word technology to modify the content of the question-answer pairs according to the specific constraints in the user's questions, generate adaptive answers and fill them into the visualization component to intuitively display the data processing results.
[0011] Furthermore, the scenario description text and log modeling statements include at least modeling target scenario descriptive text, database description text, database field text, data field text, business requirement text, data analysis description text and log modeling SQL statements.
[0012] Furthermore, the data content contained in the scene description text and the log modeling statement includes at least: Description of modeling objectives, including expected business results, directions, and purposes that modeling needs to achieve; Modeling SQL description, including specific SQL statements written during data analysis and statement comments; Description of the modeling process, including the understanding of data and scenarios during the data modeling process, as well as the data calculation logic, judgment logic, and analysis logic used; Log data structure, including description of database and table structure of analytical data during data modeling; Modeling operator description, including operators or functions used to query and process data in the database; Modeling reference texts include other documents and texts used to guide and assist modeling during the modeling process.
[0013] Furthermore, in step b, the question part of the question-answer pair comes from the scenario description and the modeling process description text, and the answer part contains the specific modeling statements or data processing logic to implement the requirements.
[0014] Furthermore, the answer part of the question and answer pair includes data visualization answer text and modeling visualization answer text. The data visualization answer text is used for data analysis to generate result data that meets visualization requirements, and the modeling visualization answer text is used for data interaction of visualization modeling.
[0015] Furthermore, the specially structured question-answer pair at least includes question, summary, modeling process, function operator, function operator method, time, data type, data table, used fields, modeling SQL and key / value pairs that can enhance business understanding and semantic understanding.
[0016] Furthermore, in step c, the text related to the database is extracted from the text content required for modeling, and is constructed into a first knowledge base after vectorization processing, and a query and matching interface is provided; the text related to modeling is extracted from the text content required for modeling, and is constructed into a second knowledge base after vectorization processing, and a query and matching interface is provided; the constructed question and answer pairs are extracted, constructed into a third knowledge base, and a query and matching interface is provided; query interaction is provided between the first, second, and third knowledge bases.
[0017] Furthermore, when the user raises a specific modeling question, the system first infers the log table field in the database to which the question belongs, and then obtains the question and answer pairs related to the log table field from the knowledge base; then, by screening out related texts and question and answer pairs from the first, second, and third knowledge bases, matching them with the user's question text, and outputting the most similar and close question and answer pairs.
[0018] Furthermore, in step f, the system automatically parses the answer and reads its associated attributes, and renders it in conjunction with the visualization component, including: generating a visualized SQL text based on the judgment of the user's question intention, as well as fields and statistical data with time granularity, while satisfying the attribute constraints of the question text.
[0019] Furthermore, in step f, the large model and prompt word technology are used to modify the content of the question and answer pair according to the specific constraints in the user's question to generate an adaptive answer, including: generating a modeling SQL text based on the judgment of the user's question intention, maintaining the modeling operators and modeling logic in the recommended question and answer, and satisfying the attribute constraints of the question text.
[0020] In summary, the present invention has the following advantages: 1. The present invention uses deep learning technology combined with semantic reasoning algorithms to deeply analyze log data, automatically identify and understand key information in the log, and generate accurate modeling query statements; at the same time, using visualization components, complex data analysis results are displayed in an intuitive chart form, which reduces the learning threshold of data analysis and improves user experience; 2. The present invention uses semantic reasoning technology, combined with large models and prompt words for adjustment, to understand the semantics of user questions, automatically classify modeling targets, and automatically generate the required modeling sentence text, which can achieve fully intelligent data analysis; 3. The method of the present invention does not require retraining of the model and can quickly adapt to business data in different vertical fields to ensure the accuracy and efficiency of data processing; 4. The present invention reserves a wealth of scenario modeling knowledge by building a knowledge base for business analysis; by constructing a special data structure, the large model can understand the data and interface content in the business, realize linkage with the visualization component, and provide a more intuitive and friendly user interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 A flowchart of a log modeling method based on semantic recommendation provided by an embodiment of the present invention; Figure 2 A schematic diagram of modeling data processing is provided for an embodiment of the present invention; Figure 3 A flowchart for constructing a question-answer pair is provided for an embodiment of the present invention; Figure 4 A flowchart of knowledge base construction is provided for an embodiment of the present invention; Figure 5 A flowchart for question intention inference and recommendation is provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The following will describe the implementation examples of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes, and the examples provided are for clear description and easy understanding, and are not intended to limit the scope of protection of the present disclosure. For the sake of clarity and conciseness, well-known functions and explanatory descriptions are omitted in the following description.
[0023] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below. As used herein, the term "model" can represent the association relationship between various data. For example, the above-mentioned association relationship can be obtained based on a variety of technical solutions currently known and / or to be developed in the future.
[0024] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multi-layer processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.
[0025] The following are some explanations of the professional terms in this article: Prompt: A prompt is a word that enhances the performance of a generative model in a specific knowledge domain by inputting specific instructions or questions to the model. The structure of a prompt usually includes the following parts, which can be flexibly combined according to task requirements: context, task description, input data, output format, and example.
[0026] SQL: Structured Query Language, a standard language for managing relational databases. It was defined by the National Institute of Standards and Technology (NIST) and adopted by the International Organization for Standardization (ISO) as a standard method for accessing and processing databases. The syntax and functionality of SQL may vary slightly in different database management systems (such as Oracle, MySQL, SQL Server, PostgreSQL, etc.), but the core concepts and most syntax are universal.
[0027] Data Definition Language (DDL): It is used to create, modify, and delete database objects such as tables, views, indexes, etc. DDL commands include CREATE TABLE, CREATE INDEX, DROP TABLE, ALTER TABLE, etc. These commands allow database administrators to define the structure and layout of the database.
[0028] Data Manipulation Language (DML): Used to insert, update, and delete data. DML commands include INSERT, UPDATE, and DELETE, which enable users to modify data stored in the database.
[0029] Data Query Language (DQL): In fact, data query language is often regarded as a part of DML, but it is mainly used to retrieve data, and the most typical command is SELECT. The basic structure of the SELECT statement is indeed a query block consisting of the SELECT clause, the FROM clause, and the WHERE clause, which is used to extract a specific data set from the database.
[0030] Data Control Language (DCL): used to manage database security and permissions, including granting or revoking user access to the database. DCL commands include GRANT and REVOKE, which are used to manage user roles and permissions to ensure data security and compliance. In addition, the ROLLBACK command is also a transaction control language, which is used to roll back uncommitted transactions and return the database state to the state before the transaction started.
[0031] The present invention provides a log modeling method based on semantic recommendation. The core of the solution is to use semantic reasoning technology to understand the user's questions semantically, and then combine the constructed prompt words to guide the large model to generate the required modeling sentence text to achieve a fully intelligent data analysis effect. This method does not require retraining the model, can quickly adapt to business data in different vertical fields, and ensure the accuracy and efficiency of data processing. At the same time, by constructing a special data structure, the large model can understand the data and interface content in the business, realize linkage with the visualization component, and provide a more intuitive and friendly user interaction experience.
[0032] Refer to Figure 1 The method mainly comprises the following steps: Step 1: Figure 1 As shown in box 101 in the figure, the scenario description text and log modeling statements are extracted from the conventional modeling work. This information carries the business logic and data processing requirements and is the data basis for the subsequent construction of special structured question-answer pairs.
[0033] The scenario description text includes analysis scenarios, business requirements, data analysis and other texts. The scenario description text can be obtained in a variety of ways, including submission, crawling, downloading, database query, etc. The data form is not limited to documents, text, voice, video, etc. Various data forms can be converted into unified text content through existing technical means. The text content at least includes modeling target scenario descriptive text, database description text, database field text, data field text, business requirement text, data analysis description text and log modeling SQL statement.
[0034] like Figure 2The figure shows a schematic diagram of providing modeling data processing, in which box 100 represents obtaining scenario data for conventional modeling work. In the conventional data analysis process of various industries, modeling work is performed on structured log data, and the data required, produced, and referenced in the process are all scenario data.
[0035] Box 110 indicates that the scenario data obtained for conventional modeling work at least includes the following data contents: modeling target description 111, i.e., the expected business results, directions, and purposes that modeling needs to achieve (e.g., analyzing file logs, counting the number of file downloads per day, and drawing trend charts); modeling SQL description 112, i.e., the specific SQL statements and statement remark information written during the data analysis process; modeling process description 113, i.e., the textual contents such as the understanding of data and scenarios, as well as the data calculation logic, judgment logic, and analysis logic used during the data modeling process; log data structure 114, i.e., the description of the database and table structure of the analyzed data during data modeling, including the data description constructed by the data query language (DQL), data manipulation language (DML), data definition language (DDL), and data control language (DCL); modeling operator description 115, i.e., the operators or functions used to query and process data in the database. They can be used to extract, filter, transform, and summarize data to meet specific query requirements (e.g., aggregation operators such as sum, maximum value, and average value); modeling reference text 116, i.e., other documents and texts used to guide and help modeling during the modeling process.
[0036] Step 2: Reference Figure 1 In box 102, the collected scene description texts and log modeling sentences are classified and sorted, and question-answer pairs with special structures are constructed, thereby improving the accuracy of semantic understanding and recommendation.
[0037] In this step, the question part of the question-answer pair usually comes from the scenario description and modeling process description text, while the answer part contains the specific modeling statements or data processing logic to implement the requirements. The question text in the question-answer pair is a question centered around business issues (for example: How many files were downloaded in the system today?), and the answer text is a response structure text that responds to the query text, which can be in a structured text format such as json and xml.
[0038] In this embodiment, the special structure of the question-answer pair adopts a key / value pair, which at least includes questions, summaries, modeling processes, function operators, function operator methods, time, data types, data tables, used fields, modeling SQL and key / value pairs that can enhance business understanding and semantic understanding.
[0039] like Figure 3The flowchart for providing question-answer pair construction is shown, and box 200 in the figure represents the scenario classification of the modeling target. Since the user's question has one or more business demands, it can be to generate a statement that can be used for normal modeling execution, it can be to recommend the original modeling statement from the knowledge base, it can be to generate a statement consistent with the constraint conditions of the question attribute, or it can be to generate a modeling statement that can be used for user data visualization rendering. Therefore, it is necessary to classify the user's demands in the scenario so as to select the correct data structure when constructing the question-answer pair later.
[0040] Taking the example shown in box 210, the questions in the question-answer pair are constructed based on the scene classification result of the modeling target in box 200, combined with the modeling target description 111 and the modeling process description 113. Since the user's question text is incomplete and the target is not clear, the constructed questions need to cover common user business problems so that subsequent recommendations can be truly used in business production.
[0041] As shown in block 220, a structured answer is constructed based on the modeled SQL description 112. Specifically, based on the data content contained in block 110, the SQL statement in the modeled SQL description 112 is extracted, and after being understood by the large model, the data in the SQL is automatically extracted, including the database name, table name, table alias, original field, alias field, field type, operator operation, etc., and the analysis logic text of the SQL is generated at the same time.
[0042] As shown in boxes 230 and 240, according to the classification results of box 200, the answer of box 220 is constructed according to data visualization answer text 230 and modeling visualization answer text 240. The answer of box 230 can be used for data analysis to generate result data that meets visualization requirements, and the answer of box 240 can be used for data interaction of visualization modeling.
[0043] Step 3: Reference Figure 1 In block 103, the text content used in business modeling work, such as database field text, log modeling SQL statements, and constructed question-answer pairs, are integrated into a knowledge base. The knowledge base construction adopts advanced information retrieval and natural language processing technology, which can automatically identify and understand the structural features of the log table in the database and the syntax and logic of the modeling statements, and provide semantic matching and recommendation capabilities.
[0044] Specifically, Figure 4Taking the flowchart of knowledge base construction shown as an example, box 300 in the figure represents the extraction of text related to the database in the text content required for modeling (for example, the extraction of database, table and description text in the database field text), which is constructed into the first knowledge base after vectorization processing, and provides query and matching interfaces; box 310 represents the extraction of text related to modeling in the text content required for modeling (for example, the extraction of modeling reference text in the log modeling SQL statement), which is constructed into the second knowledge base after vectorization processing, and provides query and matching interfaces; box 320 represents the extraction of the question-answer pair formed by box 210, box 230 and box 240, which is constructed into the third knowledge base, and provides query and matching interfaces; box 330 represents the management of the first, second and third knowledge bases, and provides query interaction. In an optional embodiment, the text is vectorized using an NLP natural language processing model.
[0045] Step 4: Reference Figure 1 In block 104, when the user raises a specific modeling question, the log table field in the database to which the question belongs is first inferred based on the modeling question text.
[0046] Specifically, based on the user's question, we use the big model and prompt word technology to understand and infer the question text (for example: How many files were downloaded to the system today?), and infer that the question involves the log table field in the database (for example: the file log table).
[0047] Step 5: Reference Figure 1 In box 105, the question-answer pair corresponding to the log table field to which the question belongs is obtained from the knowledge base, and the question-answer pair similar to or close to the question is recommended.
[0048] This step uses deep learning and semantic reasoning technology to intelligently recommend question-answer pairs that are similar or close to the question. In specific implementation, the log table fields in the database associated with the data are analyzed based on the question text, and the associated texts and question-answer pairs are screened out from the first knowledge base, the second knowledge base, and the third knowledge base, and matched with the question text, thereby outputting the most similar and close question-answer pairs.
[0049] Step 6: Reference Figure 1 In block 106, for the recommended question-answer pair, the system can automatically parse the associated attributes in the answer and render it in conjunction with the visualization component; or Figure 1 As shown in box 107 in , the original content of the question-answer pair can also be automatically modified according to the user's question to adapt to specific constraints and generate a final answer.
[0050] In this step, since the user's question may involve one or more business demands, it is necessary to analyze the user's question intent, determine the business type, and generate a correct answer in order to output modeling SQL structured data that meets the question intent.
[0051] like Figure 5 The flowchart shown is provided for inferring and recommending the question intention in this embodiment. As shown in box 500, the question intention is first analyzed to determine the business type.
[0052] In an optional implementation, based on the judgment, a visualized SQL text as shown in block 501 and fields and statistical data with time granularity can be generated, while satisfying the attribute constraints of the question text. It is understandable that the generated SQL text can automatically generate new time granularity fields to satisfy the visualization for each time scale, and generate new statistical fields using new operators to satisfy the statistical data under the time granularity.
[0053] In an optional implementation, based on the judgment, a modeling SQL text as shown in box 502 can be generated to maintain the modeling operators and modeling logic in the recommended questions and answers, while satisfying the attribute constraints of the question text. It can be understood that the recommended question and answer data comes from the constructed knowledge base, and the data is constructed from the acquired business data, so it can be determined that the modeling operators and modeling logic contained in the data are correct and can be used for business analysis. Therefore, when generating the modeling SQL text, it is necessary to maintain the modeling operators and logic in the original questions and answers, and use the large model to only modify the attribute constraints (for example: modify the time attribute constraints, modify the field value size constraints).
[0054] Step 7: Reference Figure 1 As shown in box 108 in FIG. 1 , the final answer generated in box 107 is filled into the visualization component and rendered to display the result of data processing in an intuitive form.
[0055] Specific as Figure 5 As shown in box 503, the output modeling SQL structured data enters box 504, and after the business type judgment of box 500, a suitable rendering component is selected to render the structured data to complete the question-answer interaction.
[0056] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.
Claims
1. A log modeling method based on semantic recommendation, characterized in that: The steps include: Step a, extracting text content including scenario description text and log modeling statements from conventional modeling work; Step b: classify and sort the above text content according to the modeling goal, and construct a question-answer pair with a special structure based on the classified text content; Step c: Integrate the text content required for modeling and the constructed question-answer pairs into a knowledge base; Step d: When the user raises a specific modeling question, the large model and prompt word technology are used to understand the question and infer the text content involved; Step e: using deep learning and semantic reasoning technology, obtain question-answer pairs related to the text content from the knowledge base, and recommend question-answer pairs that are similar or close to the question; Step f: For the recommended question-answer pairs, the system automatically parses the answers and reads their associated attributes, and renders them in conjunction with the visualization component; or uses large models and prompt word technology to modify the content of the question-answer pairs according to the specific constraints in the user's questions, generate adaptive answers and fill them into the visualization component to intuitively display the data processing results.
2. The log modeling method based on semantic recommendation according to claim 1, characterized in that: The scenario description text and log modeling statements include at least modeling target scenario descriptive text, database description text, database field text, data field text, business requirement text, data analysis description text and log modeling SQL statements.
3. A log modeling method based on semantic recommendation as described in claim 1 or 2, characterized in that: The data content contained in the scenario description text and the log modeling statement at least includes: Description of modeling objectives, including expected business results, directions, and purposes that modeling needs to achieve; Modeling SQL description, including specific SQL statements written during data analysis and statement comments; Description of the modeling process, including the understanding of data and scenarios during the data modeling process, as well as the data calculation logic, judgment logic, and analysis logic used; Log data structure, including description of database and table structure of analytical data during data modeling; Modeling operator description, including operators or functions used to query and process data in the database; Modeling reference texts include other documents and texts used to guide and assist modeling during the modeling process.
4. The log modeling method based on semantic recommendation according to claim 1, characterized in that: In step b, the question part of the question-answer pair comes from the scenario description and modeling process description text, and the answer part contains the specific modeling statements or data processing logic to implement the requirements.
5. The log modeling method based on semantic recommendation as claimed in claim 4, characterized in that: The answer part of the question and answer pair includes data visualization answer text and modeling visualization answer text. The data visualization answer text is used for data analysis to generate result data that meets visualization requirements, and the modeling visualization answer text is used for data interaction of visualization modeling.
6. A log modeling method based on semantic recommendation as described in any one of claims 1, 4 or 5, characterized in that: The specially structured question-answer pair contains at least question, summary, modeling process, function operator, function operator method, time, data type, data table, used fields, modeling SQL and key / value pairs that can enhance business understanding and semantic understanding.
7. The log modeling method based on semantic recommendation according to claim 1, characterized in that: In step c, the text related to the database is extracted from the text content required for modeling, and is constructed into a first knowledge base after vectorization processing, and a query and matching interface is provided; the text related to modeling is extracted from the text content required for modeling, and is constructed into a second knowledge base after vectorization processing, and a query and matching interface is provided; the constructed question and answer pairs are extracted, constructed into a third knowledge base, and a query and matching interface is provided; query interaction is provided between the first, second, and third knowledge bases.
8. The log modeling method based on semantic recommendation according to claim 7, characterized in that: When a user raises a specific modeling question, the system first infers the log table field in the database to which the question belongs, and then obtains the question-answer pairs related to the log table field from the knowledge base; then, by screening out related texts and question-answer pairs from the first, second, and third knowledge bases, it matches them with the user's question text and outputs the most similar and close question-answer pairs.
9. The log modeling method based on semantic recommendation according to claim 1, characterized in that: In step f, the system automatically parses the answer and reads its associated attributes, and renders it in conjunction with the visualization component, including: generating visual SQL text based on the judgment of the user's question intention, as well as fields and statistical data with time granularity, while satisfying the attribute constraints of the question text.
10. The log modeling method based on semantic recommendation according to claim 1, characterized in that: In step f, the large model and prompt word technology are used to modify the content of the question and answer pair according to the specific constraints in the user's question to generate an adaptive answer, including: generating a modeling SQL text based on the judgment of the user's question intention, maintaining the modeling operators and modeling logic in the recommended question and answer, and satisfying the attribute requirements of the question text.
Citation Information
Patent Citations
Intelligent question-answering system construction method and question-answering processing method and device
CN114860913A
Knowledge question-answering method and device, equipment and storage medium
CN116737908A
Question and answer pair generation method, computer equipment and storage medium
CN117633190A
Semantic reasoning method and terminal for learning target in education metaverse
US12125406B1