Method for converting natural language into data analysis thinking chain

Abstract

The invention relates to the technical field of thinking chains, and particularly discloses a method for converting a natural language into a data analysis thinking chain, which comprises the following steps of: S1, acquiring occupational names and technical nouns, and associating the technical nouns with specific occupational names; collecting occupational information of the user and a historical query text of the user; s2, identifying technical nouns in the historical query text of the user according to the technical noun library, and calculating the frequency of the technical nouns; marking the technical nouns of which the frequencies exceed a preset threshold value as high-frequency nouns of the user, and calculating according to text feature vectors of the high-frequency nouns to obtain historical center vectors; s3, generating a plurality of initial thinking chains when a user inputs a current query text; extracting technical nouns from the initial thinking chains according to a technical noun library to obtain a to-be-selected center vector of each initial thinking chain; and S4, comparing the text similarity between each to-be-selected center vector and the historical center vector, and selecting an output result.

Description

Technical Field

[0001] The present invention relates to the technical field of thought chains, and in particular to a method for converting natural language into data analysis thought chains. Background Art

[0002] In recent years, tools that convert natural language into data analysis thinking chains have become increasingly popular; behind this trend is the rapid development of natural language processing (NLP) technology and the widespread application of deep learning, especially breakthroughs in the field of large language models (LLMs).

[0003] Thought chain reasoning requires the model to explicitly output the intermediate step-by-step reasoning steps before outputting the final answer, which greatly improves the performance of LLM on complex reasoning tasks; thought chain reasoning not only improves the reasoning ability of the model, but also increases the interpretability of the model, enabling users to understand the thinking process of the model.

[0004] Although the thought chain generation tools in the existing technology can make corresponding thought chains based on the text content input by the user, they are relatively rigid when dealing with complex business scenarios; these tools can often only respond to the content input in real time, and cannot make full use of the user's historical text data to build a richer and deeper understanding of the business background; therefore, they are difficult to generate targeted and personalized thought chains, which to a certain extent limits their efficiency and effectiveness in practical applications. Summary of the invention

[0005] The purpose of the present invention is to provide a method for converting natural language into a data analysis thought chain to solve the above technical problems.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A method for converting natural language into a data analysis thought chain comprises the following steps:

[0008] Step S1: obtaining a number of occupational names and a number of technical terms, associating each technical term with an occupational name, and establishing a technical term library; obtaining the occupational name of a user, and obtaining the user's historical query text, wherein the historical query text is the text input by the user for generating a thought chain;

[0009] Step S2: according to the user's occupational name and the technical term library, all technical terms are obtained in the historical query text, and the frequency of each technical term is obtained, and the technical terms whose frequency exceeds a preset threshold are recorded as the user's high-frequency terms; the text feature vectors of all the high-frequency terms are obtained, and the central vector of the historical query text is obtained according to all the text feature vectors, which is recorded as the historical central vector;

[0010] Step S3: obtaining the query text currently input by the user, recorded as the current query text; generating several initial thought chains according to the current query text; obtaining the technical terms in the initial thought chain according to the user's occupation name and the technical term library, recorded as the initial technical terms; and obtaining the central vector of the initial thought chain according to the text feature vector of the initial technical terms, recorded as the candidate central vector;

[0011] Step S4: respectively obtaining the text similarity between each candidate central vector and the historical central vector, and selecting the initial thought chain corresponding to the candidate central vector with the highest text similarity value as the output result of the current query text.

[0012] As a further solution of the present invention: the occupation name includes the user's occupation, and the technical terms include C language, circuit analysis, stocks, debts and interest rates.

[0013] As a further solution of the present invention: the process of obtaining the frequency of the technical term includes:

[0014] The total number N of all technical terms is obtained, and for any technical term, the number n of times it appears in all technical terms is obtained, and the frequency f=n / N of the technical term is obtained.

[0015] As a further solution of the present invention: the threshold is the frequency mean of all technical terms, and the frequency mean is the average value of the frequencies of all technical terms.

[0016] As a further solution of the present invention: the process of obtaining the central vector according to all text feature vectors includes:

[0017] There are m text feature vectors, all of which are {(a11, a12, ..., a1 z ), (a21, a22, ..., a2 z ),...,(am1,am2,...,am z )}, where am z represents the value of the mth text feature vector in the zth dimension, where z is the total number of dimensions; then the center vector of the mth text feature vector is ( , , ..., ), where ai z Represents the value of the i-th text feature vector in the z-th dimension, i∈[1,m] and i is a positive integer.

[0018] As a further solution of the present invention: the text similarity is the cosine value of the angle between the candidate center vector and the historical center vector.

[0019] Beneficial effects of the present invention:

[0020] The present invention can generate a data analysis thought chain related to the user's occupation and interests by analyzing the user's historical query text and technical term library. This personalized service can improve the user experience and make users feel that the system understands their needs and background better. Automatically extracting technical terms from natural language queries and generating thought chains reduces the need for manual intervention and improves the efficiency of data processing. This is particularly important for enterprises or organizations that need to process large amounts of data and queries. By comparing the user's current query with historical queries, it helps to discover new areas or topics that the user may be interested in. This can promote in-depth exploration of knowledge and the generation of new ideas. The present invention not only improves the efficiency and accuracy of data analysis, but also enhances the user interaction experience and promotes the development of personalized services. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present invention will be further described below in conjunction with the accompanying drawings.

[0022] Figure 1 It is a flow chart of a method for converting natural language into a data analysis thought chain according to the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figure 1 As shown, the present invention is a method for converting natural language into a data analysis thought chain, comprising the following steps:

[0025] Step S1: Obtain several occupational names and several technical terms, associate each technical term with an occupational name, and establish a technical term library; obtain the user's occupational name and the user's historical query text, where the historical query text is the text entered by the user for generating a thought chain.

[0026] It is understandable that a series of occupational names and technical terms are collected. These technical terms are associated with specific occupations to form a structured technical term library; the present invention needs to know the user's occupational name, which helps to determine the technical field that the user may be interested in; at the same time, the system also obtains the user's historical query text, which is the text previously entered by the user and is used to generate the data of the thought chain.

[0027] As a preferred embodiment of the present invention, the occupation name includes the user's occupation, and the technical terms include C language, circuit analysis, stocks, debts and interest rates.

[0028] It should be noted that the occupational titles include lawyers, doctors, teachers, etc., which cover a wide range of professional fields; each occupational title is associated with a series of specific technical terms, which represent the professional terms and concepts involved in the occupation; for example, if the user's occupational title is doctor, the technical terms associated with the user's occupational title may include professional vocabulary in the medical field such as internal medicine, general surgery, and endocrinology; these technical terms not only reflect the doctor's professional direction, but may also involve diagnostic tools, treatment methods, disease names, etc. that they often use in their daily work; similarly, other occupations such as lawyers may be associated with professional terms in legal fields such as contract law, criminal law, and civil law; teachers may be associated with professional terms in educational fields such as pedagogy, psychology, and curriculum design.

[0029] Step S2: According to the user's occupational name and the technical term library, all technical terms are obtained in the historical query text, and the frequency of each technical term is obtained, and the technical terms whose frequency exceeds a preset threshold are recorded as the user's high-frequency nouns; the text feature vectors of all the high-frequency nouns are obtained, and the central vector of the historical query text is obtained based on all the text feature vectors, which is recorded as the historical central vector.

[0030] It should be noted that, according to the user's occupation name and the technical term library, the process of obtaining all technical terms in the historical query text includes:

[0031] The technical noun scope is determined in the technical noun database according to the occupational name of the user, and the technical noun scope is all technical nouns associated with the same occupational name; the historical query text is identified according to the technical noun scope to identify all technical nouns therein.

[0032] It can be understood that the technical terms in the historical query text are screened according to the user's occupational name and the technical term library; this step ensures that the content of the analysis is related to the user's occupational background; and the frequency of each technical term in the historical query text is calculated; the frequency can reflect the user's interest in certain technical fields or the intensity of demand; the technical terms with an appearance frequency exceeding the preset threshold are marked as high-frequency nouns; these high-frequency nouns represent the topics or fields that users pay most attention to and are the focus of subsequent analysis; for each high-frequency noun, its text feature vector is extracted; the text feature vector is a mathematical representation used to capture the semantic information of words; by converting these words into vector form, their meanings and the relationship between them can be better understood; based on the text feature vectors of all high-frequency nouns, a central vector representing the overall trend and preference of the user's historical queries is calculated, that is, the historical central vector; this central vector serves as the basis for subsequent analysis and comparison, helping to more accurately understand the user's needs and provide relevant data analysis thinking chains.

[0033] As a preferred embodiment of the present invention, the process of obtaining the frequency of the technical term includes:

[0034] The total number N of all technical terms is obtained, and for any technical term, the number n of times it appears in all technical terms is obtained, and the frequency f=n / N of the technical term is obtained.

[0035] As a preferred embodiment of the present invention, the threshold is the frequency mean of all technical terms, and the frequency mean is the average value of the frequencies of all technical terms.

[0036] It is understandable that the mean as a threshold can well balance the difference between high-frequency and low-frequency technical terms; if the threshold is set too high, some important but low-frequency technical terms may be missed; conversely, if it is set too low, too many irrelevant high-frequency terms may be introduced. The mean provides a middle point that can fairly screen out significant technical terms.

[0037] As a preferred embodiment of the present invention, the process of obtaining the central vector according to all text feature vectors includes:

[0038] There are m text feature vectors, all of which are {(a11, a12, ..., a1 z ), (a21, a22, ..., a2 z ),...,(am1,am2,...,am z )}, where am z represents the value of the mth text feature vector in the zth dimension, where z is the total number of dimensions; then the center vector of the mth text feature vector is ( , , ..., ), where ai z Represents the value of the i-th text feature vector in the z-th dimension, i∈[1,m] and i is a positive integer.

[0039] It can be understood that the average values ​​of each dimension are combined to form the final central vector; this central vector is a multidimensional vector, in which each element is the average of all text feature vector values ​​on the corresponding dimension; the central vector represents the central trend or "center of gravity" of all text feature vectors.

[0040] Step S3: Obtain the query text currently input by the user, recorded as the current query text; generate several initial thought chains based on the current query text; obtain the technical terms in the initial thought chain based on the user's occupational name and the technical term library, recorded as the initial technical terms; and obtain the central vector of the initial thought chain based on the text feature vector of the initial technical terms, recorded as the candidate central vector.

[0041] It should be noted that the generation process of the initial thought chain includes:

[0042] Use natural language processing technology to perform word segmentation, part-of-speech tagging, syntactic analysis, etc. on the current query text to deeply understand the structure and semantics of the current query text; for example, identify the subject, predicate and object in the sentence, and the relationship between them; consider the contextual information of the current query text, such as the subject and purpose of the previous and next sentences, paragraphs or the entire document, to ensure an accurate understanding of the meaning of the current query text; associate the technical terms extracted from the current query text with the pre-built knowledge base; the knowledge base can contain domain-related rules, facts, concepts and relationships; if necessary, more information can be obtained from external data sources (such as databases, APIs or the Internet) to enrich and expand the thinking chain; based on the associated information, use logical reasoning to generate a preliminary thinking chain, which involves causal relationships, conditional relationships, comparative relationships, etc.; check the accuracy of the generated thinking chain to ensure that it is consistent with the current query text and known knowledge;

[0043] For example, if the current query text is "Generate profit analysis for January", the thought chain generation process is:

[0044] 1. Data collection: export all transaction records for January from the database or trading system. These records usually include transaction time, transaction type (buy / sell), transaction amount, transaction fee, transaction type, etc.

[0045] 2. Profit calculation: For each transaction, calculate the profit of the transaction based on the difference between the buying and selling prices and the number of transactions; if factors such as position cost and transaction fees are involved, they also need to be considered in the calculation; add up the profits of all transactions to get the total profit for January; at the same time, you can also calculate the profits of different transaction products and different transaction types separately for subsequent analysis;

[0046] 3. Data analysis: Analyze the daily or weekly profit trends in January to see if there are obvious fluctuations or patterns; analyze the contribution of each trading product or type of transaction to the total profit to find out the main sources of profit and the main causes of loss;

[0047] 4. Chart making: Use chart making tools (such as Excel, Tableau, etc.) to present profit data in the form of charts;

[0048] 5. Interpretation of results: Interpret the generated profit chart, summarize the trading performance and profit situation in January, and point out the existing problems and directions for improvement.

[0049] It should be noted that the text feature vector is a mathematical representation of the text, which captures the semantic information of words in the text and is usually obtained through word embedding technology such as Word2Vec, GloVe or BERT.

[0050] Step S4: respectively obtaining the text similarity between each candidate central vector and the historical central vector, and selecting the initial thought chain corresponding to the candidate central vector with the highest text similarity value as the output result of the current query text.

[0051] It can be understood that for each candidate central vector, the text similarity between it and the historical central vector is calculated; text similarity is a metric used to evaluate the degree of similarity between two texts in content; once the candidate central vector with the highest text similarity is found, the initial thought chain corresponding to this vector is selected as the output result of the current query text.

[0052] As a preferred embodiment of the present invention, the text similarity is the cosine value of the angle between the candidate center vector and the historical center vector.

[0053] It should be noted that text similarity is determined by calculating the cosine value of the angle between the candidate center vector and the historical center vector; the cosine value of the angle is a value between -1 and 1, where 1 means that the two vectors are in exactly the same direction, 0 means that the two are orthogonal (i.e., there is no correlation), and -1 means that the directions are completely opposite; in text analysis, a high cosine value usually means that the two texts are semantically closer or similar; to calculate the cosine similarity between two center vectors, first you need to normalize the two vectors (i.e., convert them into unit vectors), then calculate their dot product, and finally divide the dot product by the product of the Euclidean norms (lengths) of the two vectors; the result of this calculation is the cosine value between the two vectors.

[0054] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A method for converting natural language into a data analysis thought chain, characterized in that: The following steps are involved: Step S1: obtaining a number of occupational names and a number of technical terms, associating each technical term with an occupational name, and establishing a technical term library; obtaining the occupational name of a user, and obtaining the user's historical query text, wherein the historical query text is the text input by the user for generating a thought chain; Step S2: according to the user's occupational name and the technical term library, all technical terms are obtained in the historical query text, and the frequency of each technical term is obtained, and the technical terms whose frequency exceeds a preset threshold are recorded as the user's high-frequency terms; the text feature vectors of all the high-frequency terms are obtained, and the central vector of the historical query text is obtained according to all the text feature vectors, which is recorded as the historical central vector; Step S3: obtaining the query text currently input by the user, recorded as the current query text; generating several initial thought chains according to the current query text; obtaining the technical terms in the initial thought chain according to the user's occupation name and the technical term library, recorded as the initial technical terms; and obtaining the central vector of the initial thought chain according to the text feature vector of the initial technical terms, recorded as the candidate central vector; Step S4: respectively obtaining the text similarity between each candidate central vector and the historical central vector, and selecting the initial thought chain corresponding to the candidate central vector with the highest text similarity value as the output result of the current query text.

2. A method for converting natural language into a data analysis thought chain according to claim 1, characterized in that: In step S1, the occupation name includes the user's occupation, and the technical terms include C language, circuit analysis, stocks, debts, and interest rates.

3. The method for converting natural language into a data analysis thought chain according to claim 1, characterized in that: In step S2, the process of obtaining the frequency of the technical terms includes: The total number N of all technical terms is obtained, and for any technical term, the number n of times it appears in all technical terms is obtained, and the frequency f=n / N of the technical term is obtained.

4. The method for converting natural language into a data analysis thought chain according to claim 1, characterized in that: In step S2, the threshold is the frequency mean of all technical terms, and the frequency mean is the average value of the frequencies of all technical terms.

5. The method for converting natural language into a data analysis thought chain according to claim 1, characterized in that: In step S2, the process of obtaining the central vector according to all text feature vectors includes: There are m text feature vectors, all of which are {(a11, a12, ..., a1 z ), (a21, a22, ..., a2 z ),...,(am1,am2,...,am z )}, where am z represents the value of the mth text feature vector in the zth dimension, where z is the total number of dimensions; then the center vector of the mth text feature vector is ( , , ..., ), where ai z Represents the value of the i-th text feature vector in the z-th dimension, i∈[1,m] and i is a positive integer.

6. A method for converting natural language into a data analysis thought chain according to claim 1, characterized in that: In step S4, the text similarity is the cosine value of the angle between the candidate center vector and the historical center vector.

Images