A method of converting natural language into data analysis thought chains

Abstract

The application relates to the technical field of thinking chain, and particularly discloses a method for converting natural language into a data analysis thinking chain, which comprises the following steps: step S1: obtaining professional names and technical terms, associating the technical terms with specific professional names, and collecting professional information and historical query texts of a user; step S2: identifying technical terms in the historical query texts of the user according to a technical term library, and calculating the frequency of the technical terms; marking the technical terms with a frequency exceeding a preset threshold as high-frequency terms of the user, and calculating a historical center vector according to a text feature vector of the high-frequency terms; step S3: generating a plurality of initial thinking chains when the user inputs a current query text; extracting technical terms in the initial thinking chains according to the technical term library to obtain candidate center vectors of each initial thinking chain; and step S4: comparing the text similarity of each candidate center vector and the historical center vector, and selecting an output result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of thought chain, and in particular to a method for converting natural language into data analysis thought chain. BACKGROUND

[0002] In recent years, tools for converting natural language into data analysis thought chain have become increasingly popular. Behind this trend is the rapid development of natural language processing (NLP) technology and the widespread application of deep learning, particularly breakthroughs in large language models (LLM).

[0003] Thought chain reasoning requires the model to explicitly output intermediate reasoning steps before outputting the final answer, which significantly improves the performance of LLM on complex reasoning tasks. Thought chain reasoning not only improves the reasoning ability of the model, but also increases the interpretability of the model, allowing users to understand the thinking process of the model.

[0004] Although existing thought chain generation tools can generate corresponding thought chains based on user input text content, they are relatively rigid when dealing with complex business scenarios. These tools often only react to immediate input content and cannot fully utilize user historical text data to build a more comprehensive and in-depth understanding of the business context. Therefore, they are difficult to generate targeted and personalized thought chains, which to some extent limits their efficiency and effectiveness in practical applications. SUMMARY

[0005] The purpose of the present application is to provide a method for converting natural language into data analysis thought chain, which solves the above technical problems.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] A method for converting natural language into data analysis thought chain, comprising the following steps:

[0008] Step S1: Obtain a plurality of professional names and a plurality of technical terms, associate each technical term with a professional name, and establish a technical term library; obtain the user's professional name and the user's historical query text, which is the text input by the user for generating thought chain;

[0009] Step S2: According to the user's professional name and the technical term library, obtain all technical terms in the historical query text and obtain the frequency of each technical term, and mark the technical terms with a frequency exceeding a predetermined threshold as the user's high-frequency terms; obtain the text feature vector of all high-frequency terms, and obtain the center vector of the historical query text according to all text feature vectors, marked as the historical center vector;

[0010] Step S3: obtaining a query text input by a user currently, denoted as a current query text; generating a plurality of initial thinking chains according to the current query text; obtaining technical terms in the initial thinking chains according to a professional name of the user and the technical term library, denoted as initial technical terms; and obtaining a center vector of the initial thinking chain according to a text feature vector of the initial technical term, denoted as a to-be-selected center vector;

[0011] Step S4: obtaining a text similarity between each to-be-selected center vector and the historical center vector respectively, and selecting an initial thinking chain corresponding to a to-be-selected center vector with a highest value of the text similarity as an output result of the current query text.

[0012] As a further scheme of the present application, the professional name includes a profession of the user, and the technical terms include C language, circuit analysis, stock, debt and interest.

[0013] As a further scheme of the present application, the process of obtaining the frequency of the technical term includes:

[0014] obtaining a total number N of all technical terms, obtaining a number n of occurrence of any technical term in all technical terms, and then obtaining the frequency f of the technical term as n / N.

[0015] As a further scheme of the present application, the threshold value is a mean value of the frequency of all technical terms, and the mean value of the frequency is an average value of the frequency of all technical terms.

[0016] As a further scheme of the present application, the process of obtaining the center vector according to all text feature vectors includes:

[0017] denoted as {(a11, a12,..., a1 z ), (a21, a22,..., a2 z ),..., (am1, am2,..., am z )}, wherein am z represents a value of the mth text feature vector in the zth dimension, and z is a total number of dimensions; and the center vector of the m text feature vectors is (a1 z , a2 z ,..., am z ), wherein ai z represents a value of the ith text feature vector in the zth dimension, i∈[1, m] and i is a positive integer.

[0018] As a further scheme of the present application, the text similarity is a cosine value of an included angle between the to-be-selected center vector and the historical center vector.

[0019] The present application has the following beneficial effects:

[0020] The present application can generate data analysis thought chains related to the user's profession and interests by analyzing the user's historical query text and technical term library. This personalized service can improve the user experience, making the user feel that the system understands their needs and background better. Automatically extracting technical terms from natural language queries and generating thought chains reduces the need for human intervention and improves data processing efficiency. This is particularly important for businesses or organizations that need to process large amounts of data and queries. By comparing the user's current query with historical queries, it helps to discover new areas or topics that the user may be interested in. This can promote in-depth exploration of knowledge and generation of new ideas. The present application not only improves the efficiency and accuracy of data analysis, but also enhances the user interaction experience and promotes the development of personalized services. BRIEF DESCRIPTION OF DRAWINGS

[0021] The present application will be further described below with reference to the accompanying drawings.

[0022] Figure 1 is a flowchart of a method for converting natural language into data analysis thought chains. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0024] Please refer to Figure 1 The present application is a method for converting natural language into data analysis thought chains, which includes the following steps:

[0025] Step S1: Obtain a number of professional names and a number of technical terms, associate each technical term with a professional name, and establish a technical term library; obtain the user's professional name and the user's historical query text, which is the text input by the user for generating thought chains.

[0026] It can be understood that a series of professional names and technical terms are collected. These technical terms are associated with a specific profession to form a structured technical term library; the present application needs to know the user's professional name, which helps to determine the technical field that the user may be interested in; at the same time, the system also obtains the user's historical query text, which is the data input by the user before for generating thought chains.

[0027] As a preferred embodiment of the present application, the professional name includes the user's profession, and the technical terms include C language, circuit analysis, stocks, debts, and interest rates.

[0028] It should be noted that the professional name includes lawyer, doctor, teacher, etc., which covers a wide range of professional fields; each professional name is associated with a series of specific technical terms, which represent the professional terms and concepts that the profession will involve; for example, if the user's professional name is doctor, the technical terms associated with the user's professional name may include medical terms such as internal medicine, general surgery, and endocrinology; these technical terms not only reflect the professional direction of doctors, but also may involve diagnostic tools, treatment methods, disease names, etc. that they often use in their daily work; similarly, other professions such as lawyers may be associated with legal terms such as contract law, criminal law, and civil law; teachers may be associated with education terms such as pedagogy, psychology, and curriculum design.

[0029] Step S2: According to the user's professional name and the technical term library, all technical terms in the historical query text are obtained, and the frequency of each technical term is obtained. The technical terms with a frequency exceeding a preset threshold are recorded as the user's high-frequency terms; the text feature vectors of all high-frequency terms are obtained, and the center vector of the historical query text is obtained according to all text feature vectors, which is recorded as the historical center vector.

[0030] It should be noted that according to the user's professional name and the technical term library, the process of determining all technical terms in the historical query text includes:

[0031] According to the user's professional name, the technical term range is determined in the technical term library, and the technical term range is all technical terms associated with the same professional name; according to the technical term range, the historical query text is identified to identify all technical terms therein.

[0032] It can be understood that the technical terms in the historical query text are screened according to the professional name of the user and the technical term library; this step ensures that the content of the analysis is related to the professional background of the user; and the frequency of each technical term appearing in the historical query text is calculated; the high and low of the frequency can reflect the interest degree or demand intensity of the user to certain technical fields; the technical terms with an appearance frequency exceeding a preset threshold are marked as high-frequency terms; these high-frequency terms represent the topics or fields that the user is most concerned about, and are the focus of subsequent analysis; for each high-frequency term, its text feature vector is extracted; the text feature vector is a mathematical representation for capturing the semantic information of words; by converting these words into vector form, their meanings and relationships between them can be better understood; based on the text feature vectors of all high-frequency terms, a central vector representing the overall trend and preference of the user's historical queries, i.e. the historical central vector, is calculated; this central vector serves as the basis for subsequent analysis and comparison, helping to more accurately understand the user's needs and provide relevant data analysis thought chains.

[0033] As a preferred embodiment of the present application, the process of obtaining the frequency of the technical term includes:

[0034] The total number N of all technical terms is obtained, and for any technical term, the number n of its appearance in all technical terms is obtained, and then the frequency f of the technical term is obtained.

[0035] As a preferred embodiment of the present application, the threshold value is the mean value of the frequency of all technical terms, and the mean value is the average value of the frequency of all technical terms.

[0036] It can be understood that the mean value as the threshold value can well balance the difference between high-frequency and low-frequency technical terms; if the threshold value is set too high, some important but low-frequency technical terms may be missed; on the contrary, if it is set too low, too many irrelevant high-frequency words may be introduced. The mean value provides an intermediate point, which can more fairly screen out significant technical terms.

[0037] As a preferred embodiment of the present application, the process of obtaining the central vector from all text feature vectors includes:

[0038] Let m text feature vectors be {((a11, a12,..., a1 z ), (a21, a22,..., a2 z ),..., (am1, am2,..., am z )}, where am z represents the value of the mth text feature vector in the zth dimension, and z is the total number of dimensions; then the central vector of the m text feature vectors is ( , ), where ai z represents the value of the i-th text feature vector in the z-th dimension, i ∈ [1, m] and i is a positive integer.

[0039] It can be understood that the average value of each dimension is combined to form a final center vector; this center vector is a multi-dimensional vector, where each element is the average of all text feature vector values in the corresponding dimension; the center vector represents the central tendency or "barycenter" of all text feature vectors.

[0040] Step S3: Obtain the current input query text of the user, denoted as current query text; generate several initial thinking chains according to the current query text; obtain the technical terms in the initial thinking chain according to the user's professional name and the technical term library, denoted as initial technical terms; and obtain the center vector of the initial thinking chain according to the text feature vector of the initial technical term, denoted as the candidate center vector.

[0041] It should be noted that the generation process of the initial thinking chain includes:

[0042] Use natural language processing techniques to perform word segmentation, part-of-speech tagging, and syntax analysis on the current query text to deeply understand the structure and semantics of the current query text; for example, identify the subject, predicate, and object in the sentence and their relationships; consider the context information of the current query text, such as the theme and purpose of the previous and subsequent sentences, paragraphs, or entire documents, to ensure accurate understanding of the meaning of the current query text; associate the extracted technical terms in the current query text with the pre-constructed knowledge base; the knowledge base can include domain-related rules, facts, concepts, and relationships; if necessary, more information can be obtained from external data sources (such as databases, APIs, or the Internet) to enrich and expand the thinking chain; based on the associated information, use logical reasoning to generate a preliminary thinking chain, which involves causal relationships, conditional relationships, and comparative relationships; check the accuracy of the generated thinking chain to ensure that it is consistent with the current query text and known knowledge;

[0043] For example, if the current query text is "generate a profit analysis for January", the thinking chain generation process is:

[0044] 1. Data collection: export all transaction records for January from the database or trading system. These records usually include transaction time, transaction type (buy / sell), transaction amount, transaction fee, transaction variety, etc.

[0045] ​2. Profit calculation: For each transaction, calculate the profit based on the price difference between buying and selling and the transaction quantity. If factors such as holding costs and transaction fees are involved, they also need to be considered in the calculation. Add up the profits of all transactions to get the total profit for the month. At the same time, you can also calculate the profits of different trading varieties and different trading types to facilitate subsequent analysis.

[0046] 3. Data analysis: Analyze the profit trend of each day or week in the month to observe whether there are obvious fluctuations or rules. Analyze the contribution of each trading variety or trading type to the total profit to find out the main source of profit and the main reason for loss.

[0047] 4. Chart making: Use chart making tools (such as Excel, Tableau, etc.) to display the profit data in the form of charts.

[0048] 5. Result interpretation: Interpret the generated profit chart to summarize the trading performance and profit situation in the month, and point out the existing problems and improvement direction.

[0049] It should be noted that the text feature vector is a mathematical representation of the text, which captures the semantic information of words in the text, usually obtained through word embedding techniques such as Word2Vec, GloVe or BERT, etc.

[0050] Step S4: Obtain the text similarity of each candidate center vector and the historical center vector respectively, and select the initial thought chain corresponding to the candidate center vector with the highest value of text similarity as the output result of the current query text.

[0051] It can be understood that for each candidate center vector, calculate the text similarity between it and the historical center vector; the text similarity is a measure used to evaluate the similarity of two texts in content; once the candidate center vector with the highest text similarity is found, select the initial thought chain corresponding to this vector as the output result of the current query text.

[0052] As a preferred embodiment of the present application, the text similarity is the cosine value of the angle between the candidate center vector and the historical center vector.

[0053] It should be noted that the text similarity is determined by calculating the cosine value of the included angle between the candidate center vector and the historical center vector; the cosine value of the included angle is a value between -1 and 1, wherein 1 indicates that the directions of the two vectors are completely the same, 0 indicates that the two vectors are orthogonal (i.e. no correlation), and -1 indicates that the directions are completely opposite; in text analysis, a high cosine value usually means that two texts are more close or similar in semantics; to calculate the cosine similarity between two center vectors, first, the two vectors need to be normalized (i.e. converted into unit vectors), then the dot product of the two vectors is calculated, and finally the dot product is divided by the product of the Euclidean norms (lengths) of the two vectors; the calculation result is the cosine value between the two vectors.

[0054] The above describes one embodiment of the present application in detail, but the content described is only a preferred embodiment of the present application and cannot be considered as limiting the scope of the present application. Any equivalent changes and improvements made within the scope of the present application should still belong to the patent coverage of the present application.

Claims

1. A method for converting natural language into a data analysis thought process, characterized in that, Includes the following steps: Step S1: Obtain several job titles and several technical terms, associate each technical term with a job title, and establish a technical term database; obtain the user's job title and the user's historical query text, wherein the historical query text is the text entered by the user for generating thought chains; Step S2: Based on the user's occupation name and the technical terminology database, obtain all technical terms from the historical query text and obtain the frequency of each technical term. Record the technical terms whose frequency exceeds a preset threshold as the user's high-frequency terms; obtain the text feature vectors of all the high-frequency terms, and obtain the center vector of the historical query text based on all the text feature vectors, and record it as the historical center vector. Step S3: Obtain the query text currently input by the user, denoted as the current query text; generate several initial thought chains based on the current query text; obtain the technical terms in the initial thought chains based on the user's occupation name and the technical terminology database, denoted as the initial technical terms; and obtain the center vector of the initial thought chains based on the text feature vector of the initial technical terms, denoted as the candidate center vector. Step S4: Obtain the text similarity between each candidate center vector and the historical center vector, and select the initial thought chain corresponding to the candidate center vector with the highest text similarity value as the output result of the current query text.

2. The method for converting natural language into a data analysis thought process according to claim 1, characterized in that, In step S1, the occupational name includes the user's occupation, and the technical terms include C language, circuit analysis, stocks, debt, and interest rates.

3. The method for converting natural language into a data analysis thought process according to claim 1, characterized in that, In step S2, the process of obtaining the frequency of the technical terms includes: Obtain the total number N of all technical terms. For any technical term, obtain the number of times it appears among all technical terms, n. Then, obtain the frequency f of the technical term, f = n / N.

4. The method for converting natural language into a data analysis thought process according to claim 1, characterized in that, In step S2, the threshold is the average frequency of all technical terms, and the average frequency is the average value of the frequencies of all technical terms.

5. The method for converting natural language into a data analysis thought process according to claim 1, characterized in that, In step S2, the process of obtaining the center vector based on all text feature vectors includes: Let there exist m text feature vectors, each denoted as {(a11, a12, ..., a1...}. z ), (a21, a22, ..., a2 z ),...,(am1,am2,...,am z )}, where am z Let m represent the value of the m-th text feature vector in the z-th dimension, where z is the total number of dimensions; then the center vector of the m text feature vectors is ( , ,..., ), where ai z Let i represent the value of the i-th text feature vector in the z-th dimension, where i ∈ [1, m] and i is a positive integer.

6. The method for converting natural language into a data analysis thought process according to claim 1, characterized in that, In step S4, the text similarity is the cosine of the angle between the candidate center vector and the historical center vector.

Images