Data analysis method and system based on natural semantics

Abstract

The present disclosure relates to the technical field of data processing. Disclosed are a data analysis method and system based on natural semantics, which method and system are used for improving the accuracy and efficiency of data analysis based on natural semantics. The method comprises: performing position coding on a plurality of candidate word vectors, so as to obtain a plurality of target word vectors, and performing vector conversion, so as to obtain a query vector, a key vector and a value vector of each target word vector; performing dot product similarity calculation on the key vector of each target word vector, so as to obtain a similarity set of each target word vector, and performing attention weight calculation, so as to obtain an attention weight of each target word vector; performing weighted summation on the plurality of target word vectors, so as to obtain a context semantic vector to be converted, and performing vector dimension conversion, so as to obtain a target context semantic vector; and inputting the target context semantic vector into a decoder for content generation, so as to obtain a plurality of pieces of sub-content, performing key clause generation, and performing SQL query statement generation, so as to obtain a target SQL query statement.

Description

Data analysis method and system based on natural semantics

[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on September 25, 2023, with application number 202311237337X and invention name “Data analysis method and system based on natural semantics”, the entire content of which is incorporated by reference into this disclosure. Technical Field

[0002] The present disclosure relates to the field of data processing technology, and in particular to a data analysis method and system based on natural semantics. Background Art

[0003] In the modern data-driven world, data analysis tools have become an essential means of interpreting, understanding, and utilizing data. However, these tools also have some shortcomings that cannot be ignored.

[0004] First, many data analysis tools, while offering intuitive drag-and-drop functionality and features that make data analysis and infographic creation more intuitive, can have a steep learning curve. Understanding the various features and details of these tools requires significant time and effort, making it difficult and challenging for novice users.

[0005] Second, some tools have limitations in customization and flexibility. While they may be suitable for most standard data analysis needs, they may struggle to meet the needs of users with specialized or more complex requirements. This can lead to users resorting to other tools or manually writing code to accomplish tasks that the tool itself cannot.

[0006] In addition, many tools have high requirements for data management and modeling, which may require users to have certain data processing and programming skills. Without sufficient technical support, users may find it difficult and cumbersome to use these tools.

[0007] Finally, some tools may not have intuitive user interfaces, leaving users confused or overwhelmed. The user experience and efficiency of a tool are directly impacted by the user experience. If a tool is overly complex or unintuitive, users may abandon it.

[0008] Summary of the Invention

[0009] In view of this, the embodiments of the present disclosure provide a data analysis method and system based on natural semantics, which are configured to improve the accuracy and efficiency of data analysis based on natural semantics.

[0010] The present disclosure provides a data analysis method based on natural semantics, comprising: obtaining text data to be processed, performing word segmentation processing on the text data to be processed by a preset subword segmentation algorithm to obtain multiple candidate word vectors of the text data to be processed, and performing position encoding on the multiple candidate word vectors to obtain multiple target word vectors; performing vector conversion on each of the target word vectors to obtain a query vector of each target word vector, a key vector of each target word vector, and a value vector of each target word vector; performing dot product similarity calculation on the key vector of each target word vector based on the query vector of each target word vector to obtain a similarity set of each target word vector, and performing attention weight calculation on the similarity set of each target word vector to obtain an attention weight of each target word vector. Attention weight; based on the attention weight of each target word vector, a weighted summation is performed on multiple target word vectors through the value vector of each target word vector to obtain a context semantic vector to be converted, and the context semantic vector to be converted is input into a preset feedforward neural network for vector dimension conversion based on a preset vocabulary to obtain a target context semantic vector; the target context semantic vector is input into a preset decoder for content generation to obtain multiple sub-contents; key clauses are generated through multiple sub-contents to obtain multiple target key clauses, and SQL query statements are generated through multiple target key clauses to obtain a target SQL query statement; the target SQL query statement is executed, the corresponding target text data is collected, and the target text data is transmitted to a preset data display terminal.

[0011] The present disclosure also provides a data analysis system based on natural semantics, including: an acquisition module, which is configured to acquire text data to be processed, segment the text data to be processed by a preset subword segmentation algorithm, obtain multiple candidate word vectors of the text data to be processed, and position-encode the multiple candidate word vectors to obtain multiple target word vectors; a conversion module, which is configured to perform vector conversion on each of the target word vectors to obtain a query vector of each target word vector, a key vector of each target word vector, and a value vector of each target word vector; a calculation module, which is configured to perform dot product similarity calculation on the key vector of each target word vector based on the query vector of each target word vector. The method comprises the following steps: a first step of calculating the similarity of each target word vector and performing attention weight calculation on the similarity set of each target word vector to obtain the attention weight of each target word vector; a second step of calculating the similarity of each target word vector and performing attention weight calculation on the similarity set of each target word vector to obtain the attention weight of each target word vector; a third step of calculating the similarity of each target word vector and performing weighted summation on multiple target word vectors based on the attention weight of each target word vector to obtain a context semantic vector to be converted, and inputting the context semantic vector to be converted into a preset feedforward neural network to perform vector dimension conversion based on a preset vocabulary to obtain a target context semantic vector; an third step of calculating the similarity of each target word vector and performing attention weight calculation on the similarity set of each target word vector to obtain the attention weight of each target word vector; a fourth step of calculating the similarity of each target word vector and performing attention weight calculation on the similarity set of each target word vector to obtain the attention weight of each target word vector; a fifth ... sixth step of calculating the similarity of each target word vector and performing attention weight calculation on the similarity set of each target word vector to obtain the attention weight of each target word vector; a sixth step of calculating the similarity of each target word vector and performing attention weight calculation on the similarity set of each target word vector to obtain the attention weight of each target word vector; a sixth step of calculating the similarity of each target word vector and performing attention weight calculation on the similarity set of each target word vector to obtain the attention

[0012] In the technical solution provided by the present disclosure, a preset subword segmentation algorithm is used to segment the text to be processed, which can better capture the meaning and structure of the text and reduce ambiguity. The segmented text is converted into multiple candidate word vectors and positionally encoded, which helps to retain vocabulary and position information. The attention mechanism is introduced in the vector conversion. By calculating the dot product similarity of the query vector, key vector and value vector, the target word can be associated with the context-related words, highlighting important information, thereby improving the quality of context semantic extraction. The target context semantic vector is obtained by weighted summation of multiple target word vectors using attention weights. This helps to capture the importance of the target word in the entire context and reduces information loss, thereby improving the quality of subsequent content generation and key clause generation. Using the target context semantic vector for content generation can generate more coherent and appropriate sub-content based on the context. Generating target key clauses based on sub-content helps to extract the most critical and informative content from the text, providing a more accurate basis for subsequent SQL query statement generation. Generating SQL query statements based on target key clauses can convert natural language requirements into database query language that can be understood by computers, realizing the conversion from semantics to operations. Executing the generated SQL query accurately retrieves the required data from the database, providing an efficient data retrieval method. Transmitting the acquired target text data to the data display terminal effectively connects the processing flow to the actual data. This helps present the processing results to the user in an easy-to-understand manner, providing data visualization and presentation.

[0013] Overall, this disclosed solution combines multiple steps, including word segmentation, attention mechanisms, content generation, key clause generation, SQL query, and data display. It effectively extracts useful information from the text data being processed and converts it into actual data query and display. This improves the automation and efficiency of text processing, reduces manual intervention, and enhances the accuracy and usability of data analysis. It also significantly increases the flexibility of analysis, eliminating the need for additional tools or code. As long as natural language can be clearly described, the corresponding SQL can be converted to perform data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] FIG1 is a flowchart of a data analysis method based on natural semantics in an embodiment of the present disclosure.

[0016] FIG2 is a flowchart of performing position encoding on multiple candidate word vectors to obtain multiple target word vectors in an embodiment of the present disclosure.

[0017] FIG3 is a schematic diagram of a data analysis system based on natural semantics in an embodiment of the present disclosure.

[0018] Reference numerals: 301, acquisition module; 302, conversion module; 303, calculation module; 304, summation module; 305, input module; 306, generation module; 307, transmission module. DETAILED DESCRIPTION

[0019] The technical solutions of the present disclosure are described clearly and completely below in conjunction with the accompanying drawings. Obviously, the embodiments described are only a portion of the embodiments of the present disclosure, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present disclosure without creative effort are within the scope of protection of the present disclosure.

[0020] In the description of this disclosure, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate the description of this disclosure and simplify the description. They do not indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on this disclosure. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0021] In addition, the technical features involved in different embodiments of the present disclosure described below can be combined with each other as long as they do not conflict with each other.

[0022] For ease of understanding, the specific process of the embodiment of the present disclosure is described below. Please refer to Figure 1, which is a flow chart of a data analysis method based on natural semantics in an embodiment of the present disclosure. As shown in Figure 1, the method includes the following steps:

[0023] S101, obtaining text data to be processed, segmenting the text data to be processed using a preset subword segmentation algorithm to obtain multiple candidate word vectors for the text data to be processed, and position encoding the multiple candidate word vectors to obtain multiple target word vectors;

[0024] Optionally, the text data to be processed is obtained from an external data source. This can be a paragraph, a sentence, or other form of text. The pre-set BPE subword segmentation algorithm is used to segment the text into multiple subword units. It should be noted that the Byte-Pair Encoding (BPE) subword segmentation algorithm is a statistically based text segmentation method used to segment text into multiple subword units. It is a word segmentation method commonly used in natural language processing tasks that can capture the diversity and complexity of vocabulary. The algorithm gradually merges high-frequency consecutive character sequences to generate subwords. The BPE segmentation results are applied to the text to obtain a sequence of segmented subwords. These subwords serve as the basic units for subsequent processing. For each subword, a pre-trained word vector model or embedding layer is used to map it to a corresponding candidate word vector. Each subword is represented by a corresponding word vector. To preserve the position of the subword in the sentence, a positional encoding is added to each subword. The positional encoding can be a set of special vectors that represent the position of each subword. The candidate word vector of each subword is added to the corresponding positional encoding to obtain the target word vector.

[0025] S102, performing vector conversion on each target word vector to obtain a query vector for each target word vector, a key vector for each target word vector, and a value vector for each target word vector;

[0026] Optionally, the query vector is used to determine the relevance of other words relative to the target word. To generate the query vector, a linear transformation is applied to the target word vector: a query weight matrix is ​​created and the target word vector is multiplied by the query weight matrix to obtain the query vector. The key vector is used to construct an attention score to measure the relevance of other words to the target word. Similarly, another linear transformation is performed on the target word vector: a key weight matrix is ​​created and the target word vector is multiplied by the key weight matrix to obtain the key vector. The value vector contains the information of the target word vector and will be weighted according to the attention score. Similarly, another linear transformation is performed to obtain the value vector: a value weight matrix is ​​created and the target word vector is multiplied by the value weight matrix to obtain the value vector.

[0027] This process generates three vectors for each target word: a query vector, a key vector, and a value vector. These vectors are used in the self-attention mechanism to calculate attention weights to facilitate interactions between other words. In models like the Transformer, these operations are performed in parallel across multiple attention heads to capture different semantic relationships.

[0028] S103. Based on the query vector of each target word vector, perform dot product similarity calculation on the key vector of each target word vector to obtain a similarity set, and perform attention weight calculation on the similarity set to obtain an attention weight for each target word vector;

[0029] Optionally, for each target word's query vector and corresponding key vector, a dot product similarity is calculated between them. This similarity metric measures the degree of association between the target word and other words, with higher values ​​indicating stronger associations. The dot product similarity is normalized to produce a score representing similarity. This is achieved by applying a softmax function so that the sum of the similarity scores for all words is 1. The scores in the similarity set are treated as attention weights for each word and are used to weight the value vector of the target word vector. In this way, the attention weights determine the importance of other words to the target word.

[0030] It should be noted that for each target word vector, the dot product similarity is calculated based on the query vector and the key vector. The dot product similarity is normalized to obtain a similarity set. The scores in the similarity set are treated as attention weights and used to weight the value vector of the target word vector.

[0031] S104, based on the attention weight of each target word vector, performing weighted summation on multiple target word vectors using the value vector of each target word vector to obtain a context semantic vector to be converted, and inputting the context semantic vector to be converted into a preset feedforward neural network to perform vector dimension conversion based on a preset vocabulary to obtain a target context semantic vector;

[0032] Optionally, for each target word, the previously calculated attention weights are used to perform a weighted sum on the attention weight and the target word's value vector. The attention weights determine the importance of each target word's value vector within its context. The result of this weighted summation is the contextual semantic vector to be converted for each target word. This vector contains the semantic information of the target word within its context.

[0033] It should be noted that the feedforward neural network contains several fully connected layers (Dense layers), and each layer uses an appropriate activation function. Generally speaking, the intermediate layer can use the rectified linear activation function (Rectified Linear Activation, referred to as ReLU), and the output layer uses the softmax activation function to ensure that the output elements form a probability distribution. Use the trained feedforward neural network with input context semantic vector and target output to perform inference and prediction. The context semantic vector to be converted is input into the feedforward neural network, and the feedforward neural network will output a probability distribution vector, and the probability distribution vector is used as the target context semantic vector.

[0034] S105: Input the target context semantic vector into a preset decoder to generate content, thereby obtaining a plurality of sub-contents;

[0035] Optionally, based on the target context semantic vector, the state of the decoder is initialized to obtain the initial state data of the decoder; the target context semantic vector and the initial state data are input into the decoder for word generation to obtain multiple candidate words; the state of the decoder is updated through the multiple candidate words, and the corresponding updated state parameters are collected; the multiple candidate words and the updated state parameters are input into the decoder for content generation to obtain multiple sub-contents.

[0036] S106: Generate key clauses from the multiple sub-contents to obtain multiple target key clauses, and generate a Structured Query Language (SQL) query statement from the multiple target key clauses to obtain a target SQL query statement;

[0037] Optionally, key information is extracted from multiple sub-contents respectively to obtain key information corresponding to each sub-content; key clauses are generated based on the key information corresponding to each sub-content to obtain multiple target key clauses; grammatical logic matching is performed on multiple target key clauses based on preset grammatical rules to obtain target grammatical logic; SQL query statements are generated for multiple target key clauses based on the target grammatical logic to obtain target SQL query statements.

[0038] S107: Execute the target SQL query statement, collect the corresponding target text data, and transmit the target text data to a preset data display terminal.

[0039] Optionally, you need to connect to the database where the data is stored. Use a database connection library (for example, sqlite3 or SQLAlchemy in Python) to establish a connection to the database. Use the target SQL query statements and execute these queries through the database connection. This will retrieve the target text data related to the key clause. Once the query is successfully executed, a result set will be obtained, which contains the target text data that meets the query conditions. Extract the required information from the result set. Transfer the collected target text data to a pre-configured data display terminal. This may be a web application, a graphical interface, or a command-line tool for displaying data in the database. Transfer the collected target text data to the data display terminal in a suitable format. This may involve converting the data into formats such as JSON, Hypertext Markup Language (HTML), and text for display in the terminal. In the data display terminal, the transferred target text data is displayed to the user. This can be a table, list, chart, or other visual form so that the user can more clearly understand and analyze the data.

[0040] By executing the above steps S101-S107, the text to be processed is segmented using a preset subword segmentation algorithm, which can better capture the meaning and structure of the text and reduce ambiguity. The segmented text is converted into multiple candidate word vectors and positionally encoded, which helps to retain vocabulary and position information. The attention mechanism is introduced in the vector conversion. By calculating the dot product similarity of the query vector, key vector and value vector, the target word can be associated with the context-related words, highlighting important information, thereby improving the quality of context semantic extraction. The target context semantic vector is obtained by weighted summing up the multiple target word vectors through the attention weight. This helps to capture the importance of the target word in the entire context and reduces information loss, thereby improving the quality of subsequent content generation and key clause generation. Using the target context semantic vector for content generation can generate more coherent and appropriate sub-content according to the context. Generating target key clauses based on sub-content helps to extract the most critical and informative content from the text, providing a more accurate basis for subsequent SQL query statement generation. Generating SQL queries based on target key clauses converts natural language requirements into computer-understandable database query language, achieving a transformation from semantics to operations. Executing the generated SQL queries accurately retrieves the required data from the database, providing an efficient data retrieval method. Transmitting the retrieved target text data to the data display terminal effectively connects the processing flow to the actual data. This helps present processing results to users in an easy-to-understand manner, providing data visualization and presentation.

[0041] Overall, this disclosed solution combines multiple steps, including word segmentation, attention mechanisms, content generation, key clause generation, SQL query, and data display. It effectively extracts useful information from the text data being processed and converts it into actual data query and display. This improves the automation and efficiency of text processing, reduces manual intervention, and enhances the accuracy and usability of data analysis. It also significantly increases the flexibility of analysis, eliminating the need for additional tools or code. As long as natural language can be clearly described, the corresponding SQL can be converted to perform data analysis.

[0042] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0043] (1) Obtaining the text data to be processed and constructing an initial vocabulary based on the text data to be processed;

[0044] The text data to be processed is obtained, which can be an article, a conversation, etc. The obtained text data is preprocessed to better construct the initial vocabulary. Preprocessing includes converting the text to lowercase to ensure case consistency. Punctuation, special characters, and unnecessary spaces are removed to purify the text. Based on the preprocessed text data, the initial vocabulary is constructed. The vocabulary can be a set containing all unique words, subwords, or characters that appear in the text.

[0045] (2) Based on the initial word list, the initial word list is iteratively segmented using a subword segmentation algorithm to obtain multiple candidate segmentation words;

[0046] (3) Performing frequency analysis of adjacent character combinations on multiple candidate segmentation words to obtain character combination frequencies;

[0047] (4) updating the initial vocabulary by character combination frequency to obtain the target vocabulary;

[0048] Optionally, in text processing, the initial vocabulary is a list of words, characters, or subwords extracted from the original text. These are the most basic lexical units. The subword segmentation algorithm is a technique for segmenting words into smaller units, such as BPE (Byte-Pair Encoding). It gradually merges the most frequent lexical pairs in the initial vocabulary into a new sub-vocabulary. Using the subword segmentation algorithm, the initial vocabulary is segmented for multiple rounds of iterations. In each round of iteration, based on the frequency information in the initial vocabulary, the most frequent lexical pairs are selected, merged into a new sub-vocabulary, and the vocabulary is updated. In this way, through multiple rounds of iterations, the vocabulary will gradually increase, and richer lexical units will appear, thereby better capturing the semantics of the text data.

[0049] During the iterative segmentation process, the sub-vocabularies generated are called candidate segmentation words. They can be characters, sub-words, or words. After each iteration, the candidate segmentation words are subjected to a frequency analysis of adjacent character combinations. This means counting the combinations of adjacent characters in each segmentation word and recording their frequency of occurrence in the text data. Character combination frequency refers to the frequency of adjacent character combinations that appear in the text. This frequency information helps find common character combinations, thus providing a basis for the next step of vocabulary update. The initial vocabulary is updated using the character combination frequency information. During the vocabulary update process, common character combinations are merged into larger vocabulary units based on their frequency. In this way, the vocabulary will gradually become richer and more fine-grained. The updated vocabulary is called the target vocabulary. It contains more vocabulary units, some of which are sub-vocabularies generated through iterative segmentation, which can better reflect the semantics and structure of the text data.

[0050] (5) Segmenting the text on multiple candidate segmentation words through the target word list to obtain multiple target segmentation words;

[0051] (6) Convert each target segmentation word into a real-valued vector to obtain multiple candidate word vectors for the text data to be processed;

[0052] (7) Positionally encode multiple candidate word vectors to obtain multiple target word vectors.

[0053] Optionally, a target vocabulary is used to segment the text data into multiple target tokens. This can be achieved by matching words in the target vocabulary with the text. Using a pre-trained word embedding model (such as Word2Vec or GloVe), each target token is mapped into a real-valued vector. These vectors capture the semantic information of the word. A word embedding model is a technique that maps words into continuous real-valued vectors. These vectors are trained to represent the semantic relationships between words. Each target token is mapped into a real-valued vector, known as a candidate word vector. This vector represents the word's position in the semantic space. Positional encoding is a technique that incorporates positional information into the vector representation. This allows the model to distinguish between words at different positions, thereby capturing more contextual information. In text, the positional information of a word refers to its positional order within a sentence. Positional encoding adds this information to each word vector. The final target word vector is obtained by adding the candidate word vector to the positional encoding.

[0054] In a specific embodiment, as shown in FIG2 , the process of performing position encoding on multiple candidate word vectors to obtain multiple target word vectors may specifically include the following steps:

[0055] S201, performing position analysis on each target word vector to obtain a first set of word vectors to be encoded at odd positions and a second set of word vectors to be encoded at even positions;

[0056] S202: Position-encode each first word vector to be encoded in the first word vector set to be encoded using a preset sinusoidal encoding function to obtain a first position encoding vector set;

[0057] S203, position encoding each second word vector to be encoded in the second word vector set to be encoded using a preset cosine encoding function to obtain a second position encoding vector set;

[0058] Optionally, multiple target word vectors are sorted to obtain a target word vector sequence, which is then divided according to position (odd or even) to obtain two subsequences, each containing a word vector in an odd position and an even position. The word vectors in odd positions are extracted to obtain a first set of word vectors to be encoded. These word vectors will serve as input for the first group of positional encoding. The word vectors in even positions are extracted to obtain a second set of word vectors to be encoded. These word vectors will serve as input for the second group of positional encoding. Positional encoding is performed on each word vector, and corresponding position information is added according to its position in the sequence. Sine encoding function and cosine encoding function are usually used to implement positional encoding.

[0059] It should be noted that the sinusoidal encoding function is as follows:

[0060] Among them, pos represents the position index value of the word vector to be encoded, i is the dimension index value of the word vector to be encoded, and d model Indicates the length of the word vector to be encoded, PE 1(pos,i) Represents the sinusoidal code value;

[0061] The cosine encoding function is as follows:

[0062] Among them, PE 2(pos,i) Represents a cosine-encoded value.

[0063] For example, suppose there is a target word vector sequence: [word vector 1, word vector 2, word vector 3, word vector 4, word vector 5].

[0064] Split a sequence into even and odd positions:

[0065] Odd position sequence: [word vector 1, word vector 3, word vector 5];

[0066] Even position sequence: [word vector 2, word vector 4];

[0067] A first set of word vectors to be encoded is obtained: the first set of word vectors to be encoded at odd positions is [word vector 1, word vector 3, word vector 5]. A second set of word vectors to be encoded is obtained: the second set of word vectors to be encoded at even positions is [word vector 2, word vector 4]. Position encoding is performed on each word vector in the first set of word vectors to be encoded, adding position information for odd positions. Position encoding is performed on each word vector in the second set of word vectors to be encoded, adding position information for even positions.

[0068] S204: Perform vector fusion on the first position coding vector set and the first to-be-encoded word vector set to obtain a first vector set;

[0069] S205: Perform vector fusion on the second position coding vector set and the second to-be-encoded word vector set to obtain a second vector set;

[0070] S206: Concatenate the first vector set and the second vector set to obtain multiple target word vectors.

[0071] It should be noted that vector fusion is the process of weighted fusion of two sets of vectors to retain the information of each vector. This can be achieved through methods such as weighted summation. Here, position encoding vectors and word vectors can be used for vector fusion. For the first position encoding vector set and the first word vector set to be encoded, weighted fusion can be performed to obtain a first vector set. Similarly, the second position encoding vector set and the second word vector set to be encoded are weighted fused to obtain a second vector set. Finally, the first vector set and the second vector set are data spliced ​​to obtain multiple target word vectors. It should be noted that data splicing is to connect two sets of vectors in a certain dimension to create a larger vector set. Here, the first vector set and the second vector set are data spliced ​​in a specific dimension to obtain multiple target word vectors.

[0072] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0073] (1) Based on the query vector of each target word vector, the dot product similarity calculation is performed on the key vector of each target word vector to obtain the similarity set of each target word vector;

[0074] (2) Normalize the similarity set of each target word vector separately to obtain the attention weight of each target word vector.

[0075] Optionally, for each target word vector, perform a dot product similarity calculation on its query vector and all key vectors. This yields a set of similarity scores representing the similarity between the target word vector and the other word vectors. For each target word vector, these scores are mapped to the range [0, 1] using a softmax function. This operation ensures that the similarity scores are mapped to probabilities for use in the subsequent weighted summation.

[0076] For example, a target word vector sequence: [Query1, Key1, Query2, Key2].

[0077] For "Query1", the similarity score "Sim1" is obtained by performing a dot product similarity calculation with "Key1". For "Query2", the similarity score "Sim2" is obtained by performing a dot product similarity calculation with "Key2". The similarity scores "Sim1" and "Sim2" are normalized using the softmax function to obtain the attention weights "Weight1" and "Weight2".

[0078] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0079] (1) Initialize the decoder state based on the target context semantic vector to obtain the decoder's initial state data;

[0080] (2) Inputting the target context semantic vector and the initial state data into the decoder to generate words and obtain multiple candidate words;

[0081] (3) updating the decoder state through multiple candidate words and collecting corresponding updated state parameters;

[0082] (4) Inputting multiple candidate words and updated state parameters into the decoder to generate content and obtain multiple sub-contents.

[0083] Optionally, the decoder state is initialized based on the target context semantic vector. In a generative model, the decoder is responsible for combining information from the source text with the context of the target language to generate the corresponding target text. First, the target context semantic vector is input and initialized with the decoder state so that the decoder can consider contextual information during generation. This allows the decoder to begin generating the target language in the correct context. The target context semantic vector and the decoder's initial state data are input to the decoder. The decoder's task is to gradually generate a sequence of words in the target text. At each time step, the decoder generates a new candidate word based on the current input (which can be a word generated in the previous time step or a special start token) and the current state. After generating each candidate word, the decoder state needs to be updated to generate the next candidate word at the next time step. This update process involves calculating state transitions and updating parameters within the decoder. In this step, you need to record the updated decoder state parameters for use in subsequent steps. Multiple candidate words and the collected updated state parameters are input to the decoder. The decoder uses these parameters to generate the next candidate word in the target text. This process repeats until multiple sub-contents are generated.

[0084] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0085] (1) Extract key information from multiple sub-contents respectively to obtain key information corresponding to each sub-content;

[0086] (2) Generate key clauses based on the key information corresponding to each sub-content to obtain multiple target key clauses;

[0087] (3) Performing grammatical logic matching on multiple target key clauses using preset grammatical rules to obtain the target grammatical logic;

[0088] (4) Generate SQL query statements for multiple target key clauses through target grammar logic to obtain target SQL query statements.

[0089] First, there are multiple target key clauses that have passed grammatical logic matching verification. These clauses contain the key information and conditions to be queried. A set of grammar rules is preset to convert the target key clauses into different parts of the SQL query. These rules may include rules related to syntactic structure, query operations, and condition filtering. For each target key clause, the preset grammar rules will be applied to parse it into different components of the SQL query. This includes the fields to be selected (SELECT), the data table (FROM), and the conditions (WHERE). Once each target key clause is parsed into different SQL query parts, these parts are combined to generate the final SQL query statement. This combination may involve connecting multiple query conditions, using logical operators, etc. The final generated SQL query statement will contain the information provided by all target key clauses and comply with SQL grammar rules. This query statement will be able to extract data that meets all key conditions from the database.

[0090] In a specific embodiment, the process of executing step S107 may specifically include the following steps:

[0091] (1) Match the target SQL query statement to the target database to obtain the target query database;

[0092] (2) Execute the target SQL query statement, query data from the target query database, and obtain the target query data;

[0093] (3) generating text from the target query data to obtain target text data;

[0094] (4) Transmit the target text data to a preset data display terminal.

[0095] It should be noted that an appropriate database should be selected based on the tables and conditions involved in the generated SQL query. This database should contain storage for the relevant data. Connect to the selected database to execute the SQL query in subsequent steps. This may require database connection parameters such as the host name, username, and password. Use an appropriate database tool or programming language to send the generated SQL query statement to the database server. After executing the SQL query, the database will return a result set. This result set contains the data that meets the query conditions and may be a set of records, rows, or fields. Format and process the data extracted from the database to convert it into readable text. This may include operations such as merging, separating, and transforming. Convert the processed data into natural language text. If the data is structured, it needs to be converted into meaningful sentences or paragraphs. Use appropriate communication methods to send the generated target text data to a pre-configured data display terminal. This may be transmitted via a network protocol (such as HTTP) or other communication methods. On the data display terminal, present the target text data in a user-friendly manner based on the user interface design. This may be in the form of a graphical interface, a web page, a mobile application, etc.

[0096] The present disclosure also provides a data analysis system based on natural semantics, as shown in FIG3 . The data analysis system based on natural semantics specifically includes:

[0097] An acquisition module 301 is configured to acquire text data to be processed, segment the text data to be processed using a preset subword segmentation algorithm to obtain multiple candidate word vectors for the text data to be processed, and position-encode the multiple candidate word vectors to obtain multiple target word vectors;

[0098] A conversion module 302 is configured to perform vector conversion on each target word vector to obtain a query vector for each target word vector, a key vector for each target word vector, and a value vector for each target word vector;

[0099] The calculation module 303 is configured to perform a dot product similarity calculation on the key vector of each target word vector based on the query vector of each target word vector to obtain a similarity set of each target word vector, and perform an attention weight calculation on the similarity set of each target word vector to obtain an attention weight of each target word vector;

[0100] A summing module 304 is configured to perform a weighted summation of the target word vectors based on the attention weight of each target word vector using the value vector of each target word vector to obtain a context semantic vector to be converted, and input the context semantic vector to be converted into a preset feedforward neural network for vector dimension conversion based on a preset vocabulary to obtain a target context semantic vector;

[0101] An input module 305 is configured to input the target context semantic vector into a preset decoder for content generation to obtain a plurality of sub-contents;

[0102] The generating module 306 is configured to generate key clauses from the plurality of sub-contents to obtain a plurality of target key clauses, and generate an SQL query statement from the plurality of target key clauses to obtain a target SQL query statement;

[0103] The transmission module 307 is configured to execute the target SQL query statement, collect corresponding target text data, and transmit the target text data to a preset data display terminal.

[0104] Through the collaborative work of the aforementioned modules, a pre-configured subword segmentation algorithm is used to segment the processed text, better capturing the text's meaning and structure and reducing ambiguity. The segmented text is converted into multiple candidate word vectors and positionally encoded, helping to preserve lexical and positional information. An attention mechanism is introduced into the vector conversion. By calculating the dot product similarity between the query, key, and value vectors, the target word is associated with contextually related words, highlighting important information and improving the extraction of contextual semantics. The target contextual semantic vector is obtained by weighted summing the multiple target word vectors using the attention weights. This helps capture the importance of the target word within the entire context and reduces information loss, thereby improving the quality of subsequent content generation and key clause generation. Using the target contextual semantic vector for content generation enables more coherent and appropriate sub-content based on the context. Generating target key clauses based on the sub-content helps extract the most critical and informative content from the text, providing a more accurate foundation for subsequent SQL query generation. Generating SQL queries based on the target key clauses converts natural language requirements into computer-understandable database query language, achieving a transition from semantics to operations. Executing the generated SQL query accurately retrieves the required data from the database, providing an efficient data retrieval method. Transmitting the acquired target text data to the data display terminal effectively connects the processing flow to the actual data. This helps present the processing results to the user in an easy-to-understand manner, providing data visualization and presentation.

[0105] Overall, this disclosed solution combines multiple steps, including word segmentation, attention mechanisms, content generation, key clause generation, SQL query, and data display. It effectively extracts useful information from the text data being processed and converts it into actual data query and display. This improves the automation and efficiency of text processing, reduces manual intervention, and enhances the accuracy and usability of data analysis. It also significantly increases the flexibility of analysis, eliminating the need for additional tools or code. As long as natural language can be clearly described, the corresponding SQL can be converted to perform data analysis.

[0106] The above embodiments are only used to illustrate the technical solutions of the present disclosure rather than to limit them. Although the present disclosure has been described in detail with reference to the embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present disclosure can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present disclosure should be included in the scope of the claims of the present disclosure.

Claims

1. A data analysis method based on natural semantics, comprising: Acquire text data to be processed, perform word segmentation on the text data to be processed by a preset subword segmentation algorithm to obtain multiple candidate word vectors of the text data to be processed, and position encode the multiple candidate word vectors to obtain multiple target word vectors; Performing vector conversion on each of the target word vectors to obtain a query vector of each of the target word vectors, a key vector of each of the target word vectors, and a value vector of each of the target word vectors; Based on the query vector of each of the target word vectors, perform dot product similarity calculation on the key vector of each of the target word vectors to obtain a similarity set of each of the target word vectors, and perform attention weight calculation on the similarity set of each of the target word vectors to obtain an attention weight of each of the target word vectors; Based on the attention weight of each target word vector, a weighted sum is performed on the multiple target word vectors through the value vector of each target word vector to obtain a context semantic vector to be converted, and the context semantic vector to be converted is input into a preset feedforward neural network to perform vector dimension conversion based on a preset vocabulary to obtain a target context semantic vector; Inputting the target context semantic vector into a preset decoder to generate content, thereby obtaining a plurality of sub-contents; Generate key clauses through the multiple sub-contents to obtain multiple target key clauses, and generate SQL query statements through the multiple target key clauses to obtain target SQL query statements; The target SQL query statement is executed, corresponding target text data is collected, and the target text data is transmitted to a preset data display terminal.

2. The data analysis method based on natural semantics according to claim 1, wherein: The step of obtaining the text data to be processed, performing word segmentation processing on the text data to be processed by a preset subword segmentation algorithm, obtaining a plurality of candidate word vectors of the text data to be processed, and position encoding the plurality of candidate word vectors to obtain a plurality of target word vectors includes: Acquire the text data to be processed, and construct an initial vocabulary using the text data to be processed; Based on the initial word list, iteratively segmenting the initial word list by the subword segmentation algorithm to obtain a plurality of candidate segmentation words; Performing adjacent character combination frequency analysis on the plurality of candidate segmentation words to obtain character combination frequencies; Update the initial vocabulary by using the character combination frequency to obtain a target vocabulary; Performing text segmentation on multiple candidate segmentation words through the target word list to obtain multiple target segmentation words; Performing real value vector conversion on each of the target segmented words to obtain multiple candidate word vectors of the text data to be processed; Perform position encoding on multiple candidate word vectors to obtain multiple target word vectors.

3. The data analysis method based on natural semantics according to claim 2, wherein: The position encoding of the multiple candidate word vectors to obtain multiple target word vectors includes: Performing position analysis on each of the target word vectors to obtain a first set of word vectors to be encoded at odd positions and a second set of word vectors to be encoded at even positions; Performing position encoding on each first word vector to be encoded in the first word vector set to be encoded by using a preset sinusoidal encoding function to obtain a first position encoding vector set; Performing position encoding on each second word vector to be encoded in the second word vector set to be encoded by using a preset cosine encoding function to obtain a second position encoding vector set; Performing vector fusion on the first position encoding vector set and the first to-be-encoded word vector set to obtain a first vector set; Performing vector fusion on the second position encoding vector set and the second to-be-encoded word vector set to obtain a second vector set; The first vector set and the second vector set are concatenated to obtain a plurality of target word vectors.

4. The data analysis method based on natural semantics according to claim 1, wherein: The query vector based on each of the target word vectors, performing dot product similarity calculation on the key vector of each of the target word vectors, obtaining a similarity set of each of the target word vectors, and performing attention weight calculation on the similarity set of each of the target word vectors, obtaining an attention weight of each of the target word vectors, including: Based on the query vector of each of the target word vectors, perform dot product similarity calculation on the key vector of each of the target word vectors to obtain a similarity set of each of the target word vectors; The similarity sets of each target word vector are normalized respectively to obtain the attention weight of each target word vector.

5. The data analysis method based on natural semantics according to claim 1, wherein: The target context semantic vector is input into a preset decoder for content generation to obtain a plurality of sub-contents, including: Based on the target context semantic vector, the decoder is initialized to obtain the Initial state data of the decoder; Inputting the target context semantic vector and the initial state data into the decoder to generate words, thereby obtaining a plurality of candidate words; updating the state of the decoder through the plurality of candidate words and collecting corresponding update state parameters; The plurality of candidate words and the update status parameters are input into the decoder to generate content, thereby obtaining a plurality of sub-contents.

6. The data analysis method based on natural semantics according to claim 1, wherein: The generating of key clauses through the plurality of sub-contents to obtain a plurality of target key clauses, and generating a SQL query statement through the plurality of target key clauses to obtain a target SQL query statement, includes: Extracting key information from the plurality of sub-contents respectively to obtain key information corresponding to each sub-content; Generate key clauses through key information corresponding to each of the sub-contents to obtain a plurality of target key clauses; Performing grammatical logic matching on the plurality of target key clauses through preset grammatical rules to obtain target grammatical logic; The target SQL query statement is generated for the plurality of target key clauses through the target grammar logic to obtain a target SQL query statement.

7. The data analysis method based on natural semantics according to claim 1, wherein: The executing the target SQL query statement, collecting corresponding target text data, and transmitting the target text data to a preset data display terminal includes: Performing query database matching on the target SQL query statement to obtain a target query database; Execute the target SQL query statement, perform data query from the target query database, and obtain target query data; Generating text for the target query data to obtain target text data; The target text data is transmitted to a preset data display terminal.

8. A natural semantics-based data analysis system, configured to execute the natural semantics-based data analysis method according to any one of claims 1 to 7, comprising: The acquisition module is configured to acquire the text data to be processed, perform word segmentation on the text data to be processed by a preset subword segmentation algorithm, obtain multiple candidate word vectors of the text data to be processed, and Perform position encoding on multiple candidate word vectors to obtain multiple target word vectors; A conversion module is configured to perform vector conversion on each of the target word vectors to obtain a query vector of each of the target word vectors, a key vector of each of the target word vectors, and a value vector of each of the target word vectors; A calculation module is configured to perform dot product similarity calculation on the key vector of each target word vector based on the query vector of each target word vector, to obtain a similarity set of each target word vector, and perform attention weight calculation on the similarity set of each target word vector to obtain an attention weight of each target word vector; A summing module is configured to perform weighted summation of the target word vectors based on the attention weight of each target word vector through the value vector of each target word vector to obtain a context semantic vector to be converted, and input the context semantic vector to be converted into a preset feedforward neural network to perform vector dimension conversion based on a preset vocabulary to obtain a target context semantic vector; An input module, configured to input the target context semantic vector into a preset decoder for content generation to obtain a plurality of sub-contents; A generating module is configured to generate key clauses through the plurality of sub-contents to obtain a plurality of target key clauses, and to generate SQL query statements through the plurality of target key clauses to obtain a target SQL query statement; The transmission module is configured to execute the target SQL query statement, collect corresponding target text data, and transmit the target text data to a preset data display terminal.

Images