Order data aggregation method, system and device
Through Text2SQL algorithm and vectorization processing, natural language query is converted into structured query language, the pre- and post-verbal relationship query intervals of variable characters are calculated, and SQL statements are gathered, which solves the problems of inefficiency and semantic ambiguity in traditional systems, and achieves efficient, Accurate order data management and query.
Patent Information
- Application Number
- CN202411171634.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-26
AI Technical Summary
Traditional order data management and query systems are inefficient. Users need to manually write SQL statements, which are prone to syntax or logical errors, and natural language queries are prone to semantic ambiguity, resulting in inaccurate query results and long response time, which cannot fundamentally solve the bottleneck of query performance.
The Text2SQL algorithm converts natural language query into a structured query language, and uses vectorization and parallel processing to calculate the query interval of the front and back language relationship of variable characters, gather structured query statements, and optimize the processing of SQL statements.
It realizes efficient, accurate and flexible order data aggregation and query management, reduces database response time and maintenance costs, and improves system performance.
Smart Images

Figure CN118689904B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of distributed big data processing, and in particular to an order data aggregation method, system and device. Background Art
[0002] In modern information systems, the management and query of order data is a crucial link. With the rapid development of e-commerce, the amount of order data has increased dramatically, and traditional order data management systems face many challenges. The following is the status quo of traditional order data management and query technologies and the main problems they have. In many traditional systems, users need to manually write SQL statements to query and manage order data. This requires users to have certain SQL knowledge and skills. Manually writing SQL statements is time-consuming and laborious, especially when querying complex data. Users are prone to syntax errors or logical errors when writing SQL statements, resulting in query failures or inaccurate results. Many users lack the ability to write SQL statements, which limits the availability of the system. Some systems also provide predefined query interfaces, such as a natural language-based database query method and system described in the patent document with publication number CN115827674A, which allows users to obtain data through simple parameterized queries, but predefined query interfaces usually cannot meet complex and diverse query requirements. Whenever a new query function is needed, the system needs to be developed and deployed accordingly, which increases maintenance costs. In recent years, with the development of natural language processing (NLP) technology, natural language-based query systems have begun to appear, allowing users to use natural language for queries. However, natural language queries are prone to semantic ambiguity, resulting in inaccurate query results. Among them, the process of converting natural language into SQL is complicated, which can easily lead to long system response time, especially during peak hours. In order to improve query performance, traditional systems use technologies such as indexing, caching and query optimization, such as the multi-threaded concurrent processing method and multi-threaded concurrent processing system described in the patent document with publication number CN107193539B. However, these technologies can only improve query efficiency to a certain extent and cannot fundamentally solve the query performance bottleneck. In addition, indexes and caches need to be constantly updated and maintained, which will also increase the complexity of the system. Summary of the invention
[0003] The purpose of the present invention is to propose an order data aggregation method, system and device to solve one or more technical problems existing in the prior art and at least provide a beneficial choice or create conditions.
[0004] In order to achieve the above objective, according to one aspect of the present disclosure, a method for aggregating order data is provided, the method comprising the following steps:
[0005] The database opens multiple interfaces to allow external users to query and write order data, receive natural language queries, and convert them into structured query language through the Text2SQL algorithm;
[0006] Each structured query is segmented, and after extracting grammatical characters and variable characters, the variable characters are vectorized to form a variable character vector sequence, and each variable character vector is divided into a pre-language character vector and a post-language character vector, and the pre-language relationship distance and the post-language relationship distance are calculated; the cosine similarity between each variable character vector and its pre-language relationship distance and post-language relationship distance is calculated, and the pre-language relationship distance chord value and the post-language relationship distance chord value are extracted therefrom, and the pre-language relationship distance chord value and the post-language relationship distance chord value in the variable character vector sequence are combined to form a pre-language relationship query interval according to the numerical value;
[0007] Using the context relationship query interval of each structured query statement, the structured query statements are aggregated to obtain a number of structured query statement data packets;
[0008] The database processes the structured query statements in each structured query statement data packet respectively.
[0009] In the data request, the SQL statement is composed of strings separated by spaces. The SQL statement can be segmented to obtain several grammatical characters and variable characters. If a SQL query statement is split by spaces, multiple elements can be obtained. These elements can be divided into grammatical characters and variable characters. Among them, grammatical characters refer to characters preset by the database such as SELECT, DELETE, WHERE and FROM, and variable characters refer to user-defined characters such as column names and table names in the table in the database. Since the query operation at least includes the table name of the data request table and at least one column name therein and the constraints related to the column name, each SQL statement contains multiple different variable characters.
[0010] Furthermore, each structured query statement is segmented, and the variable characters therein after the segmentation are selected in order to form a variable character sequence; in each variable character sequence, each variable character is vectorized to obtain a variable character vector to form a variable character vector sequence.
[0011] Preferably, in some embodiments in which each variable character is vectorized to obtain a variable character vector, each variable character can be word-embedded using pre-trained word vectors such as GloVe to obtain a variable character vector corresponding to each variable character.
[0012] Furthermore, the preceding and following relationship query intervals are formed according to the numerical values of the preceding and following relationship distance chord values in the variable character vector sequence, specifically:
[0013] In each variable character vector sequence, respectively calculate the preceding relationship distance chord value and the succeeding relationship distance chord value of each variable character vector, and then calculate the product of the preceding relationship distance chord value and the succeeding relationship distance chord value of each variable character vector as the preceding and succeeding relationship value of the variable character vector;
[0014] Selecting the minimum value of the context relation values of each variable character vector in the variable character vector sequence as the context relation valley value in the variable character vector sequence, and selecting the maximum value of the context relation values of each variable character vector in the variable character vector sequence as the context relation peak value in the variable character vector sequence;
[0015] In each variable character vector sequence, an interval consisting of the valley value of the context relationship as the lower limit and the peak value of the context relationship as the upper limit is used as the context relationship query interval of the original structured query statement corresponding to the variable character vector sequence.
[0016] Furthermore, by dividing each variable character vector into a preceding language character vector and a succeeding language character vector, the preceding language relation distance and the succeeding language relation distance are calculated, specifically:
[0017] In each variable character vector sequence, if a variable character vector is preceded by a variable character vector, the variable character vector preceding it is called the predecessor character vector of the variable character vector.
[0018] Subtract the variable character vector from its predecessor character vector to obtain the predecessor relationship distance between the predecessor character vector and its predecessor character vectors.
[0019] If a variable character vector is followed by another variable character vector, the variable character vector following it is called the suffix character vector of the variable character vector.
[0020] The variable character vector is subtracted from its subsequent language character vector to obtain the subsequent language relationship distance between the subsequent language character vector and each subsequent language character vector.
[0021] For a variable character vector without a preceding character vector, a vector of all zeros with the same dimension size can be added to it as its preceding relationship distance; for a variable character vector without a succeeding character vector, a vector of all zeros with the same dimension size can be added to it as its succeeding relationship distance.
[0022] Furthermore, the cosine similarity between each variable character vector and its preceding and succeeding language relation distances is calculated, and the preceding and succeeding language relation distance chord values and the succeeding language relation distance chord values are extracted therefrom, specifically:
[0023] Calculate the cosine similarity between each variable character vector and its preceding relation distance, and select the cosine similarity with the largest value as the preceding relation chord value of the variable character vector; calculate the cosine similarity between each variable character vector and its succeeding relation distance, and select the cosine similarity with the largest value as the succeeding relation chord value of the variable character vector.
[0024] Furthermore, the structured query statements are aggregated using the context relationship query intervals of each structured query statement to obtain a number of structured query statement data packets, specifically:
[0025] After receiving multiple structured query statements, the database divides them into several receiving time periods according to the receiving time, marks the structured query statements corresponding to each receiving time period, and obtains the context relationship query interval of each structured query statement;
[0026] According to the overlap ratio between the query intervals of the context relationship of the structured query statements in the same receiving time period, classification or clustering is performed to obtain a number of structured query statement data packets;
[0027] The database processes the structured query statements in each structured query statement data packet respectively.
[0028] The present disclosure also provides an order data aggregation system, the order data aggregation system comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor implementing the steps in the order data aggregation method when executing the computer program, the order data aggregation system can be run in computing devices such as desktop computers, notebook computers, PDAs, and cloud data centers, and the executable system may include, but is not limited to, a processor, a memory, and a server cluster, and the processor executing the computer program runs in the following system units:
[0029] A receiving and converting unit, used for receiving a natural language query and converting it into a structured query language;
[0030] A vectorization unit is used to segment each structured query, extract grammatical characters and variable characters, vectorize the variable characters to form a variable character vector sequence, divide each variable character vector into a preceding language character vector and a succeeding language character vector, and calculate the preceding language relationship distance and the succeeding language relationship distance;
[0031] An interval calculation unit is used to calculate the cosine similarity between each variable character vector and its preceding and succeeding language relationship distance values, and extract the preceding and succeeding language relationship distance chord values therefrom, and combine the preceding and succeeding language relationship distance chord values in the variable character vector sequence to form a preceding and succeeding language relationship query interval according to the numerical values;
[0032] The aggregation unit is used to aggregate the structured query statements using the context relationship query intervals of each structured query statement to obtain a number of structured query statement data packets.
[0033] Correspondingly, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product:
[0034] An electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the order data aggregation method and the steps therein.
[0035] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the order data aggregation method and the steps therein.
[0036] A computer program product includes a computer program, which, when executed by a processor, implements the order data aggregation method and the steps thereof.
[0037] The beneficial effects of the present disclosure are as follows: the present disclosure provides an order data aggregation method, system and device, wherein the database opens multiple interfaces to allow external users to perform query and write operations, receives natural language queries, and converts them into structured query language through the Text2SQL algorithm; by performing word segmentation on each structured query, extracting grammatical characters and variable characters, vectorizing the variable characters to form a variable character vector sequence, and dividing each variable character vector into a pre-language character vector and a post-language character vector, calculating the pre-language relationship distance and the post-language relationship distance; calculating the cosine similarity between each variable character vector and its pre-language relationship distance and post-language relationship distance, respectively, and extracting the pre-language relationship distance chord value and the post-language relationship distance chord value therefrom, and combining the pre-language relationship distance chord value and the post-language relationship distance chord value in the variable character vector sequence to form a context relationship query interval according to the numerical value; using the context relationship query interval of each structured query statement, the structured query statements are aggregated to obtain a number of structured query statement data packets. The present invention solves the problems of low efficiency, semantic ambiguity and performance bottleneck in traditional order data management and query systems through optimization of natural language to SQL, vectorized processing, parallel processing and flexible query classification and clustering, and realizes efficient, accurate and flexible order data aggregation and query management. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The above and other features of the present disclosure will become more apparent by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present disclosure represent the same or similar elements. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:
[0039] Figure 1 Shown is a flow chart of an order data aggregation method;
[0040] Figure 2 Shown is a system structure diagram of an order data aggregation system. DETAILED DESCRIPTION
[0041] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present disclosure, so as to fully understand the purpose, scheme and effect of the present disclosure. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.
[0042] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0043] like Figure 1 The flowchart of a method for aggregating order data according to the present invention is shown below. Figure 1 To illustrate an order data aggregation method, system and device according to an embodiment of the present invention.
[0044] The present disclosure proposes a method for aggregating order data, which specifically includes the following steps:
[0045] The database opens multiple interfaces to allow external users to query and write order data, receive natural language queries and convert them into structured query language;
[0046] Each structured query is segmented, grammatical characters and variable characters are extracted, and then the variable characters are vectorized to form a variable character vector sequence. Each variable character vector is divided into a pre-language character vector and a post-language character vector, and the pre-language relationship distance and the post-language relationship distance are calculated.
[0047] Calculate the cosine similarity between each variable character vector and its preceding relation distance value and succeeding relation distance value, extract the preceding relation distance chord value and succeeding relation distance chord value from them, and combine the preceding relation distance chord value and succeeding relation distance chord value in the variable character vector sequence to form a preceding and succeeding relation query interval according to the numerical value;
[0048] The structured query statements are aggregated using the context relationship query intervals of each structured query statement to obtain a number of structured query statement data packets.
[0049] In some embodiments, multiple tables are stored in a distributed database storage access system, each table contains a shared and mutually referenced data relationship, and the required data is usually written in a database system. There are multiple different tables in a database system, and there are shared and mutually referenced relationships between the data in each table. A database system opens one or more API interfaces for external users to query and write. In order to speed up the efficiency of query statement writing and query, external users can enter natural language query statements at the interface terminal, and then each natural language query statement is converted into each structured query language (SQL statement) through the Text2SQL algorithm, and the API interface is accessed with each SQL statement, thereby representing a data request with one SQL query statement. Such an operation can facilitate users to quickly query a large amount of data without having to manually write SQL statements, especially during peak data query periods, which can greatly improve efficiency. However, it is gradually found in current use that although it is simple, the conversion of natural language into structured query language will inevitably cause redundant parts in the generated structured query language due to semantic ambiguity, which will increase the running time and computing power cost of database response, and easily lead to a longer database response time or even freeze. Order data refers to the data of service request orders that add, delete, and modify the data in the tables in the database, including various structured query statements.
[0050] In a data request, an SQL statement consists of strings separated by spaces. The SQL statement can be segmented to obtain a number of grammatical characters and variable characters. If a SQL query statement is split by spaces, multiple elements can be obtained. These elements can be divided into grammatical characters and variable characters. Among them, grammatical characters refer to characters preset by the database, such as SELECT, DELETE, WHERE, and FROM, and variable characters refer to user-defined characters such as column names and table names in tables in the database. Since the query operation at least includes the table name of the table requested for data and at least one column name therein and the constraints related to the column name, each SQL statement contains multiple different variable characters.
[0051] Furthermore, each structured query statement is segmented, and the variable characters therein after the segmentation are selected in order to form a variable character sequence; in each variable character sequence, each variable character is vectorized to obtain a variable character vector to form a variable character vector sequence.
[0052] In some embodiments where each variable character is vectorized to obtain a variable character vector, a pre-trained word vector such as GloVe can be used to embed each variable character to obtain a variable character vector corresponding to each variable character;
[0053] In some preferred embodiments, a pre-trained language model such as BERT may be used to align the variable character sequence with the tokens in the SQL statement to which it originally corresponds, and then embed them together, and then obtain the corresponding embedding vector for each variable character as the variable character vector corresponding to each variable character;
[0054] In some less preferred embodiments, the Elmo word embedding algorithm can also be used to first embed the SQL statement originally corresponding to the variable character sequence, and then compare the variable characters in the variable character sequence through a text similarity comparison algorithm, so that each variable character obtains a corresponding Elmo word embedding vector as the variable character vector corresponding to each variable character.
[0055] Through word vectorization and semantic similarity calculation, natural language queries are accurately converted into SQL statements, improving the accuracy of queries.
[0056] Furthermore, the preceding and following relationship query intervals are formed according to the numerical values of the preceding and following relationship distance chord values in the variable character vector sequence, specifically:
[0057] In each variable character vector sequence, respectively calculate the preceding relationship distance chord value and the succeeding relationship distance chord value of each variable character vector, and then calculate the product of the preceding relationship distance chord value and the succeeding relationship distance chord value of each variable character vector as the preceding and succeeding relationship value of the variable character vector;
[0058] Selecting the minimum value of the context relation values of each variable character vector in the variable character vector sequence as the context relation valley value in the variable character vector sequence, and selecting the maximum value of the context relation values of each variable character vector in the variable character vector sequence as the context relation peak value in the variable character vector sequence;
[0059] In each variable character vector sequence, an interval consisting of the valley value of the context relationship as the lower limit and the peak value of the context relationship as the upper limit is used as the context relationship query interval of the original structured query statement corresponding to the variable character vector sequence.
[0060] The context relation value is a condensed expression of the probability distribution of the existence of a triple logical relationship in the front and back of the variable character vector in the structured query statement. In fact, the data distribution in the database has been projected into a vector space based on the semantics of the structured query statement. The context relation value can be used as an anchor point for the structured query statement to be projected in the database, and the context relation query interval of the structured query statement is a designated area projected in the database, which can quickly indicate the spatial direction of the convergence for the structured query statement. The present invention introduces vectorization processing, embeds the variable characters in the SQL statement, and uses cosine similarity calculation to optimize the query, thereby reducing the database response time. By calculating the context relation distance and the context relation query interval, efficient convergence and processing of SQL statements are achieved, further improving the query performance.
[0061] Furthermore, by dividing each variable character vector into a preceding language character vector and a succeeding language character vector, the preceding language relation distance and the succeeding language relation distance are calculated, specifically:
[0062] In each variable character vector sequence, if a variable character vector is preceded by a variable character vector, the variable character vector preceding it is called the predecessor character vector of the variable character vector.
[0063] Subtract the variable character vector from its predecessor character vector to obtain the predecessor relationship distance between the predecessor character vector and its predecessor character vectors.
[0064] If a variable character vector is followed by another variable character vector, the variable character vector following it is called the suffix character vector of the variable character vector.
[0065] The variable character vector is subtracted from its subsequent language character vector to obtain the subsequent language relationship distance between the subsequent language character vector and each subsequent language character vector.
[0066] In some embodiments, for example, when the variable character vector sequence includes five variable character vectors, namely v1, v2, v3, v4 and v5, if the variable character vector is selected as v3, then v1 and v2 are both the pre-language character vectors of v3, and v4 and v5 are the post-language character vectors of v3, the pre-language character vector represents the character vector serialized before the selected target in the structured query statement, and the post-language character vector represents the character vector serialized after the selected target in the structured query statement. If the variable character vector is selected as v1, the pre-language character vector of v1 does not exist, and if there is no pre-language character vector, the subsequent values related to the pre-language character vector are correspondingly set to zero; if the variable character vector is selected as v5, the post-language character vector of v5 does not exist, and if there is no post-language character vector, the subsequent values related to the post-language character vector are correspondingly set to zero.
[0067] For v3, the vector obtained by subtracting v3 from v1 and v2 point by point in dimension is calculated as the pre-language relation distance of v3, and the vector obtained by subtracting v4 and v2 from v3 point by point in dimension is calculated as the post-language relation distance of v3. A variable character vector has as many pre-language relation distances as its pre-language character vectors, and a variable character vector has as many post-language relation distances as its post-language character vectors.
[0068] For a variable character vector without a preceding character vector, a full-zero vector with the same dimension size is added to it as its preceding relationship distance; for a variable character vector without a succeeding character vector, a full-zero vector with the same dimension size is added to it as its succeeding relationship distance.
[0069] Furthermore, the cosine similarity between each variable character vector and its preceding and succeeding language relation distances is calculated, and the preceding and succeeding language relation distance chord values and the succeeding language relation distance chord values are extracted therefrom, specifically:
[0070] Calculate the cosine similarity between each variable character vector and its preceding relation distance, and select the cosine similarity with the largest value as the preceding relation chord value of the variable character vector; calculate the cosine similarity between each variable character vector and its succeeding relation distance, and select the cosine similarity with the largest value as the succeeding relation chord value of the variable character vector.
[0071] In the theory of vectorized embedding of knowledge graphs, the semantic vector of the preceding subject combined with the semantic vector of the entity relationship can be mapped to the semantic vector of the following object, and the knowledge graph is a graphical representation of a semantically structured query database. Calculating the cosine similarity between each variable character vector and its preceding relationship distance and following relationship distance is helpful to measure the probability distribution of the existence of a triple logical relationship between the selected variable character vector in the structured query statement. In the past, such probability distribution extraction could be achieved using repeated matrix decomposition in the attention mechanism. However, the preceding relationship distance and following relationship distance in the method described in the present invention are operated using addition (because they can be regarded as negative numbers added). Compared with the large-scale vector multiplication-style feature extraction in the attention mechanism, the calculation method of the preceding relationship distance and following relationship distance in the method described in the present invention is faster but more appropriate for the feature representation of structured query statements, and is more suitable for solving practical scenarios of rapid database response.
[0072] In some less preferred embodiments, an attention mechanism network such as a Transformer model structure can also be used to calculate the attention value between each selected variable character vector and its preceding language relationship distance as its preceding language relationship distance chord value, and calculate the attention value between each selected variable character vector and its succeeding language relationship distance as its succeeding language relationship distance chord value. However, in actual operation, it is found that since the attention layer must first perform matrix decomposition on each preceding language relationship distance and each succeeding language relationship distance into three feature vectors of K, Q and V when calculating the attention matrix, it is time-consuming and will also increase the response load of the database. Although it can be used in theory, it is not as robust as in the preferred embodiment above.
[0073] Furthermore, the structured query statements are aggregated using the context relationship query intervals of each structured query statement to obtain a number of structured query statement data packets, specifically:
[0074] After receiving multiple structured query statements, the database divides them into several receiving time periods according to the receiving time, marks the structured query statements corresponding to each receiving time period, and obtains the context relationship query interval of each structured query statement;
[0075] According to the overlap ratio between the query intervals of the context relationship of the structured query statements in the same receiving time period, classification or clustering is performed to obtain a number of structured query statement data packets;
[0076] The database processes the structured query statements in each structured query statement data packet respectively.
[0077] In some embodiments, the structured query statements may be aggregated by a classification algorithm or a clustering algorithm according to the overlap ratio between the query intervals of the context relationship of the structured query statements within the same receiving time period to obtain a number of structured query statement data packets, wherein for technical scenarios where the number of divisions of the structured query statement data packets is fixed, a supervised learning classification algorithm (including but not limited to KNN) may be used; wherein, for technical scenarios where the number of divisions of the structured query statement data packets is not fixed, an unsupervised learning clustering algorithm (including but not limited to K-Means) may be used.
[0078] In some embodiments, the overlap ratio between the context relationship query intervals of the structured query statement can be the overlapping range between the two intervals (represented as overlapping_area in the code), or the percentage of the overlapping part between the first context relationship query interval and the second context relationship query interval in the first context relationship query interval (represented as ratio in the code).
[0079] In some embodiments, the code implementation part of the algorithm described in the present invention can be implemented using Python 3.0 or above, but it does not exclude the use of Scala language for editing. The database part can use database applications including but not limited to DBpedia, Oracle, Neo4J and TigerGraph, and the parallel processing part can use workflow management tools (Open-source Workflow Manager) such as Azkaban assisted by ZooKeeper for processing to improve the distributed database to perform synchronous and / or asynchronous processing of structured query statements in each structured query statement data packet. The present invention uses workflow management tools (such as Azkaban) and distributed systems (such as ZooKeeper) to achieve synchronous and asynchronous processing of large-scale order data, significantly improving the scalability and processing efficiency of the system. In this way, by processing SQL statement data packets in parallel, the query load of a single database node is reduced, and the overall performance of the system is improved.
[0080] The order data aggregation system runs on any computing device such as a desktop computer, a laptop computer, a PDA or a cloud data center. The computing device includes: a processor, a memory and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps in the order data aggregation method are implemented. The executable system may include, but is not limited to, a processor, a memory, and a server cluster.
[0081] An order data aggregation system provided by an embodiment of the present disclosure is as follows: Figure 2 As shown, an order data aggregation system of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned order data aggregation method embodiment are implemented. The processor executes the computer program to run in the following system units:
[0082] A receiving and converting unit, used for receiving a natural language query and converting it into a structured query language;
[0083] A vectorization unit is used to segment each structured query, extract grammatical characters and variable characters, vectorize the variable characters to form a variable character vector sequence, divide each variable character vector into a preceding language character vector and a succeeding language character vector, and calculate the preceding language relationship distance and the succeeding language relationship distance;
[0084] An interval calculation unit is used to calculate the cosine similarity between each variable character vector and its preceding and succeeding language relationship distance values, and extract the preceding and succeeding language relationship distance chord values therefrom, and combine the preceding and succeeding language relationship distance chord values in the variable character vector sequence to form a preceding and succeeding language relationship query interval according to the numerical values;
[0085] The aggregation unit is used to aggregate the structured query statements using the context relationship query intervals of each structured query statement to obtain a number of structured query statement data packets.
[0086] Among them, in order to better unify the numerical linear relationship and probabilistic connection between physical quantities of different units of measurement, different physical quantities can be dimensionally processed.
[0087] Among them, preferably, all undefined variables in the present invention, if not clearly defined, can be manually set thresholds.
[0088] The order data aggregation system can be run in computing devices such as desktop computers, laptop computers, PDAs, and cloud data centers. The order data aggregation system includes, but is not limited to, a processor and a memory. Those skilled in the art can understand that the example is only an example of an order data aggregation method, system, and device, and does not constitute a limitation of an order data aggregation method, system, and device, and may include more or less components than the example, or a combination of certain components, or different components. For example, the order data aggregation system may also include input and output devices, network access devices, buses, etc.
[0089] The present disclosure also provides an electronic device, a readable storage medium, and a computer program product:
[0090] An electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the order data aggregation method and the steps therein.
[0091] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the order data aggregation method and the steps therein.
[0092] A computer program product includes a computer program, which, when executed by a processor, implements the order data aggregation method and the steps thereof.
[0093] Wherein, electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0094] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0095] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0096] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0097] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0098] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0099] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.
[0100] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete component gate circuits or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the order data aggregation system, and uses various interfaces and lines to connect the various sub-areas of the entire order data aggregation system.
[0101] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the order data aggregation method, system and device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0102] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0103] The present disclosure provides an order data aggregation method, system and device, which convert natural language queries into structured query languages, vectorize variable characters to form a variable character vector sequence, divide each variable character vector into a preceding language character vector and a succeeding language character vector, calculate the preceding language relationship distance and the succeeding language relationship distance; calculate the cosine similarity between each variable character vector and its preceding language relationship distance and succeeding language relationship distance, respectively, extract the preceding language relationship distance chord value and the succeeding language relationship distance chord value, combine the preceding language relationship distance chord value and the succeeding language relationship distance chord value in the variable character vector sequence to form a preceding language relationship query interval according to the numerical value, aggregate structured query statements, and obtain a plurality of structured query statement data packets, thereby realizing efficient, accurate and flexible order data aggregation and query management.
[0104] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A method for aggregating order data, It is characterized in that The method comprises: The database opens multiple interfaces to allow external users to perform order data query and write operations, receives natural language queries and converts them into structured query language; performs word segmentation on each structured query, extracts grammatical characters and variable characters, and then vectorizes the variable characters to form a variable character vector sequence, and calculates the pre-language relationship distance and the post-language relationship distance by dividing each variable character vector into a pre-language character vector and a post-language character vector; calculates the cosine similarity between each variable character vector and its pre-language relationship distance and post-language relationship distance, and extracts the pre-language relationship distance chord value and the post-language relationship distance chord value, and combines the pre-language relationship distance chord value and the post-language relationship distance chord value in the variable character vector sequence to form a pre-language relationship query interval according to the numerical value; uses the pre-language relationship query interval of each structured query statement to aggregate the structured query statements to obtain a number of structured query statement data packets; Among them, the preceding and following relationship query intervals are formed according to the numerical values in combination with the preceding and following relationship distance chord values in the variable character vector sequence, specifically: In each variable character vector sequence, respectively calculate the preceding relationship distance chord value and the succeeding relationship distance chord value of each variable character vector, and then calculate the product of the preceding relationship distance chord value and the succeeding relationship distance chord value of each variable character vector as the preceding and succeeding relationship value of the variable character vector; Selecting the minimum value of the context relation values of each variable character vector in the variable character vector sequence as the context relation valley value in the variable character vector sequence, and selecting the maximum value of the context relation values of each variable character vector in the variable character vector sequence as the context relation peak value in the variable character vector sequence; In each variable character vector sequence, the interval consisting of the valley value of the context relationship as the lower limit and the peak value of the context relationship as the upper limit is used as the context relationship query interval of the original structured query statement corresponding to the variable character vector sequence; Among them, by dividing each variable character vector into a preceding language character vector and a succeeding language character vector, the preceding language relation distance and the succeeding language relation distance are calculated, specifically: In each variable character vector sequence, if a variable character vector is preceded by a variable character vector, the variable character vector preceding it is called the predecessor character vector of the variable character vector. Subtract the variable character vector from its predecessor character vector to obtain the predecessor relationship distance between the predecessor character vector and its predecessor character vectors. If a variable character vector is followed by another variable character vector, the variable character vector following it is called the suffix character vector of the variable character vector. The variable character vector is subtracted from its subsequent language character vector to obtain the subsequent language relationship distance between the subsequent language character vector and each subsequent language character vector; Among them, the cosine similarity between each variable character vector and its preceding and succeeding language relation distances is calculated, and the preceding and succeeding language relation distance chord values are extracted from them, specifically: Calculate the cosine similarity between each variable character vector and its preceding relation distance, and select the cosine similarity with the largest value as the preceding relation chord value of the variable character vector; calculate the cosine similarity between each variable character vector and its succeeding relation distance, and select the cosine similarity with the largest value as the succeeding relation chord value of the variable character vector.
2. The order data aggregation method according to claim 1, It is characterized in that in, Each structured query statement is segmented, and the variable characters after the segmentation are selected in order to form a variable character sequence; in each variable character sequence, each variable character is vectorized to obtain a variable character vector to form a variable character vector sequence.
3. A method for aggregating order data according to any one of claims 1 to 2, It is characterized in that in, The structured query statements are aggregated using the context relationship query interval of each structured query statement to obtain several structured query statement data packets, specifically: After receiving multiple structured query statements, the database divides them into several receiving time periods according to the receiving time, marks the structured query statements corresponding to each receiving time period, and obtains the context relationship query interval of each structured query statement; According to the overlap ratio between the query intervals of the context relationship of the structured query statements in the same receiving time period, classification or clustering is performed to obtain a number of structured query statement data packets; The database processes the structured query statements in each structured query statement data packet respectively.
4. An order data aggregation system, It is characterized in that The order data aggregation system runs in any computing device such as a desktop computer, a laptop computer or a cloud data center. The computing device includes: a processor, a memory and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps in the order data aggregation method as described in any one of claims 1 to 2 are implemented.
5. An electronic device, include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 2.
