Cross-table flow-oriented computing method and system, electronic equipment and medium

Abstract

The invention provides a cross-table flow-oriented computing method and system, electronic equipment and a medium, and is applied to a cross-table flow-oriented computing system, and the cross-table flow-oriented computing system comprises a client and a server; the method comprises the steps that grammar verification is conducted on a preset SQL statement through a client side, and after verification is passed, statement content is generated based on the SQL statement and sent to a server side; wherein the SQL statement is used for performing streaming computation on the time series data in different cache tables; and creating a streaming computing task by creating streaming computing information on the basis of the statement content through the server, executing the streaming computing task, and reading time sequence data from different cache tables to perform streaming computing to obtain a streaming computing result. The workload of background data acquisition can be reduced, and the data processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of stream computing technology, and in particular to a cross-table stream computing method, system, electronic device and medium. Background Art

[0002] With the rapid popularization of the Internet of Things (IoT), devices such as sensors and smart devices continuously generate a large amount of time-stamped data. These data cover a variety of types such as temperature, humidity, pressure, vibration, etc., and have extremely high real-time and time-series correlation, which poses a severe challenge to traditional data storage and processing methods. Therefore, time series databases came into being, realizing efficient and real-time processing of massive time series data. However, traditional database systems are difficult to effectively process such data because they need to wait for the data to accumulate to a certain amount before batch processing, which cannot meet the real-time requirements of the Internet of Things. Therefore, the combination of time series databases and streaming computing technology has become the key to solving this problem.

[0003] At present, stream computing usually requires the help of external systems or tools. For example: Apache Kafka Streams, Apache Flink, AWS Kinesis, etc. Considering that more and more application scenarios require real-time processing and analysis of data streams, the traditional batch processing mode usually has a large delay and cannot achieve a response in seconds or even milliseconds, so it cannot meet these scenarios with high real-time requirements.

[0004] In this case, new requirements are put forward for the streaming computing technology of time series databases. In the time series database, when processing multiple tables and multiple databases, a single streaming query cannot meet the requirements. If the amount of data is large, the processing is relatively slow, and the client needs to occupy the database connection all the time, which is likely to cause network congestion. In addition, on the server side, there will be an LRU cache to store the index of the data file, which can locate the data block very quickly and execute streaming tasks asynchronously. However, in some scenarios, it is necessary to perform some calculation operations on two or more tables. For example, the scenario is to calculate the difference in electric energy consumption of two circuit tables, or to calculate the difference in water level between the upstream and downstream of a river. The traditional method is generally to collect data separately, and then the background performs calculation operations on the acquired data. The workload of the background to obtain data is large, and the data processing efficiency is low. Summary of the invention

[0005] In view of this, an object of the present invention is to provide a cross-table streaming computing method, system, electronic device and medium, which can reduce the workload of background data acquisition and improve data processing efficiency.

[0006] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0007] In a first aspect, an embodiment of the present invention provides a cross-table streaming computing method, which is applied to a cross-table streaming computing system, and the cross-table streaming computing system includes: a client and a server; the method includes: performing a syntax check on a pre-set SQL statement by the client, and after the check is passed, generating a statement content based on the SQL statement and sending it to the server; wherein the SQL statement is used to perform streaming computing on time series data in different cache tables; creating a streaming computing task based on the statement content by the server, and executing the streaming computing task, reading the time series data from different cache tables for streaming computing to obtain a streaming computing result.

[0008] In one implementation, the client performs syntax verification on the preset SQL statement, including: verifying the time information, function information, and table information of the SQL statement.

[0009] In one implementation, before generating statement content based on the SQL statement and sending it to the server, it also includes: obtaining metadata information from the database through the client and checking the metadata information.

[0010] In one embodiment, statement content is generated based on the SQL statement and sent to the server, including: serializing the streaming calculation name, super table information, table information, function information and corresponding column information, generating statement content, and sending the statement content to the server.

[0011] In one implementation, before creating a streaming computing task based on the statement content through the server, the method further includes: deserializing the statement content through the server, and verifying the library and table permissions of the information obtained through the deserialization.

[0012] In one embodiment, a streaming computing task is created based on the statement content by a server, and the streaming computing task is executed, and the time series data is read from different cache tables for streaming computing to obtain the streaming computing result, including: the server creates streaming computing information and a super table sub-table structure based on the statement content, and generates a streaming computing task; the streaming computing task is added to a task list, and the streaming computing task is executed, and the time series data is read from different cache tables for streaming computing to obtain the streaming computing result; and the streaming computing result is saved in the super table sub-table structure.

[0013] In one embodiment, a streaming computing task is executed, and time series data is read from different cache tables to perform streaming computing to obtain a streaming computing result, including: determining the time information of the streaming computing task, and obtaining the corresponding data file from the database based on the time information; obtaining the corresponding data block from the data file based on the UID information of the cache table, and performing function calculation on the data block to obtain the streaming computing result.

[0014] In the second aspect, an embodiment of the present invention provides a cross-table streaming computing system, including: a client and a server; the client is used to perform syntax verification on a pre-set SQL statement, and after the verification is passed, generate statement content based on the SQL statement and send it to the server; wherein the SQL statement is used to perform streaming computing on time series data in different cache tables; the server is used to create a streaming computing task based on the statement content, and execute the streaming computing task, read the time series data from different cache tables for streaming computing to obtain the streaming computing results.

[0015] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the steps of any one of the methods provided in the first aspect above.

[0016] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods provided in the first aspect are executed.

[0017] The embodiments of the present invention bring the following beneficial effects:

[0018] The above-mentioned cross-table streaming computing method, system, electronic device and medium provided by the embodiments of the present invention are applied to a cross-table streaming computing system, and the cross-table streaming computing system includes: a client and a server; the method includes: performing syntax check on a pre-set SQL statement through the client, and after the check is passed, generating statement content based on the SQL statement and sending it to the server; wherein the SQL statement is used to perform streaming computing on time series data in different cache tables; creating a streaming computing task based on the statement content through the server, and executing the streaming computing task, reading time series data from different cache tables for streaming computing to obtain streaming computing results. The above-mentioned method realizes cross-table streaming computing through pre-set SQL statements, and obtains the streaming computing results by taking out the data of the corresponding table from the database through the server, thereby reducing the workload of the background to obtain data, and directly generating the processing results from the database, avoiding the background user to perform data verification based on the results for the second time, and improving the data processing efficiency at the same time.

[0019] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A flow chart of a cross-table streaming computing method provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of the structure of a cross-table streaming computing system provided by an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] At present, in the time series database, when dealing with multiple tables and multiple databases, a single streaming query cannot meet the requirements. If the amount of data is large, the processing is relatively slow, and the client needs to occupy the database connection all the time, which is likely to cause network congestion. In addition, on the server side, there will be an LRU cache to store the index of the data file, which can locate the data block very quickly and execute streaming tasks asynchronously. However, in some scenarios, it is necessary to perform some calculation operations on two or more tables. For example, the scenario is to calculate the difference in power consumption of two circuit tables, or to calculate the difference in water level between the upstream and downstream of a river. The traditional method is generally to collect data separately, and then the background performs calculation operations on the acquired data. The workload of the background to obtain data is large, and the data processing efficiency is low.

[0027] Based on this, the embodiments of the present invention provide a cross-table streaming computing method, system, electronic device and medium, which can reduce the workload of backend data acquisition and improve data processing efficiency.

[0028] To facilitate understanding of this embodiment, a cross-table stream computing method disclosed in an embodiment of the present invention is first described in detail. The method is applied to a cross-table stream computing system, and the cross-table stream computing system includes: a client and a server. Figure 1 The flowchart of a cross-table streaming computing method shown in FIG. 1 illustrates that the method mainly includes the following steps S101 to S102:

[0029] Step S101: The client performs syntax check on the preset SQL statement, and after the check passes, generates statement content based on the SQL statement and sends it to the server.

[0030] The SQL statement is used to perform streaming computing on time series data in different cache tables. In one implementation, the built-in cross-table streaming computing of the database can be implemented through the SQL statement, which mainly involves the design of subquery. The SQL statement example is as follows:

[0031] CREATE STREAM[IF NOT EXISTS]stream_name[stream_options]INTO stb_name[(field1_name,field2_name[PRIMARY KEY],...)][TAGS(create_definition[,create_definition]...)]SUBTABLE(expression)AS subquery.

[0032] Among them, subquery is a subset of the select normal query syntax.

[0033] select now,func1(col1),func2(col2)from tbname'A'and tbname'B';

[0034] Among them, now refers to the time, func1 and func2 refer to the function names used to perform operations on a certain column, and from is followed by the table name. The cross-tables are A and B.

[0035] In the embodiment of the present invention, after the relevant rules (i.e., SQL statements) are defined, the client performs syntax analysis, verifies the SQL statements according to the specific defined rules, generates the syntax content as a syntax tree and sends it to the server. Specifically, the time information, function information, table information, etc. of the SQL statements can be verified.

[0036] Step S102: a streaming computing task is created based on the statement content by the server, and the streaming computing task is executed to read the time series data from different cache tables to perform streaming computing and obtain the streaming computing result.

[0037] In one implementation, after receiving the statement content sent by the client, the server can create streaming computing information and generate a streaming computing task; then, execute the streaming computing task, read time series data from different cache tables for streaming computing, and obtain the streaming computing results.

[0038] The cross-table streaming calculation method provided by the embodiment of the present invention realizes cross-table streaming calculation through pre-set SQL statements, and obtains the streaming calculation results by taking out the data of the corresponding table from the database through the server, thereby reducing the workload of the background to obtain data, and directly generating the processing results from the database, avoiding the background user to perform data verification based on the results for the second time, and improving the data processing efficiency.

[0039] In one implementation, before generating statement content based on the SQL statement and sending it to the server, it also includes: obtaining metadata information from the database through the client, and checking the metadata information.

[0040] In the specific implementation, the above SQL statement is used to first check the syntax problem, then the client pulls the metadata information (including information about libraries, tables, columns, etc.) from the database and performs a series of checks based on the metadata. For example:

[0041] / / Judge whether the two tables are subtables of the same structure

[0042] checkSubtableStrucInfo(const char*tbname1,const char*tbname2);

[0043] / / Check function: whether it exists and whether the column data matches

[0044] checkFuncInfo(funcname,colInfo).

[0045] In one embodiment, for the aforementioned step S101, that is, when generating statement content based on the SQL statement and sending it to the server, the following methods can be used, including but not limited to: serializing the streaming computing name, super table information, table information, function information and corresponding column information, generating statement content, and sending the statement content to the server.

[0046] In specific implementation, the collected information can be summarized and serialized to create a message (ie, the sentence content), and then the message can be sent to the server for corresponding processing.

[0047] Specifically, the streaming computing name, various modes, super table information, table information, function information and corresponding column information are collected and put into the structure StreamInfo, and then the message type is set, and the corresponding message content is serialized and sent to the server, which performs corresponding processing.

[0048] buildCmdMsg(pCxt,(FSerializeFunc)tSerializeSCreateStreamReq,&StreamInfo);

[0049] tmsgSendInfo(epSet,pMsg).

[0050] In one implementation, before creating a streaming computing task based on the statement content through the server, it also includes: deserializing the statement content through the server, and verifying the library and table permissions of the information obtained through the deserialization.

[0051] In specific implementation, after receiving the statement content sent by the client, the server can first deserialize the client message (ie, the statement content) to obtain the corresponding information, and then perform a secondary check on the user's library table permissions and special points.

[0052] Further, for the aforementioned step S102, that is, when a streaming computing task is created based on the statement content by the server, and the streaming computing task is executed, and the time series data is read from different cache tables for streaming computing to obtain the streaming computing result, the following methods may be used, including but not limited to:

[0053] First, the server creates streaming computing information and super table subtable structure based on the statement content, and generates streaming computing tasks.

[0054] Then, the streaming computing task is added to the task list, and the streaming computing task is executed to read the time series data from different cache tables for streaming computing to obtain the streaming computing results.

[0055] Finally, the streaming calculation results are saved in the super table sub-table structure.

[0056] In the specific implementation, since the data files of the database are strictly sharded according to time (the sharding time is executed by the parameters of the database to which the table belongs), it is first necessary to determine the time range (i.e., time information) spanned by the streaming computing task in this embodiment to facilitate multi-channel processing; then, data aggregation processing is performed on each shard file, and data blocks are taken out from the data file according to the UID information of the table, and the data in the data block is calculated to obtain the streaming computing result.

[0057] Specifically, first determine the time information of the streaming computing task, and obtain the corresponding data file from the database based on the time information; then obtain the corresponding data block from the data file based on the UID information of the cache table, and perform function calculation on the data block to obtain the streaming computing result.

[0058] In a specific implementation, the streaming computing task can obtain the data required by the streaming sunquery from each vnode, and then the processing of the streaming computing task belongs to a separate processing task queue; finally, the processed data is saved in the created super table subtable.

[0059] mndProcessCreateStreamReq(SRpcMsg*pReq){

[0060] mndCheckStreamReq(mnode,createStreamReq); / / Permission, quantity, etc. check, and perform secondary check on items that the client cannot check;

[0061] mndCreateStream(); / / Create a streaming computing task (which includes the subsequent super table sub-table structure, etc.).

[0062] mndTransPrepare(); / / Execute streaming calculation and add streaming calculation to the streaming task queue

[0063] }.

[0064] When executing streaming computing, first decompose the streaming computing task (the specific decomposition depends on the time range involved in the streaming task), and then obtain the corresponding data file from the data storage file path Vnode, which is specifically divided into the following parts:

[0065] (1) First, determine the time information. Because the data files stored in the database are segmented strictly according to time, obtain the .data file, head file, stt file, and sma file of the corresponding time according to the streaming computing task, and perform time-segment processing.

[0066] (2) The corresponding subtable is obtained in the head file through the UID and other information of the subtable name involved, and the index information of the datablock in the data file is obtained. Then, the data in the data block is taken out respectively through the index information and temporarily stored in the memory.

[0067] (3) When it comes to function processing, it needs to correspond to a specific timestamp. In this embodiment, function processing calculations are performed only for sub-tables with the same timestamp, and then the results are aggregated into a data slot. The data in the data slot is sorted in memory, and then the data is stored in sequence in the super table sub-table structure generated by streaming.

[0068] In the embodiment of the present invention, the database can support the calculation of column data between certain sub-tables, aggregate the results and output them, reduce the need to obtain sub-table data separately and then perform calculations. If the calculation is performed in the database inner layer and the results are output, the workload on the user side can be significantly reduced, especially when there are a lot of statistical data. For specific implementation, please refer to the following content:

[0069] MndProcessFunc(SRpcMsg*req){

[0070] / / Get the processing function according to the type of the sql column of the streaming subquery

[0071] Fun1=GetFunc(subquerytype1);

[0072] Func2=getFunc(subquerytype2); ....

[0074] / / Process the streaming task according to the corresponding column processing function

[0075] Res = Func1(req);

[0076] Res2 = func2(req);

[0077] / / Aggregate the results and output them to the corresponding table structure

[0078] AggResAndOutTable(res,res2);

[0079] }.

[0080] In another implementation, after the server receives the client message, the server processing mainly involves: deserializing the client message, obtaining the corresponding information, and then performing pre-operations such as permission verification, and then performing the following operations:

[0081] ① Save the newly created streaming computing information to the metadata information and synchronize it to each node in the cluster. Specifically, directly save and synchronize its information records according to the rules of the streaming computing system table.

[0082] ② Create the corresponding super table sub-table structure through the client message to save the result information of the streaming calculation.

[0083] ③ Perform data query and calculation through subquery information.

[0084] In the embodiment of the present invention, after the stream computing task is executed, a certain data check may be performed on the table data generated after the stream computing task is processed to check whether the computing result is correct.

[0085] The above method provided by the embodiment of the present invention performs user-defined streaming calculations on the cache of the sub-table last_row and last_value through the database, and then pushes the user streaming calculation results into the table. Through the designed SQL statement, cross-table streaming calculations are supported, and the latest data of the corresponding table is taken out from the memory, and the pre-set calculation is performed, and the calculation results are retained. By caching the latest data of the sub-table, it is possible to obtain data quickly and in real time without querying the file, and then return the calculation results. The above method has the following advantages:

[0086] 1. Efficiency: It can reduce the workload of back-end data acquisition and directly generate processing results from within the database, thus avoiding back-end users from doing data verification based on the results for the second time.

[0087] 2. Ease of use: Users only need a designed SQL statement to generate corresponding processing data according to the set ideas, thereby improving the user experience.

[0088] 3. Data accuracy: Since the data is generated through the server, human intervention is greatly reduced, which can better ensure the accuracy of the data and reduce errors or omissions caused by manual statistics.

[0089] 4. Applicability: It is extremely applicable when tasks such as statistics of multiple tables of time series data are relatively complicated, and it can also reduce manual workload.

[0090] For the cross-table stream computing method provided in the above embodiment, the present invention also provides a cross-table stream computing system, see Figure 2 The structural diagram of a cross-table stream computing system shown in the figure illustrates that the system mainly includes the following parts: a client 201 and a server 202.

[0091] Among them, the client 201 is used to perform syntax check on the pre-set SQL statement, and after the check is passed, generate statement content based on the SQL statement and send it to the server; wherein the SQL statement is used to perform streaming calculations on the time series data in different cache tables; the server 202 is used to create a streaming calculation task based on the statement content, and execute the streaming calculation task, read the time series data from different cache tables for streaming calculations to obtain the streaming calculation results.

[0092] The above-mentioned cross-table streaming computing system provided by the embodiment of the present invention realizes cross-table streaming computing through pre-set SQL statements, and obtains the streaming computing results by taking out the data of the corresponding table from the database through the server side, thereby reducing the workload of the background to obtain data, and directly generating the processing results from the database, avoiding the background user to perform data verification based on the results for the second time, and at the same time improving the data processing efficiency.

[0093] In one implementation, the client 201 is specifically used to verify the time information, function information, and table information of the SQL statement.

[0094] In one implementation, the client 201 is specifically used to: obtain metadata information from a database through the client, and check the metadata information.

[0095] In one implementation, the client 201 is specifically used to serialize the stream computing name, super table information, table information, function information, and corresponding column information, generate statement content, and send the statement content to the server.

[0096] In one implementation, the server 202 is specifically used to: deserialize the statement content through the server, and verify the library and table permissions of the information obtained through the deserialization.

[0097] In one embodiment, the server 202 is specifically used to: create streaming computing information and a super table sub-table structure based on statement content through the server, and generate a streaming computing task; add the streaming computing task to the task list, and execute the streaming computing task to obtain the streaming computing result; save the streaming computing result to the super table sub-table structure.

[0098] In one embodiment, the server 202 is specifically used to: determine the time information of the streaming computing task, and obtain the corresponding data file from the database based on the time information; obtain the corresponding data block from the data file based on the UID information of the table, and perform function calculation on the data block to obtain the streaming computing result.

[0099] It should be noted that the implementation principle and technical effects of the system provided in the embodiment of the present invention are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the system embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0100] An embodiment of the present invention further provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above implementation methods.

[0101] Figure 3 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 30, a memory 31, a bus 32 and a communication interface 33, wherein the processor 30, the communication interface 33 and the memory 31 are connected via the bus 32; the processor 30 is used to execute an executable module stored in the memory 31, such as a computer program.

[0102] The memory 31 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 33 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0103] The bus 32 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0104] Among them, the memory 31 is used to store programs, and the processor 30 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiments of the present invention can be applied to the processor 30 or implemented by the processor 30.

[0105] The processor 30 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 30. The above processor 30 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 31, and the processor 30 reads the information in the memory 31 and completes the steps of the above method in combination with its hardware.

[0106] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0107] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0108] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A cross-table streaming computing method, characterized in that: Applied to a cross-table stream computing system, the cross-table stream computing system includes a client and a server; the method includes: The client performs syntax verification on a preset SQL statement, and after the verification passes, generates statement content based on the SQL statement and sends it to the server; wherein the SQL statement is used to perform streaming calculations on time series data in different cache tables; The server creates a streaming computing task based on the statement content, executes the streaming computing task, reads time series data from different cache tables, performs streaming computing, and obtains streaming computing results.

2. The method according to claim 1, characterized in that The client performs syntax checking on a preset SQL statement, including: The time information, function information, and table information of the SQL statement are verified.

3. The method according to claim 1, characterized in that Before generating statement content based on the SQL statement and sending it to the server, the method further includes: The metadata information is obtained from the database through the client, and the metadata information is checked.

4. The method according to claim 1, characterized in that Generating statement content based on the SQL statement and sending it to the server includes: The stream computing name, super table information, table information, function information and corresponding column information are serialized to generate statement content, and the statement content is sent to the server.

5. The method according to claim 1, characterized in that Before creating a stream computing task based on the statement content by the server, the method further includes: The statement content is deserialized by the server, and the library table permissions are verified for the information obtained by the deserialization.

6. The method according to claim 1, characterized in that The server creates a stream computing task based on the statement content, executes the stream computing task, reads time series data from different cache tables, performs stream computing, and obtains stream computing results, including: The server creates streaming computing information and a super table sub-table structure based on the statement content, and generates a streaming computing task; Add the stream computing task to the task list, execute the stream computing task, read the time series data from different cache tables, perform stream computing, and obtain the stream computing result; The stream computing result is saved in the super table sub-table structure.

7. The method according to claim 6, characterized in that Executing the stream computing task, reading the time series data from different cache tables to perform stream computing to obtain the stream computing result, including: Determine the time information of the streaming computing task, and obtain the corresponding data file from the database based on the time information; Based on the UID information of the cache table, a corresponding data block is obtained from the data file, and function calculation is performed on the data block to obtain a streaming calculation result.

8. A cross-table streaming computing system, characterized in that: include: Client and server; The client is used to perform syntax verification on a preset SQL statement, and after the verification passes, generates statement content based on the SQL statement and sends it to the server; wherein the SQL statement is used to perform streaming calculations on time series data in different cache tables; The server is used to create a streaming computing task based on the statement content, execute the streaming computing task, read the time series data from different cache tables, perform streaming computing, and obtain a streaming computing result.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are performed.

Images