Stream processing method and system based on graph structure and SQL (Structured Query Language) statement

Abstract

The invention relates to a stream processing method and system based on a graph structure and SQL statements.The method comprises the steps that a plurality of SQL nodes used for meeting the calculation requirement of stream processing services are created, and one SQL node corresponds to one SQL statement; the SQL nodes are dragged and connected according to the sequence of the calculation process of the stream processing service, and an SQL structure chart used for completing the stream processing service is generated. According to the method and the device, the ingenious combination of the graph structure and the SQL statements is realized, the single SQL statement is abstracted into the single node in the graph structure, in other words, the plurality of SQL statements for completely describing the flow processing business can be used as the plurality of SQL nodes to be stored in the structure graph, the intuitive management is facilitated, and meanwhile, the SQL nodes are used for replacing the traditional graph nodes with single computing power, so that the calculation efficiency is improved. The number of nodes required by the stream processing service is greatly reduced, maintenance is easy, and the problem of how to improve the management and maintenance convenience of the stream processing service is solved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a stream processing method, system, apparatus and medium based on graph structure and SQL statements. Background Technology

[0002] Stream processing is a real-time data stream analysis technology that achieves low-latency, high-throughput real-time computing by continuously processing dynamic data streams. It is widely used in scenarios such as financial risk control, IoT monitoring, and edge computing. In actual production environments, stream processing computing commonly employs two technical implementation approaches: First, describing the stream processing workflow using SQL statements. However, a single SQL statement is often insufficient to fully describe the entire workflow, serving only as one part of the process. Multiple SQL statements are required to fully describe the entire workflow, making it difficult to manage intuitively. Second, using graphs to describe the stream processing workflow. Graph descriptions are more flexible than SQL statements, allowing data processing channels to be connected through dragging and dropping. Theoretically, a complete workflow can be covered by a single graph. However, the computational capabilities of graph nodes are often limited. A stream processing flow that a single SQL statement can express might require a dozen or more nodes in the corresponding graph rules, easily leading to an excessive number of graph nodes and making maintenance difficult.

[0003] In existing solutions, patent application number 202111311866.0 discloses a streaming data processing method. This method generates an SQL graph by identifying each SQL command in an SQL statement as a node and each logical keyword (such as AND, OR, NOT, etc.) as an edge, to schedule and process streaming data, thereby improving computational efficiency. In other words, this patent essentially identifies each element in a single SQL statement as a node or edge, thus converting the single SQL statement into a graph. It's evident that when streaming processing requires multiple SQL statements, it means converting multiple SQL statements into multiple graphs, each containing numerous nodes and edges, making management and maintenance more difficult. Patent application number 202010676621.7 discloses a streaming processing method that directly configures a DAG graph based on the streaming processing task. It optimizes the DAG graph by matching higher-order operators to monitor the progress of the streaming processing task, eliminating the need for SQL statements and reducing the cost of learning SQL for users. This is precisely the solution mentioned above that uses graphs to describe streaming processing tasks, but it suffers from maintenance difficulties.

[0004] Currently, no effective solution has been proposed to address the issue of improving the ease of management and maintenance of stream processing services in related technologies. Summary of the Invention

[0005] This application provides a stream processing method, system, apparatus, and medium based on graph structure and SQL statements, to at least address the problem of how to improve the ease of management and maintenance of stream processing services in related technologies.

[0006] In a first aspect, embodiments of this application provide a stream processing method based on graph structures and SQL statements, the method comprising: Based on the computing requirements of stream processing services, several SQL nodes are created to meet the computing requirements, wherein one SQL node corresponds to one SQL statement. Based on the computation process of the stream processing service, the SQL nodes are dragged and connected in the order of the computation process to generate an SQL structure diagram for completing the stream processing service.

[0007] In some embodiments, based on the computational requirements of stream processing services, several SQL nodes are created to meet those requirements, wherein one SQL node corresponds to one SQL statement, including: Based on the computing requirements of stream processing services, several SQL nodes are created to meet these computing requirements, wherein each SQL node serves as a node for visualization operations. Create SQL statements for executing the stream processing business, wherein each SQL statement serves as a stream processing sub-rule describing a stage in the stream processing business; The SQL nodes are associated and bound one by one with the SQL statements, so that the SQL nodes have the ability to call the corresponding SQL statements to complete the stream processing business.

[0008] In some embodiments, creating several SQL nodes to meet the computational requirements of stream processing services includes: Based on the computing requirements of the current stream processing business, several SQL nodes are created to meet the computing requirements, and a dedicated data subscription pipeline is declared for each SQL node.

[0009] In some embodiments, associating and binding the SQL node with the SQL statement one by one, so that the SQL node has the ability to call the corresponding SQL statement to complete the stream processing business, includes: Create corresponding stream processing sub-rules based on the SQL statement; Each stream processing sub-rule's data pipeline is defined as a dedicated data subscription pipeline for the corresponding SQL node, enabling the SQL node to call the corresponding SQL statement to complete the stream processing business. The dedicated data subscription pipeline is a memory data subscription pipeline for a dedicated topic.

[0010] In some embodiments, defining the data channel of each stream processing sub-rule as a dedicated data subscription pipeline for the corresponding SQL node includes: Each stream processing sub-rule's data input is associated and bound to the corresponding SQL node's data input, and each stream processing sub-rule's data output is associated and bound to the corresponding SQL node's data output, thus defining the data pipelines of all stream processing sub-rules as dedicated data subscription pipelines for the corresponding SQL nodes.

[0011] In some embodiments, based on the computation flow of the stream processing service, the SQL nodes are dragged and connected in the order of the computation flow to generate an SQL structure diagram for completing the stream processing service, including: Based on the calculation process of the stream processing service, the SQL nodes are dragged and connected in the order of the calculation process to obtain the data flow channel. In the data flow channel, the data source is declared with the node name of the current SQL node as a prefix, and the data source is linked to the data output of the previous SQL node. That is, the stream processing sub-rules of the current SQL node directly obtain the data sent by the previous SQL node through the data pipeline to generate an SQL structure diagram for completing the stream processing business.

[0012] In some embodiments, after creating the corresponding stream processing sub-rule based on the SQL statement, the method further includes: Add a lifecycle management tag to the stream processing sub-rule, wherein the lifecycle management tag is bound to the process of the corresponding SQL node; After the stream processing business is completed through the generated SQL structure diagram, if the process of the SQL node inside the SQL structure diagram is closed, the corresponding stream processing sub-rules of the SQL node will be deleted.

[0013] Secondly, embodiments of this application provide a stream processing system based on graph structure and SQL statements. The system is used to execute the method described in the first aspect above. The system includes a node creation module and a structure graph generation module. The node creation module is used to create a number of SQL nodes to meet the computing requirements of the stream processing business, wherein one SQL node corresponds to one SQL statement. The structure diagram generation module is used to drag and connect the SQL nodes in the order of the calculation process of the stream processing business to generate an SQL structure diagram for completing the stream processing business.

[0014] Thirdly, embodiments of this application provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0016] Compared to related technologies, this application provides a stream processing method, system, device, and medium based on graph structure and SQL statements. The method creates several SQL nodes to meet the computational requirements of stream processing services, where each SQL node corresponds to a single SQL statement. Based on the computational flow of the stream processing service, the SQL nodes are dragged and connected in the order of the computational flow to generate an SQL structure graph for completing the stream processing service. This achieves a clever combination of graph structure and SQL statements, abstracting a single SQL statement into a single node in the graph structure. In other words, multiple SQL statements that fully describe the stream processing service can be stored as multiple SQL nodes in a single structure graph, facilitating intuitive management. Furthermore, by replacing traditional graph nodes with single-capacity computational nodes using SQL nodes, the number of nodes required for stream processing services is greatly reduced, making maintenance easier and solving the problem of improving the convenience of managing and maintaining stream processing services. Attached Figure Description

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 This is a flowchart of the steps of a stream processing method based on graph structure and SQL statements according to an embodiment of this application; Figure 2 This is a schematic diagram of data flow based on SQL nodes according to an embodiment of this application; Figure 3 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0019] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0020] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0021] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0022] This application provides a stream processing method based on graph structure and SQL statements. Figure 1This is a flowchart illustrating the steps of a stream processing method based on graph structure and SQL statements according to an embodiment of this application, as follows: Figure 1 As shown, the method includes the following steps: Step S102: Based on the computing requirements of the stream processing business, create several SQL nodes to meet the computing requirements, wherein one SQL node corresponds to one SQL statement. Step S102 specifically includes the following steps: Step S1021: Based on the computing requirements of the stream processing business, create several SQL nodes to meet the computing requirements, wherein each SQL node serves as a node for visualization operations. Specifically, step S1021 involves creating several SQL nodes to meet the computing requirements of the current stream processing business, and declaring a dedicated data subscription pipeline for each SQL node.

[0023] Step S1022: Create SQL statements for executing stream processing business, wherein each SQL statement serves as a stream processing sub-rule describing a stage in the stream processing business. Step S1023: Associate and bind SQL nodes with SQL statements one by one, so that SQL nodes have the ability to call the corresponding SQL statements to complete stream processing business.

[0024] Step S1023 specifically involves creating corresponding stream processing sub-rules based on SQL statements; defining the data pipeline of each stream processing sub-rule as a dedicated data subscription pipeline for the corresponding SQL node, enabling the SQL node to call the corresponding SQL statement to complete stream processing business. The dedicated data subscription pipeline is a memory data subscription pipeline for a dedicated topic. Preferably, the data input of each stream processing sub-rule is associated and bound to the data input of the corresponding SQL node, and the data output of each stream processing sub-rule is associated and bound to the data output of the corresponding SQL node, thereby defining the data pipeline of all stream processing sub-rules as dedicated data subscription pipelines for the corresponding SQL nodes.

[0025] It should be noted that in steps S1021 to S1023 above, a new computing node named SQL node is designed according to the graph rules. Unlike the computing nodes in common graph rules, the SQL node can directly accept an SQL statement as a parameter to describe the data processing flow. Figure 2 This is a schematic diagram of data flow based on SQL nodes according to an embodiment of this application, such as... Figure 2As shown, this SQL node can be linked with any other node in the graph rule computation, including other SQL nodes, through drag-and-drop, forming a data processing channel for the graph rules. In other words, it cleverly combines graph structure and SQL statements, abstracting a single SQL statement into a corresponding single node in the graph structure. That is, multiple SQL statements that fully describe the stream processing business can be stored as multiple SQL nodes in a single structure graph, which is convenient for intuitive management. At the same time, by using SQL nodes to replace the traditional graph nodes with single computing capabilities, the number of nodes required for stream processing business is greatly reduced, and maintenance is easier.

[0026] It's important to further explain that in step S1023, a memory data subscription pipeline with a specific topic is used to connect the data transmission between the sub-rules and the SQL node. When creating an SQL node, it creates and declares a memory data subscription pipeline with a specific topic. When creating a stream processing sub-rule, both the data input and output of the sub-rule are defined as memory data subscription pipelines for the corresponding topic, thus achieving the connection. Specifically, after creating the stream processing sub-rule, the input and output of the sub-rule need to be connected to the input and output of the SQL node (the memory data subscription pipeline for the specific topic). That is, the input of the SQL node is connected to the input of the sub-rule, and the output of the sub-rule is connected to the output of the SQL node. This allows the data in the graph rule to be transferred to the SQL node, and then the SQL node transfers the data to the sub-rule for processing. Furthermore, the memory data subscription pipeline for a topic describes a memory-based real-time data subscription architecture. It logically isolates the data stream through topics (message topics) and caches or directly processes the data during the processing in memory.

[0027] Step S104: Based on the calculation process of the stream processing business, drag and drop the SQL nodes in the order of the calculation process to generate an SQL structure diagram for completing the stream processing business.

[0028] Step S104 specifically includes the following steps: Step S1041: Based on the calculation process of the stream processing business, drag and drop the SQL nodes in the order of the calculation process to obtain the data flow channel. In step S1042, in the data flow channel, the data source is declared with the node name of the current SQL node as a prefix, and the data source is linked to the data output of the previous SQL node. That is, the stream processing sub-rules of the current SQL node directly obtain the data sent by the previous SQL node through the data pipeline to generate an SQL structure diagram for completing the stream processing business.

[0029] It's important to note that stream processing sub-rules are essentially sub-rules created from a single SQL statement. This SQL statement must declare a data source; therefore, the data source in the SQL statement corresponding to the sub-rule must be able to connect to the SQL node's in-memory data subscription pipeline. In other words, the data source declaration in the SQL statement cannot use a pre-defined real data source. In step S104, the current SQL node in the data flow channel declares its data source using its own node name as a prefix (the data source in the SQL statement of the current SQL node's sub-rule must also have this name; otherwise, an error will occur during the verification phase). This data source is a subscription pipeline with the same topic as the previous SQL node's in-memory data subscription pipeline. Thus, after creation, the current SQL node's sub-rule can retrieve data sent from the previous SQL node from the established in-memory data subscription pipeline.

[0030] Through the steps described above in this application embodiment, a clever combination of graph structure and SQL statement is achieved. A single SQL statement is abstracted into a single node in the graph structure. In other words, multiple SQL statements that fully describe the stream processing business can be stored as multiple SQL nodes in a single structure graph, which is convenient for intuitive management. At the same time, by using SQL nodes to replace the traditional graph nodes with single computing power, the number of nodes required for the stream processing business is greatly reduced, making it easier to maintain and solving the problem of how to improve the convenience of management and maintenance of stream processing business.

[0031] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] In some embodiments, after creating the corresponding stream processing sub-rules based on the SQL statements, the method further includes: Add a lifecycle management tag to the stream processing sub-rules, where the lifecycle management tag is bound to the process of the corresponding SQL node; After the stream processing business is completed through the generated SQL structure diagram, if the process of the SQL node inside the SQL structure diagram is shut down, the corresponding stream processing sub-rules of the SQL node will be deleted.

[0033] It should be noted that stream processing sub-rules are essentially sub-rules created from a single SQL statement, and the lifecycle of the sub-rule and the SQL structure must be guaranteed. Figure 1When the SQL structure graph is created, the corresponding SQL nodes within it will create their sub-rules along with it. When the SQL structure graph is closed, the corresponding SQL nodes within it will delete their sub-rules. All sub-rules are tagged with a special tag to identify them as sub-rules. When a process restarts and needs to start a rule, it needs to iterate through all rules and delete the sub-rules from storage first. Then, it will execute the logic to restart the original rules. In other words, the entire lifecycle of a sub-rule must be managed by its corresponding SQL node.

[0034] This application provides a stream processing system based on graph structure and SQL statements. The system is used to execute the method provided in the above embodiments. The system includes a node creation module and a structure graph generation module. The node creation module is used to create several SQL nodes to meet the computing needs of the stream processing business. Each SQL node corresponds to one SQL statement. The structure diagram generation module is used to drag and connect SQL nodes in the order of the calculation process of the stream processing business to generate an SQL structure diagram for completing the stream processing business.

[0035] The node creation module and structure graph generation module in this application embodiment achieve a clever combination of graph structure and SQL statement, abstracting a single SQL statement into a single node in the graph structure. In other words, multiple SQL statements that fully describe the stream processing business can be stored as multiple SQL nodes in a structure graph, which is convenient for intuitive management. At the same time, by using SQL nodes to replace the traditional graph nodes with single computing power, the number of nodes required for the stream processing business is greatly reduced, which is easy to maintain and solves the problem of how to improve the convenience of management and maintenance of the stream processing business.

[0036] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0037] This embodiment provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0038] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0039] Optionally, the electronic device may further include a processor, memory, network interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a stream processing method based on graph structures and SQL statements. The display screen may be a liquid crystal display (LCD) or an e-ink display. The input device may be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0040] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0041] Furthermore, in conjunction with the stream processing methods based on graph structures and SQL statements in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the stream processing methods based on graph structures and SQL statements in the above embodiments.

[0042] In one embodiment, Figure 3 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 3 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 3 As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores the operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network connection, the internal memory provides an environment for the operation of the operating system and computer programs, the computer programs are executed by the processor to implement a stream processing method based on graph structures and SQL statements, and the database stores data.

[0043] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0044] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0045] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0046] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A stream processing method based on graph structure and SQL statements, characterized in that, The method includes: Based on the computing requirements of stream processing services, several SQL nodes are created to meet the computing requirements, wherein one SQL node corresponds to one SQL statement. Based on the computation process of the stream processing service, the SQL nodes are dragged and connected in the order of the computation process to generate an SQL structure diagram for completing the stream processing service.

2. The method according to claim 1, characterized in that, Based on the computational requirements of stream processing services, several SQL nodes are created to meet these requirements. Each SQL node corresponds to one SQL statement, including: Based on the computing requirements of stream processing services, several SQL nodes are created to meet these computing requirements, wherein each SQL node serves as a node for visualization operations. Create SQL statements for executing the stream processing business, wherein each SQL statement serves as a stream processing sub-rule describing a stage in the stream processing business; The SQL nodes are associated and bound one by one with the SQL statements, so that the SQL nodes have the ability to call the corresponding SQL statements to complete the stream processing business.

3. The method according to claim 2, characterized in that, Based on the computational requirements of stream processing services, several SQL nodes are created to meet these computational requirements, including: Based on the computing requirements of the current stream processing business, several SQL nodes are created to meet the computing requirements, and a dedicated data subscription pipeline is declared for each SQL node.

4. The method according to claim 3, characterized in that, Associating and binding each SQL node with a SQL statement, enabling the SQL node to call the corresponding SQL statement to complete stream processing tasks, includes: Create corresponding stream processing sub-rules based on the SQL statement; Each stream processing sub-rule's data pipeline is defined as a dedicated data subscription pipeline for the corresponding SQL node, enabling the SQL node to call the corresponding SQL statement to complete the stream processing business. The dedicated data subscription pipeline is a memory data subscription pipeline for a dedicated topic.

5. The method according to claim 4, characterized in that, Defining the data channel of each stream processing sub-rule as a dedicated data subscription pipeline for the corresponding SQL node includes: Each stream processing sub-rule's data input is associated and bound to the corresponding SQL node's data input, and each stream processing sub-rule's data output is associated and bound to the corresponding SQL node's data output, thus defining the data pipelines of all stream processing sub-rules as dedicated data subscription pipelines for the corresponding SQL nodes.

6. The method according to claim 5, characterized in that, Based on the computation flow of the stream processing service, the SQL nodes are dragged and connected in the order of the computation flow to generate an SQL structure diagram for completing the stream processing service, including: Based on the calculation process of the stream processing service, the SQL nodes are dragged and connected in the order of the calculation process to obtain the data flow channel. In the data flow channel, the data source is declared with the node name of the current SQL node as a prefix, and the data source is linked to the data output of the previous SQL node. That is, the stream processing sub-rules of the current SQL node directly obtain the data sent by the previous SQL node through the data pipeline to generate an SQL structure diagram for completing the stream processing business.

7. The method according to claim 4, characterized in that, After creating the corresponding stream processing sub-rule based on the SQL statement, the method further includes: Add a lifecycle management tag to the stream processing sub-rule, wherein the lifecycle management tag is bound to the process of the corresponding SQL node; After the stream processing business is completed through the generated SQL structure diagram, if the process of the SQL node inside the SQL structure diagram is closed, the corresponding stream processing sub-rules of the SQL node will be deleted.

8. A stream processing system based on graph structure and SQL statements, characterized in that, The system is used to perform the method according to any one of claims 1 to 7, the system comprising a node creation module and a structure diagram generation module; The node creation module is used to create a number of SQL nodes to meet the computing requirements of the stream processing business, wherein one SQL node corresponds to one SQL statement. The structure diagram generation module is used to drag and connect the SQL nodes in the order of the calculation process of the stream processing business to generate an SQL structure diagram for completing the stream processing business.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images