Real-time data stream processing method, system and device based on RisingWave

Through the real-time data stream processing system based on RisingWave database, SQL query and materialized view technology are used to solve the problems of high cost of real-time data processing in the financial field, efficient real-time data analysis and ease of use, and reduce development complexity degree and cost.

CN118503290BActive Publication Date: 2024-11-15CFETS FINANCIAL DATA CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410686444.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-30
Publication Date
2024-11-15
Estimated Expiration
2044-05-30

AI Technical Summary

Technical Problem

In the real-time data processing in the fields of finance and other fields, the process of converting data models into production available forms is complex and costly. The Flink Stream API requires coding modification and deployment when processing high-frequency complex factor application scenarios, and Flink itself is not Database increases the complexity of system integration, and users need to learn a proprietary scripting language to use the DolphinDB database.

Method used

Based on the RisingWave database, a real-time data stream processing system is built through the stream processing platform and SQL query mechanism, and real-time data query and analysis is realized using materialized view technology, supporting custom function expansion, reducing development difficulty and cost.

Benefits of technology

It realizes efficient real-time data query and analysis, meets the needs of high-performance real-time analysis, reduces the threshold for use, improves system universality and ease of use, and reduces development costs and time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118503290B_ABST
    Figure CN118503290B_ABST
Patent Text Reader

Abstract

The present invention discloses a real-time data stream processing method, system and device based on RisingWave, and relates to the technical field of stream data processing. The method comprises: constructing a real-time data stream processing system based on RisingWave, the system comprising a RisingWave database and a stream processing platform; the RisingWave database is used to: configure data sources and table structures; obtain SQL query statements, the SQL query statements are used to filter, analyze and / or convert data; create a materialized view, parse the query SQL into a streaming process and execute it in the background, and use the real-time calculation results as the query results of the materialized view; configure data output, and perform data result output processing. The present invention realizes efficient real-time data query and analysis, and at the same time meets the requirements of high-performance real-time analysis of streaming data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data stream processing, and in particular to a real-time data stream processing method, system and device based on RisingWave. Background Art

[0002] Real-time data processing and streaming computing are widely used in fields such as finance and the Internet of Things that have high requirements for data storage, query, and analysis. Taking the foreign exchange market and stock trading business in the financial field as an example, by analyzing real-time market data, traders can make decisions quickly and improve trading efficiency and accuracy. In existing business scenarios, processing real-time market data usually involves calculating and generating new data values ​​based on preset specific data model algorithms.

[0003] At present, the conversion of data models into production-usable forms is usually divided into two stages: in the first stage, data analysts use tools such as Python or SQL to perform data analysis and factor model (data model for factor analysis) description to establish data models; the second stage is the model conversion development stage, in which developers convert the model into a Flink Stream program (i.e., a Flink stream application) to achieve the required performance goals. For data analysts, the first stage is relatively easy to implement, but the performance of the established data model is often difficult to meet the needs of real-time data processing. In order to meet the high-performance real-time analysis requirements of real-time data, the existing technology also provides some database applications that support complex computing and stream processing, such as the DolphinDB database. As a high-performance distributed time series database, the DolphinDB database integrates a powerful programming language and a high-capacity and high-speed stream data analysis system, providing a one-stop solution for the rapid storage, retrieval, analysis and calculation of structured data, and is particularly suitable for scenarios with extremely high requirements for data storage, query and analysis. Among them, DolphinDB supports the processing of real-time data streams through its proprietary scripting language, and has built-in rich financial functions for analysts to use directly. However, the DolphinDB database is a closed source database, the model is charged, and business personnel need to learn and use a proprietary scripting language when using it, which places high demands on business personnel.

[0004] The development process of the second phase is often long and costly. In addition, although Flink Stream API (Stream API is used to operate on collection data) performs well in processing high-frequency and complex factor application scenarios, it needs to be modified and expanded through coding, and the deployment process is cumbersome; at the same time, Flink SQL often needs to rely on middleware such as Kafka when processing derivative factors, which may affect the final performance; in addition, since Flink itself is not designed as a database, its data storage and management functions require external system support, which increases the complexity of system integration.

[0005] On the other hand, RisingWave is an open-source distributed streaming database based on the Apache 2.0 protocol for real-time data processing. RisingWave uses PostgreSQL to express streaming computing. It uses SQL as an interface, allowing users to define tables, materialized views, etc., so as to quickly build streaming computing tasks, greatly simplifying the workflow of data analysts; using SQL as an interface also reduces the difficulty and cost of developing real-time data solutions. RisingWave provides users with a user experience similar to PostgreSQL, and has performance that is several to hundreds of times higher than traditional streaming computing engines such as Apache Flink, and at a lower cost.

[0006] The present invention is based on the RisingWave database and combines the current real-time analysis requirements of high-frequency financial data to provide a real-time data stream processing solution based on RisingWave. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a real-time data stream processing method, system and device based on RisingWave. The real-time data stream processing solution provided by the present invention includes a stream processing platform and a RisingWave database. The real-time data stream is generated by the stream processing platform, and the materialized view result set based on the SQL query is updated and stored in real time using the RisingWave database, thereby realizing efficient real-time data query and analysis, and meeting the requirements of high-performance real-time analysis of streaming data.

[0008] To achieve the above objectives, the present invention provides the following technical solutions:

[0009] A real-time data stream processing method based on RisingWave, comprising: constructing a real-time data stream processing system based on RisingWave, wherein the real-time data stream processing system comprises a RisingWave database and a stream processing platform; the stream processing platform is used to form a real-time data stream from received real-time data, and transmit the real-time data stream to the RisingWave database; the RisingWave database supports data ingestion, streaming computing and real-time output through structured query language SQL;

[0010] Among them, the RisingWave database is configured as follows: configure the data source Source and the table structure Table, obtain the real-time data stream from the configured data source; obtain the SQL query statement, the SQL query statement is used to filter, analyze and / or convert the data; create a materialized view, parse the query SQL into a streaming processing process, execute the streaming computing task in the background, and use the real-time computing results as the query results of the materialized view. When the data source changes, the query results of the materialized view are automatically updated; configure the data output Sink to perform data result output processing.

[0011] Further, the RisingWave database stores the query results of the materialized view as the pre-data, and outputs the pre-data to the stream processing platform in real time by configuring the data output Sink;

[0012] The stream processing platform is configured to: form a real-time data stream after receiving the pre-data transmitted by the RisingWave database, and transmit the real-time data stream to the RisingWave database;

[0013] At this time, the RisingWave database is configured to: receive the streaming pre-data transmitted by the stream processing platform; obtain the SQL query statement for the pre-data; create a materialized view for the pre-data, and after executing the streaming computing task corresponding to the aforementioned SQL query in the background, obtain the materialized view query result for the pre-data; when the data source changes, calculate the change of the pre-data, and calculate the change of the materialized view query result for the pre-data based on the change of the pre-data, and automatically update the query result.

[0014] Furthermore, the materialized view query result for the front-end data can be stored as the front-end data and output to the stream processing platform in real time to form a real-time data stream; for the real-time data stream, a new materialized view is established again based on the SQL query.

[0015] Further, the stream processing platform of the real-time data stream processing system is connected to a data provider;

[0016] The data provider continuously generates real-time data and transmits it to the stream processing platform;

[0017] The stream processing platform is configured to: after receiving the aforementioned real-time data, process the received real-time data as a continuous stream of events or messages to form a real-time data stream, and then deliver the real-time data stream to the RisingWave database.

[0018] Further, the RisingWave database and stream processing platform of the real-time data stream processing system are connected with the business system;

[0019] The business system is used to receive the user's SQL query request and send the SQL query request to the RisingWave database;

[0020] The stream processing platform is configured to: obtain the materialized view query result transmitted by the RisingWave database, and send the materialized view query result to the business system.

[0021] Furthermore, the stream processing platform is Kafka. At this time, the data source configured for the RisingWave database is Kafka.

[0022] Furthermore, the real-time data stream processing system further comprises a function extension module, wherein the function extension module is used for a user to configure a user-defined function UDF;

[0023] The RisingWave database is configured to: obtain the UDF information configured by the user in the function extension module, and register the UDF in the server of the RisingWave database; after the UDF is successfully registered, the RisingWave database can call the aforementioned UDF from the server for calculation.

[0024] Furthermore, the UDF configuration methods provided by the function extension module include:

[0025] Create UDFs within the RisingWave database using SQL to directly extend simple function combinations; or,

[0026] Use Python or Java to create the corresponding UDF in RisingWave and start an independent process to provide function services; or,

[0027] Use Rust to create UDFs in the RisingWave database, embed WASM binaries in SQL as base64 encoding, or load WASM binaries from object storage.

[0028] The present invention also provides a real-time data stream processing system based on RisingWave, the system comprising a RisingWave database and a stream processing platform;

[0029] The stream processing platform is used to form a real-time data stream from the received real-time data, and transmit the real-time data stream to the RisingWave database;

[0030] The RisingWave database supports data ingestion, streaming computing, and real-time output through structured query language SQL;

[0031] Among them, the RisingWave database is configured as follows: configure the data source Source and the table structure Table, obtain the real-time data stream from the configured data source; obtain the SQL query statement, the SQL query statement is used to filter, analyze and / or convert the data; create a materialized view, parse the query SQL into a streaming processing process, execute the streaming computing task in the background, and use the real-time computing results as the query results of the materialized view. When the data source changes, the query results of the materialized view are automatically updated; configure the data output Sink to perform data result output processing.

[0032] The present invention also provides an electronic device, comprising:

[0033] a storage device having a computer program stored thereon;

[0034] A processing device is used to execute the computer program in the storage device to implement the steps of any of the above methods.

[0035] Due to the adoption of the above technical scheme, the present invention has the following advantages and positive effects compared with the prior art, as an example: the real-time data stream processing scheme provided by the present invention includes a stream processing platform and a RisingWave database, which generates real-time data streams through the stream processing platform, and uses the RisingWave database to update and store materialized view result sets based on SQL queries in real time, thereby realizing efficient real-time data query and analysis, and being able to meet the requirements of high-performance real-time analysis of streaming data.

[0036] On the one hand, the real-time data stream processing system adopts a SQL-based streaming processing mechanism. Compared with existing technologies such as DolphinDB, users do not need to learn proprietary scripting languages ​​or programming languages, which lowers the usage threshold and improves the versatility and ease of use of the system.

[0037] On the other hand, based on the materialized view technology provided by RisingWave, the materialized view result set based on SQL query is updated and stored in real time, realizing efficient real-time data query and analysis.

[0038] On the other hand, the real-time materialized view query results are stored as pre-data, which can be queried through SQL and a new materialized view can be established based on the query, thus meeting the user's needs for generating and analyzing derivative data. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 An information transmission diagram of a real-time data stream processing method based on RisingWave provided in an embodiment of the present invention.

[0040] Figure 2 A data processing flow chart of the RisingWave database provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following is a further detailed description of the real-time data stream processing method, system and device based on RisingWave disclosed in the present invention in conjunction with the accompanying drawings and specific embodiments. It should be noted that the technologies (including methods and devices) known to ordinary technicians in the relevant fields may not be discussed in detail, but in appropriate cases, the above-mentioned known technologies are considered to be part of the specification. At the same time, other examples of exemplary embodiments may have different values. The structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification, so that people familiar with this technology can understand and read, and are not used to limit the limiting conditions for the implementation of the invention.

[0042] In the description of the embodiments of the present application, " / " means or, and "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" means: A and B exist alone, B exists alone, and A and B exist at the same time. In the description of the embodiments of the present application, "multiple" means two or more.

[0043] Terminology explanation:

[0044] 1) SQL: Structured Query Language. SQL is a special-purpose programming language, a database query and programming language, commonly used to access data and query, update and manage relational database systems.

[0045] 2) Stream processing platform: Streaming System, which is a data processing engine for processing infinite data sets, mainly used to process real-time data streams. The stream processing platform of the present invention is preferably Kafka.

[0046] 3) Kafka: A distributed stream processing platform that is mainly used to build real-time streaming data pipelines and streaming applications, that is, to build real-time streaming data pipelines that can reliably obtain data between systems or applications, and to build real-time streaming applications that transform or respond to data streams.

[0047] 4) RisingWave Database: A distributed SQL streaming database for real-time data processing. RisingWave uses PostgreSQL to express stream computing, with SQL as the interface, allowing users to define tables, materialized views, etc.

[0048] The technical concept and solution of the present invention are introduced below based on exemplary application scenarios.

[0049] Example

[0050] A real-time data stream processing method based on RisingWave includes: building a real-time data stream processing system based on RisingWave, wherein the real-time data stream processing system includes a RisingWave database and a stream processing platform.

[0051] The stream processing platform is used to form the received real-time data into a real-time data stream, and transmit the real-time data stream to the RisingWave database.

[0052] For details, see Figure 1 As shown, the stream processing platform of the real-time data stream processing system can be connected to a data provider.

[0053] The data provider continuously generates real-time data and transmits it to the stream processing platform Kafka. After receiving the real-time data, Kafka processes the received real-time data as a continuous stream of events or messages to form a real-time data stream, and then delivers the real-time data stream to the RisingWave database.

[0054] The RisingWave database supports data ingestion, streaming computing, and real-time output through structured query language SQL.

[0055] The RisingWave database is configured as follows: configure the data source Source and the table structure Table, obtain the real-time data stream from the configured data source; obtain the SQL query statement, which is used to filter, analyze and / or transform the data; create a materialized view, parse the query SQL into a streaming process, execute the streaming computing task in the background, and use the real-time computing result as the query result of the materialized view. When the data source changes, the query result of the materialized view is automatically updated; configure the data output Sink to perform data result output processing, see Figure 2 shown.

[0056] To use the RisingWave database to obtain real-time data, you first need to configure a data source Source. In a specific embodiment, the stream processing platform is preferably Kafka. At this time, Kafka should be configured as the data source, that is, the RisingWave database obtains the real-time data stream from Kafka.

[0057] Taking Kafka as an example, the SQL code for creating the data source Source and table structure through the RisingWave database can be as follows:

[0058] --Create table

[0059] CREATETABLE yesterday_bond(

[0060] datedate,

[0061] symbol string,

[0062] price numeric );

[0064] select*from yesterday_bondyb;

[0065] --Create a data source and access Kafka real-time data

[0066] CREATESOURCEIFNOTEXISTS source_bond_tick(

[0067] timestamptimestamptz,

[0068] symbol string,

[0069] ofr numeric,

[0070] bid numeric )

[0072] WITH(

[0073] connector = 'kafka',

[0074] topic = 'bond_tick',

[0075] properties.bootstrap.server='message_queue:9092',

[0076] scan.startup.mode='earliest'

[0077] )FORMAT PLAIN ENCODE JSON;

[0078] Continue to see Figure 1 As shown, the RisingWave database and stream processing platform of the real-time data stream processing system are also connected to the business system.

[0079] The business system is used to receive the SQL query request of the user and send the SQL query request to the RisingWave database. The SQL query request includes an SQL query statement for filtering, analyzing and / or converting the data.

[0080] The stream processing platform may also be configured to: obtain the materialized view query result transmitted by the RisingWave database, and send the materialized view query result to the business system.

[0081] In this embodiment, the RisingWave database may also store the query results of the aforementioned materialized view as pre-data, and output the aforementioned pre-data in real time to the aforementioned stream processing platform, such as Kafka, by configuring the data output Sink.

[0082] At this time, the stream processing platform is configured to: form a real-time data stream after receiving the front-end data transmitted by the RisingWave database, and pass the real-time data stream to the RisingWave database. At this time, the RisingWave database is configured to: receive the streaming front-end data transmitted by the stream processing platform; obtain the SQL query statement for the front-end data; create a materialized view for the front-end data, and after executing the streaming computing task corresponding to the aforementioned SQL query in the background, obtain the materialized view query result for the front-end data; when the data source changes, calculate the change of the front-end data, and calculate the change of the materialized view query result for the front-end data based on the change of the front-end data, and automatically update the query result.

[0083] In this way, a SQL query can be triggered again for the filtering, analysis and / or conversion results of the materialized view, and a new materialized view can be established again based on the SQL query to perform derivative data analysis.

[0084] The present invention creates a materialized view, uses the query SQL as a background running task, and uses the real-time processing result as the query result of the materialized view. If any data changes, RisingWave will automatically increase and update the materialized view query result.

[0085] As an example and not a limitation, the SQL code for creating a materialized view (analyzed through a select statement) through RisingWave can be as follows:

[0086] createMATERIALIZEDVIEWmv_bond_anomaly as

[0087] select bt.timestamp,bt.symbol,bt.bid,yb.price

[0088] from bond_tickbt

[0089] leftjoinyesterday_bondyb on bt.symbol=yb.symbol

[0090] where bt.bid <yb.price*0.75;

[0091] --You can directly query the materialized view results

[0092] select * from bond_anomaly;

[0093] After configuring the data source and materialized view, you can start ingesting data. The materialized view of the RisingWave database is dynamic and changes in real time based on the data from the data source.

[0094] Furthermore, the materialized view query result for the front-end data can be stored as the front-end data and output to the stream processing platform in real time to form a real-time data stream; for the real-time data stream, a new materialized view is established again based on the SQL query to analyze the multi-level derivative data. Preferably, derivative identification information is configured for each materialized view, and the derivative identification information includes the data derivative level and the front-end data information. Through the derivative identification information, it is possible to obtain the level of derivative data to which the materialized view belongs and the front-end data information related to the materialized view.

[0095] In this embodiment, the materialized view includes data filtering, analysis, and conversion processing tasks. When the built-in functions of the RisingWave database cannot meet specific data processing requirements, you can configure related custom functions UDF (User-Defined Functions) to achieve this. In this way, users can configure custom functions to use custom logic in materialized view queries, such as data filtering (cleaning), data analysis, data conversion processing, etc. for specific needs.

[0096] Specifically, the real-time data stream processing system may further include a function extension module, and the function extension module is used for a user to configure a user-defined function UDF.

[0097] The RisingWave database is configured to: obtain the UDF information configured by the user in the function extension module, and register the UDF in the server of the RisingWave database; after the UDF is successfully registered, the RisingWave database can call the aforementioned UDF from the server for calculation.

[0098] In this embodiment, the UDF configuration method provided by the function extension module may be as follows.

[0099] Method 1: Create a UDF (i.e. SQL UDF) directly in the RisingWave database through SQL language to directly expand simple function combinations. For example, you can configure a SQL UDF to add two parameters. For input parameters, you can choose whether to name them when defining them.

[0100] Method 2: Use Python or Java to create the corresponding UDF in RisingWave and start an independent process to provide function services. Compared with UDFs defined in other languages, creating JavaScript UDF is simpler because the QuickJS virtual machine is built into the RisingWave database.

[0101] Method 3: Use Rust to create UDF in the RisingWave database; embed the WASM binary file into SQL in base64 encoding, or load the WASM binary file from object storage. Specifically, implement the interface provided by RisingWave through Rust, then package it into a WASM package and upload it to the RisingWave server.

[0102] This embodiment creates a data output Sink through RisingWave, which can output the data query results.

[0103] As an example and not a limitation, the SQL code for creating a data output Sink through RisingWave can be as follows:

[0104] create sink sink_bond_anomaly from mv_bond_anomaly

[0105] WITH(

[0106] connector = 'kafka',

[0107] topic = 'bond_tick',

[0108] properties.bootstrap.server='message_queue:9092',

[0109] topic = 'bond_anomaly'

[0110] )FORMAT PLAIN ENCODE JSON;

[0111] As needed, you can also configure the Sink to send data analysis results to other downstream systems.

[0112] The present invention also provides a real-time data stream processing system based on RisingWave, the system comprising a RisingWave database and a stream processing platform;

[0113] The stream processing platform is used to form a real-time data stream from the received real-time data, and transmit the real-time data stream to the RisingWave database;

[0114] The RisingWave database supports data ingestion, streaming computing, and real-time output through structured query language SQL;

[0115] Among them, the RisingWave database is configured as follows: configure the data source Source and the table structure Table, obtain the real-time data stream from the configured data source; obtain the SQL query statement, the SQL query statement is used to filter, analyze and / or convert the data; create a materialized view, parse the query SQL into a streaming processing process, execute the streaming computing task in the background, and use the real-time computing results as the query results of the materialized view. When the data source changes, the query results of the materialized view are automatically updated; configure the data output Sink to perform data result output processing.

[0116] For other technical features, please refer to the description of the previous embodiment and will not be repeated here.

[0117] The present invention also provides an electronic device, comprising:

[0118] a storage device having a computer program stored thereon;

[0119] A processing device is used to execute the computer program in the storage device to implement the steps of any of the above methods.

[0120] For other technical features, please refer to the description of the previous embodiment and will not be repeated here.

[0121] The present invention also provides a computer-readable storage medium for storing a computer program that can be executed by a processing unit, wherein the computer program implements the above-mentioned method when executed by the processing unit.

[0122] The storage medium may include any medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0123] For other technical features, please refer to the description of the previous embodiment and will not be repeated here.

[0124] In the above description, the disclosure of the present invention is not intended to limit itself to these aspects. Rather, within the scope of the target protection of the present disclosure, the components can be selectively and operatively combined in any number. In addition, terms such as "include", "encompass" and "have" should be interpreted as inclusive or open by default, rather than exclusive or closed, unless they are explicitly defined to the contrary. All technical, scientific or other terms are consistent with the meanings understood by those skilled in the art, unless they are defined to the contrary. Common terms found in dictionaries should not be interpreted too idealistically or too impractically in the context of relevant technical documents, unless the present disclosure explicitly defines them as such. Any changes and modifications made by a person of ordinary skill in the field of the present invention based on the above disclosure belong to the scope of protection of the claims.

Claims

1. A real-time data stream processing method based on RisingWave, characterized in that: Build a real-time data stream processing system based on RisingWave, which includes a RisingWave database and a stream processing platform; the stream processing platform is used to form a real-time data stream from the received real-time data and transfer the real-time data stream to the RisingWave database; the RisingWave database supports data ingestion, streaming computing and real-time output through the structured query language SQL; The RisingWave database is configured as follows: configure the data source Source and the table structure Table, obtain the real-time data stream from the configured data source; obtain the SQL query statement, which is used to filter, analyze and / or convert the data; create a materialized view, parse the query SQL into a streaming process, execute the streaming computing task in the background, use the real-time computing result as the query result of the materialized view, and automatically update the query result of the materialized view when the data source changes; configure the data output Sink to perform data result output processing; Among them, the stream processing platform is Kafka. At this time, the data source configured for the RisingWave database is Kafka.

2. The method according to claim 1, characterized in that: The RisingWave database stores the query results of the materialized view as the pre-data, and outputs the pre-data to the stream processing platform in real time by configuring the data output Sink; The stream processing platform is configured to: form a real-time data stream after receiving the pre-data transmitted by the RisingWave database, and transmit the real-time data stream to the RisingWave database; At this time, the RisingWave database is configured to: receive the streaming pre-data transmitted by the stream processing platform; obtain the SQL query statement for the pre-data; create a materialized view for the pre-data, and after executing the streaming computing task corresponding to the aforementioned SQL query in the background, obtain the query result of the materialized view for the pre-data; When the data source changes, the change of the preceding data is calculated, and based on the change of the preceding data, the change of the materialized view query result for the preceding data is calculated, and the query result is automatically updated.

3. The method according to claim 2, characterized in that: The materialized view query result for the front-end data can be stored as the front-end data and output to the stream processing platform in real time to form a real-time data stream; for the real-time data stream, a new materialized view is established again based on the SQL query.

4. The method according to any one of claims 1 to 3, characterized in that: The stream processing platform of the real-time data stream processing system is connected with a data provider; The data provider continuously generates real-time data and transmits it to the stream processing platform; The stream processing platform is configured to: after receiving the aforementioned real-time data, process the received real-time data as a continuous stream of events or messages to form a real-time data stream, and then deliver the real-time data stream to the RisingWave database.

5. The method according to any one of claims 1 to 3, characterized in that: The RisingWave database and stream processing platform of the real-time data stream processing system are connected with the business system; The business system is used to receive the user's SQL query request and send the SQL query request to the RisingWave database; The stream processing platform is configured to: obtain the materialized view query result transmitted by the RisingWave database, and send the materialized view query result to the business system.

6. The method according to claim 1, characterized in that: The real-time data stream processing system further comprises a function extension module, wherein the function extension module is used for a user to configure a user-defined function UDF; The RisingWave database is configured to: obtain the UDF information configured by the user in the function extension module, and register the UDF in the server of the RisingWave database; after the UDF is successfully registered, the RisingWave database can call the aforementioned UDF from the server to perform calculations.

7. The method according to claim 6, characterized in that The UDF configuration methods provided by the function extension module include: Create UDFs within the RisingWave database using SQL to directly extend simple function combinations; or, Use Python or Java to create the corresponding UDF in RisingWave and start an independent process to provide function services; or, Use Rust to create UDFs in the RisingWave database, embed WASM binaries into SQL in base64 encoding, or load WASM binaries from object storage.

8. A real-time data stream processing system based on RisingWave, characterized in that: Includes the RisingWave database and stream processing platform; The stream processing platform is Kafka, which is used to form a real-time data stream from the received real-time data and transmit the real-time data stream to the RisingWave database; The RisingWave database supports data ingestion, streaming computing, and real-time output through structured query language SQL; Among them, the RisingWave database is configured as follows: configure the data source Source and the table structure Table, obtain real-time data stream from the configured data source, the data source is Kafka; obtain SQL query statements, the SQL query statements are used to filter, analyze and / or convert data; create materialized views, parse the query SQL into a streaming processing process, execute streaming computing tasks in the background, and use the real-time computing results as the query results of the materialized view. When the data source changes, the query results of the materialized view are automatically updated; configure the data output Sink to perform data result output processing.

9. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 7.