Real-time data processing and analysis system and method based on low-code platform

Abstract

The invention discloses a real-time data processing and analyzing system and method based on a low-code platform, and the system comprises the low-code platform, a data connection connector, a data mapping module, a database system module, a template management module, a process analysis engine, an Flink real-time data processing module and a real-time data display module. The man-machine interaction module is an entrance of the data leading connector, the data mapping module, the template management module and the real-time data display module and is used for man-machine interaction; according to the method, real-time data processing and analysis are carried out based on the system; according to the invention, real-time data connection, dynamic configuration of the data processing model, real-time data processing and real-time processing result display can be realized.

Description

Technical Field

[0001] This invention relates to a data processing and analysis system and method, and more particularly to a real-time data processing and analysis system and method based on a low-code platform. Background Technology

[0002] This section provides only background information relevant to this disclosure and is not necessarily prior art.

[0003] With the rapid development of networks and sensors, the volume of data has increased unprecedentedly. To efficiently utilize this data, real-time data processing technology has emerged. However, there are some differences in real-time data processing across different fields. To meet the real-time data processing requirements of certain specific fields, it is necessary to provide a flexible real-time data processing and analysis system.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a real-time data processing and analysis system and method based on a low-code platform, which addresses the shortcomings of the prior art.

[0006] To address the aforementioned technical problems, this invention discloses a real-time data processing and analysis system and method based on a low-code platform, wherein the system includes:

[0007] The system includes a low-code platform, data connectors, data mapping modules, a database system module, a template management module, a process parsing engine, a Flink real-time data processing module, and a real-time data visualization module; among these,

[0008] The low-code platform serves as the entry point for the data connector, data mapping module, template management module, and real-time data display module, and is used for human-computer interaction.

[0009] The data connector is used to connect real-time data generated by different data sources;

[0010] The data mapping module and the database system module map and store the incoming real-time data, converting it into internal data that conforms to the system configuration;

[0011] The template management module configures data processing methods by configuring templates and uses the templates to build real-time data processing workflows.

[0012] The process parsing engine parses the real-time data processing process into executable code;

[0013] The Flink real-time data processing module loads executable code and performs real-time data processing to obtain the processing results;

[0014] The real-time data display module displays the processing results according to the configured format.

[0015] Furthermore, the data connector is used to acquire real-time data, the format of which is configured by the low-code platform. The acquired real-time data is sent to the data receiving bus of the low-code platform. For different data sources, corresponding data connector mirrors and data receiving bus mirrors are created. The data connector includes a database connector, an HTTP request connector, and a local file connector. The database connector includes relational databases and non-relational databases. The HTTP request connector obtains the remote server address in a dynamically configured manner. The local file connector includes JSON file connectors and XML file connectors.

[0016] Furthermore, the data mapping module includes a target data table format editing submodule and a data conversion program engine; wherein, the target data table format editing submodule is used to edit the target data table format and send it to the data conversion program engine; the data conversion engine creates a database table in the database according to the data format provided by the low-code platform and the data content provided by the target data table format editing submodule.

[0017] Furthermore, the database system module consists of multiple DM database servers, realizing a database cluster;

[0018] Furthermore, the template management module configures a real-time data processing method for template management, including a data preprocessing submodule and a data calculation submodule. The template contains data preprocessing and data calculation methods, which are parsed and displayed on a low-code platform to construct a real-time data processing flow. Once the real-time data processing flow is constructed, the flow parsing engine is triggered.

[0019] Furthermore, the process parsing engine loads the real-time data processing process, checks the format specifications and data specifications of the real-time data processing process, and converts the real-time data processing process into executable code, which is used for real-time data processing.

[0020] Furthermore, the Flink real-time data processing module, including the code loading module and the Hadoop cluster, adopts a cluster deployment method; the code loading module loads the executable code obtained after the process parsing engine is converted into the Flink platform, triggering the Flink platform to perform real-time data processing according to the executable code.

[0021] This invention also proposes a real-time data processing and analysis method based on a low-code platform. The method is implemented using the aforementioned system and specifically includes:

[0022] Step 1: Use the data connector to connect to real-time data generated by different data sources;

[0023] Step 2: Using the data mapping module and the database system module, the incoming real-time data is mapped and stored, and converted into internal data that conforms to the system configuration;

[0024] Step 3: Configure the data processing method by configuring a preset template or adding a new template;

[0025] Step 4: Use the template to build a real-time data processing flow;

[0026] Step 5: Use the process parsing engine to parse the real-time data processing process into executable code;

[0027] Step 6: The Flink real-time data processing module loads the executable code and performs real-time data processing to obtain the processing result;

[0028] Step 7: Use the real-time data display module to display the processing results in the configured format.

[0029] Furthermore, step 5, which involves parsing the real-time data processing flow into executable code, includes:

[0030] The data processing flowchart is parsed as character data; the character data is parsed as model parameters; and the preset code is activated based on the model parameters.

[0031] Furthermore, step 4 involves using the template to construct a real-time data processing flow, which means using the template to generate a data processing flow chart by dragging and dropping.

[0032] Beneficial effects:

[0033] This invention can achieve data input from various data sources, meet the requirements of dynamic editing data processing methods, support the addition of data processing methods, support the storage and display of data processing results, and support a real-time data processing and analysis system architecture with high real-time performance and high concurrency. Attached Figure Description

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0035] Figure 1 This is a flowchart of a real-time data processing method provided in an embodiment of the present invention.

[0036] Figure 2 This is a flowchart of a data transfer method provided in an embodiment of the present invention.

[0037] Figure 3 This is a master-slave database structure diagram provided in an embodiment of the present invention.

[0038] Figure 4 This is a template management flowchart provided in an embodiment of the present invention.

[0039] Figure 5 This is a flowchart of a data processing method provided in an embodiment of the present invention.

[0040] Figure 6 This is a flowchart of a real-time data processing process on a Flink platform provided in an embodiment of the present invention.

[0041] Figure 7 This is a flowchart of a real-time processing result display provided by an embodiment of the present invention. Detailed Implementation

[0042] The purpose of this invention is to fully utilize the modular, visual, flexible and convenient features of low-code platforms, so that operators can quickly configure real-time data processing systems, thereby optimizing data acquisition, processing and display modes, and realizing a real-time, efficient, flexible and scalable real-time data processing and analysis system, which can efficiently utilize real-time data in complex environments.

[0043] The technical solution adopted by this invention to achieve the above objectives is as follows: a real-time data processing and analysis system based on a low-code platform. This system is based on a low-code platform and connects a data acquisition module, a data processing module, and a data display terminal in a loosely coupled manner, allowing them to interact and form a user-facing system architecture. The system includes a low-code platform, a data acquisition connector, a data mapping module, a database system module, a template management module, a process parsing engine, a Flink real-time data processing module, and a real-time data display module.

[0044] The data connectors include a database connector, an HTTP request connector, and a local file connector.

[0045] The database connector includes relational databases and non-relational databases. The relational databases include MySQL, Oracle, and DM databases, while the non-relational databases include Redis and Neo4j.

[0046] The HTTP request connector obtains the remote server address through dynamic configuration, which includes two configuration methods: low-code platform configuration and local XML configuration.

[0047] The local file connectors include JSON file connectors and XML file connectors.

[0048] The data connector is used to acquire real-time data. The data format acquired by the connector needs to be configured through a low-code platform. The acquired real-time data is then sent to the data receiving bus. If multiple different data sources exist, the system will create multiple data connector mirrors and data receiving bus mirrors to ensure that different data sources do not interfere with each other.

[0049] The data mapping module includes a target data table format editing and data conversion program engine.

[0050] The source and target data fields are obtained through a data connector and a source database table. The source and target data formats will be displayed on the low-code platform.

[0051] The target data table format and the mapping between source and target data are completed online through a low-code platform. The edited format is then published from the low-code platform to the data transformation engine.

[0052] The data transformation engine creates corresponding database tables in the database according to the data format provided by the low-code platform, and the mapping between source data and target data is completed by the data processing tool.

[0053] The data format file encoding method can be configured through a configuration file.

[0054] The data transformation engine is responsible for transforming the source data using data processing tools and saving it to the target database.

[0055] Data processing tools can create multiple mirrors to handle different tasks.

[0056] The database system is implemented by a database cluster consisting of multiple DM database servers.

[0057] Each database server is responsible for reading and writing the segments it manages.

[0058] Each database server uses master-slave replication to ensure data consistency and backup.

[0059] The template management module is used to manage real-time data processing methods.

[0060] The real-time data processing method includes a data preprocessing module and a data calculation module.

[0061] Template management offers two methods: preset templates and new templates. Preset templates are built into the system during deployment and include common data preprocessing and calculation methods. New templates are developed based on the specific needs of the user environment and utilize data preprocessing and calculation methods for specific functions, occasions, and data.

[0062] The newly added templates must be developed in strict accordance with the prescribed development specifications. After development is completed, the templates must be packaged according to the prescribed packaging method and uploaded to the low-code platform before they can be parsed by the template management module for real-time data processing.

[0063] The available templates mentioned above will be displayed on the low-code platform for use in building subsequent real-time data processing workflows.

[0064] The real-time data processing workflow is built on a low-code platform and is a data processing method built on the template method.

[0065] Once the real-time data processing flow is constructed, the flow parsing engine is triggered.

[0066] The process parsing engine loads real-time data processing process data, checks the process format specifications and data specifications, and converts the process data into executable code after the checks are passed. This code is used for real-time data processing.

[0067] The Flink real-time data processing module includes a code loading module and a Hadoop cluster, and is deployed using a standalone cluster approach.

[0068] The code loading module is used to load the module converted by the parsing engine into the Flink platform. After loading is completed, the platform is triggered and begins to perform real-time data processing according to the code.

[0069] The real-time data processing results are sent to the message bus. The low-code platform can obtain the real-time data processing results and display them according to the configured display format. The display results support dynamic configuration and take effect immediately after configuration.

[0070] Example 1:

[0071] As mentioned above, in real-time computing tasks in related technologies, the processing methods include a lot of redundant work that needs to be performed manually, including data import and writing data processing methods. This results in long processing flows and low efficiency for real-time data computing tasks. Data processing tasks are heavily dependent on data formats, and data processing logic is heavily dependent on professional coders. It is difficult for non-professional operators to dynamically import data and edit data processing logic according to requirements.

[0072] This disclosure provides a real-time data processing and analysis system based on a low-code platform. Leveraging the convenient, efficient, and easy-to-configure features of the low-code platform, it allows even non-professional operators to quickly configure the real-time data processing system. This optimizes data acquisition, processing, and display modes, enabling a real-time, efficient, flexible, and scalable real-time data processing and analysis system, facilitating efficient utilization of real-time data in complex battlefield environments.

[0073] First, the steps of a real-time data processing and analysis system based on a low-code platform in this exemplary embodiment will be described in more detail with reference to the accompanying drawings and embodiments.

[0074] refer to Figure 1 Real-time data processing and analysis systems based on low-code platforms include...

[0075] Real-time target data is obtained through data interception. This requires configuring source database configuration information and the source data REST request address. Source database configuration information includes the source database type, source database connection configuration information, and a list of fields; the source data REST request address includes the source target address field, source target port number, source target request prefix, and a list of fields.

[0076] The data mapping module maps the source data format to the target data format. The configuration information includes the target data format, target database configuration information, mapping field table, etc.

[0077] The database system consists of a master-slave database configuration and a task allocation engine. The master-slave database configuration improves the system's fault tolerance and read / write efficiency. The task allocation engine is used to assign tasks to different databases. When the engine receives a data read / write request, it forwards the task to the appropriate database based on the content of the request.

[0078] Template management is a module for managing data processing logic, supporting both preset templates and the addition of new templates. For example, operators can use built-in template methods to process real-time data during actual system use. When new data processing needs arise, if the system itself does not include corresponding template methods, operators can write appropriate template methods according to development specifications, upload them, and then use those methods to process real-time data.

[0079] Once the operator has edited the data processing workflow through the low-code platform, the workflow will be passed to the workflow parsing engine.

[0080] The process parsing engine will parse the edited process and convert it into executable code stored in a fixed location. After the process parsing engine finishes processing, it will trigger the Flink real-time data processing part.

[0081] After receiving the trigger command, the Flink platform obtains the parsed code and loads it into the Flink platform.

[0082] The Flink platform executes the corresponding code for real-time data processing.

[0083] The real-time data processing results will be stored in a fixed location, and the data processing results will also be sent to the message bus.

[0084] After the low-code platform obtains data from the message bus, it performs some simple processing and displays it on the front-end interface. The display results serve as a reference for strategic decision-making.

[0085] Figure 2 A flowchart of the data transfer method in this embodiment is shown.

[0086] When the data source is a database, you need to configure the source database configuration information, the source database REST request address, etc. The source database configuration information includes the source database type, source database connection configuration information, field list, etc.

[0087] When the data source is a REST request, the REST request address needs to be configured, including the source target address field, source target port number, source target request prefix, field list, etc.

[0088] Once the data source parameters are configured, the system will connect to the data source and display the data source field format on the front end.

[0089] The next step is to configure the source data format. This involves dynamically creating the database table format through a visual interface, such as the database table field names and data types.

[0090] After the data source format is edited, the system will create the corresponding database table in the target database.

[0091] The next step is to configure the mapping relationship. The operation interface allows you to easily establish mappings between different fields of two tables, and also supports mapping multiple fields to the same field.

[0092] After configuring the mapping relationship, trigger the data import operation. The system will retrieve real-time data from the data source and store it in the target database according to the mapping relationship.

[0093] Figure 3The diagram illustrates the master-slave database structure provided in this embodiment. This system employs a conventional master-slave database design, with the master database used for write operations and the slave database used for read operations. Unlike conventional database systems, the database system provided in this embodiment functionally includes storage for data ingress, a data source for real-time processing, and storage for real-time processing results. The primary function of the database is reading and writing, not modification; therefore, the core design principle of this database system is to ensure data consistency, high availability, and load balancing for read operations.

[0094] To ensure data consistency and high availability, the master database is used only for write operations, not regular read and write operations, while the slave database is used only for read operations.

[0095] As mentioned above, another purpose of a database system is load balancing. This embodiment of the invention also provides a load balancing engine that uses a load balancing algorithm to send different requests to different database systems. A common implementation of this load balancing algorithm is through key hash mapping. Other implementation methods are not listed here.

[0096] Figure 4 This embodiment illustrates a template management flowchart. A template is a configuration file and executable code configured according to certain specifications. The template file defines information such as model structure, data types, data preprocessing strategies, and data processing methods. The process parsing engine can obtain data processing methods through the template. This system provides a variety of preset templates, including commonly used data preprocessing strategies and data loading methods.

[0097] The template will be displayed as a component in the low-code platform, and operators can build different real-time data processing strategies by dragging and dropping template components.

[0098] Figure 5 This embodiment illustrates a data processing method editing flowchart. Using a low-code platform, the operator edits the data processing flowchart via drag-and-drop on a visual interface. This flowchart is then passed to the flowchart parsing engine. Simply put, the flowchart parsing engine maintains a dictionary containing all currently included template methods. Upon receiving the data processing flowchart, the engine initiates a detection mechanism to check if the data formats on the method chain match based on the input and output. If a mismatch is found, an error message is sent to the low-code platform, requesting the operator to check the flowchart. After passing the detection mechanism, the code assembly process begins. Since Java lacks a decorator mechanism and hook methods similar to Python, dynamic code assembly is achieved here using the chain of responsibility pattern and interface callback functions.

[0099] Figure 6This embodiment illustrates a real-time data processing flowchart for the Flink platform. After being parsed by the process parsing engine in the previous step, the data processing flowchart is converted into corresponding code and saved. A small program is used here to handle the issue of loading the code onto the Flink platform. This small program is launched when the process parsing engine finishes saving the executable code. The program submits the code to the Flink platform. After this step, the Flink platform's job manager is launched, and the job manager distributes the corresponding code to the task manager to begin the real-time data processing task. Since this is not the focus of this invention, it will not be elaborated upon further. After completing the data processing, the Flink platform saves the real-time data processing results.

[0100] Figure 7 This embodiment illustrates a flowchart for displaying real-time processing results. Based on the above, after completing data processing, the Flink platform saves the real-time data processing results to the database and pushes them to the message bus. The low-code platform receives the data results from the message bus and formats them according to the display style edited by the operator. After processing, it uses Echarts to visualize the results. It is worth noting that this process is completely decoupled from the real-time data processing task. Operators can edit the result display style during the real-time data processing task without affecting the real-time data processing. Furthermore, the edited style takes effect immediately.

[0101] Example 2:

[0102] This embodiment constructs a "real-time target monitoring and early warning system" to import target data from multiple heterogeneous data sources (such as radar detection databases and satellite reconnaissance REST interfaces) in real time, perform fusion, deduplication, threat calculation and other processing, and visualize the processed target situation and early warning information in real time to provide decision support for commanders.

[0103] Data import configuration

[0104] 1. Configure the radar database source:

[0105] Source database type: PostgreSQL

[0106] Connection configuration: IP address (192.168.1.100), port (5432), database name (radar_db), username / password.

[0107] The o field list includes: target_id (target ID), longitude (longitude), latitude (latitude), speed (speed), timestamp (timestamp), and radar_id (radar station number).

[0108] 2. Configure satellite REST data source:

[0109] oREST request address: http: / / satellite-data.com / api / v1 / targets

[0110] Source target port number: 80

[0111] O source target request prefix: / api / v1

[0112] o Source target addressing field: target_code (corresponding to the target_id of the radar data)

[0113] The o field list includes: target_code, lat, lon, altitude (height), type (type), and time.

[0114] 3. The system connects to two data sources respectively and displays the field structure of the radar data table (radar_targets) and the satellite data table (sat_targets) on the low-code platform interface.

[0115] Define target data format and create tables

[0116] 1. Operators dynamically define the format of the merged target data table (fused_targets) within the visual interface of the low-code platform:

[0117] o Table name: fused_targets

[0118] o fields: uid (primary key, VARCHAR), longitude (DOUBLE), latitude (DOUBLE), altitude (DOUBLE), speed (DOUBLE), type (VARCHAR), threat_level (INT, threat level), source (VARCHAR, data source), update_time (TIMESTAMP).

[0119] 2. After configuration, the system automatically creates the fused_targets table in the target database (which has a master-slave structure configured).

[0120] Configure data mapping relationship

[0121] 1. In the mapping configuration interface of the low-code platform, establish field mappings between the source data and the target data:

[0122] oradar_targets.target_id + radar_id → fused_targets.uid (for example: T001-Radar02)

[0123] osat_targets.target_code → fused_targets.uid (prefixed with SAT-)

[0124] oradar_targets.longitude / sat_targets.lon → fused_targets.longitude

[0125] oradar_targets.latitude / sat_targets.lat → fused_targets.latitude

[0126] osat_targets.altitude → fused_targets.altitude

[0127] oradar_targets.speed → fused_targets.speed

[0128] osat_targets.type → fused_targets.type

[0129] oSystem time → fused_targets.update_time

[0130] 2. Configure "Multi-source association" logic: When radar and satellite data describe the same target (associating target_id and target_code through specific rules), the satellite's altitude and type information are used first and fused into a single record.

[0131] Edit the real-time data processing workflow (low-code platform)

[0132] 1. The operator enters the flowchart editing interface of the low-code platform.

[0133] 2. Drag and drop pre-built components from the template management module onto the canvas:

[0134] o Data source component: Points to the configured radar_targets and sat_targets (logically as Kafka streams).

[0135] o Data fusion component (pre-built template): Based on spatiotemporal proximity and ID association rules, it performs JOIN and deduplication on two streams.

[0136] o Threat Assessment Component (Custom Upload Template): Calculates threat level (1-5) based on rules such as target speed, type, and distance from critical areas.

[0137] o Data output component: Specifies the output to the target database fused_targets table and message bus (such as KafkaTopic target-result).

[0138] 3. Connect the components in the order of "Data Source → Fusion → Threat Assessment → Output" using connecting lines to form a complete data processing flowchart.

[0139] Process Analysis and Code Generation

[0140] 1. When the operator clicks the "Deploy" button, the flowchart is sent to the process parsing engine.

[0141] 2. The engine checks the validity of the process (e.g., whether the data types match: whether the latitude and longitude are double), and the check passes.

[0142] 3. Based on the internal dictionary, the engine parses the "data fusion component" into specific FlinkCoProcessFunction or Interval Join code segments, and the "threat assessment component" into MapFunction code segments.

[0143] 4. The engine uses the chain of responsibility pattern to assemble these code segments with the source and output code into a complete FlinkStream Job (Java code) and saves it to the code repository.

[0144] Flink task submission and execution

[0145] 1. The process parsing engine triggers the task submission mini-program.

[0146] 2. This mini-program packages the generated Flink Job code (JAR) and submits it to the Flink cluster via the REST API.

[0147] 3. The Flink JobManager receives tasks and assigns them to TaskManagers for execution.

[0148] 4. Task begins execution: Consume radar and satellite data streams in real time, perform fusion and threat calculations, and continuously write the results to the main database (write operation), while also outputting to the message bus Topic target-result.

[0149] Results Display

[0150] 1. The low-code platform's presentation module subscribes to the message bus Topic target-result to obtain the processed target data in real time.

[0151] 2. Operators can drag and drop map components, table components, and threat level pie chart components in the platform's "Visual Dashboard" editing interface.

[0152] o Configure the map component: bind the longitude and latitude fields to coordinates, and use the threshold_level field to determine the target icon color.

[0153] o Configure table component: Display target details.

[0154] Configure alarm rules: When threat_level >= 4, a red warning banner will pop up at the top of the interface.

[0155] 3. Once editing is complete, the dashboard refreshes immediately. The map begins to display the trajectory of dynamically moving targets in real time, with different colors distinguishing threat levels, and the table data updates continuously. When a high-threat target appears, the system automatically pops up an alert.

[0156] Through this embodiment, operators do not need to write complex backend processing code (Java / Scala) and real-time computing framework (Flink) code. They can quickly build and deploy a high-efficiency "real-time target monitoring and early warning system" simply by visual configuration and drag-and-drop. This achieves a closed-loop process from multi-source data acquisition and real-time fusion processing to decision visualization, which greatly improves system development efficiency and business response speed.

[0157] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content and some or all of the steps in various embodiments regarding a real-time data processing and analysis system and method based on a low-code platform. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0158] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MCU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0159] This invention provides an idea and method for a real-time data processing and analysis system and method based on a low-code platform. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A real-time data processing and analysis system based on a low-code platform, characterized in that, The system includes: The system includes a low-code platform, data connectors, data mapping modules, a database system module, a template management module, a process parsing engine, a Flink real-time data processing module, and a real-time data visualization module; among these, The low-code platform serves as the entry point for the data connector, data mapping module, template management module, and real-time data display module, and is used for human-computer interaction. The data connector is used to connect real-time data generated by different data sources; The data mapping module and the database system module map and store the incoming real-time data, converting it into internal data that conforms to the system configuration; The template management module configures data processing methods by configuring templates and uses the templates to build real-time data processing workflows. The process parsing engine parses the real-time data processing process into executable code; The Flink real-time data processing module loads executable code and performs real-time data processing to obtain the processing results; The real-time data display module displays the processing results according to the configured format.

2. The real-time data processing and analysis system based on a low-code platform according to claim 1, characterized in that, The data connector is used to acquire real-time data. The format of the real-time data is configured by the low-code platform, and the acquired real-time data is sent to the data receiving bus of the low-code platform. For different data sources, create corresponding data connector images and data receive bus images; The data connection connector includes a database connector, an HTTP request connector, and a local file connector; wherein, the database connection connector includes relational databases and non-relational databases; the HTTP request connector obtains the remote server address in a dynamically configured manner; The local file connector includes a JSON file connector and an XML file connector.

3. The real-time data processing and analysis system based on a low-code platform according to claim 2, characterized in that, The data mapping module includes a target data table format editing submodule and a data conversion program engine; wherein, the target data table format editing submodule is used to edit the target data table format and send it to the data conversion program engine; the data conversion engine creates a database table in the database according to the data format provided by the low-code platform and the data content provided by the target data table format editing submodule.

4. The real-time data processing and analysis system based on a low-code platform according to claim 3, characterized in that, The database system module consists of multiple DM database servers, realizing a database cluster.

5. A real-time data processing and analysis system based on a low-code platform according to claim 4, characterized in that, The template management module configures templates to manage real-time data processing methods, including a data preprocessing submodule and a data calculation submodule. The template contains data preprocessing and data calculation methods, which are parsed and displayed on a low-code platform to build a real-time data processing flow. Once the real-time data processing flow is built, the flow parsing engine is triggered.

6. A real-time data processing and analysis system based on a low-code platform according to claim 5, characterized in that, The process parsing engine loads the real-time data processing process, checks the format and data specifications of the real-time data processing process, and converts the real-time data processing process into executable code, which is used for real-time data processing.

7. A real-time data processing and analysis system based on a low-code platform according to claim 6, characterized in that, The Flink real-time data processing module includes a code loading module and a Hadoop cluster, and is deployed in a cluster manner. The code loading module loads the executable code obtained after the process parsing engine is converted into the Flink platform, triggering the Flink platform to perform real-time data processing according to the executable code.

8. A real-time data processing and analysis method based on a low-code platform, characterized in that, The method is implemented using any of the systems described in claims 1-7, and specifically includes: Step 1: Use the data connector to connect to real-time data generated by different data sources; Step 2: Using the data mapping module and the database system module, the incoming real-time data is mapped and stored, and converted into internal data that conforms to the system configuration; Step 3: Configure the data processing method by configuring a preset template or adding a new template; Step 4: Use the template to build a real-time data processing flow; Step 5: Use the process parsing engine to parse the real-time data processing process into executable code; Step 6: The Flink real-time data processing module loads the executable code and performs real-time data processing to obtain the processing result; Step 7: Use the real-time data display module to display the processing results in the configured format.

9. A real-time data processing and analysis method based on a low-code platform according to claim 8, characterized in that, Step 5, which involves parsing the real-time data processing flow into executable code, includes: The data processing flowchart is parsed as character data; the character data is parsed as model parameters; and the preset code is activated based on the model parameters.

10. A real-time data processing and analysis method based on a low-code platform according to claim 9, characterized in that, The step 4 describes using the template to build a real-time data processing flow, which means using the template to generate a data processing flow chart by dragging and dropping.

Images