A method for realizing BI visualization large screen high-concurrency data real-time updating based on WebSocket

Abstract

The application relates to the field of data updating, and particularly discloses a method for realizing high-concurrency data real-time updating of BI visual large screens based on WebSocket, which comprises the following steps: building a BI large screen through dragging and dropping components; adding components to a scene in a scene editor through the mode of dragging and dropping; configuring and connecting to a scene WebSocket service; setting a component data source type as WebSocket, configuring an interface name agreed on by a server, and realizing bidirectional communication between the component and the server through an intermediate layer data management module. Through the innovative WebSocket communication mechanism, the intermediate layer data management module, dynamic connection management and load balancing technology and the like, the application provides an efficient, reliable, safe and easily-extended solution for data real-time updating of BI visual large screens in a high-concurrency scene, and has obvious practical value and wide application prospect.

Description

Technical Field

[0001] This invention relates to the field of data update technology, and in particular to a method for real-time, high-concurrency data updates for BI visualization dashboards based on WebSocket. Background Technology

[0002] With the rapid development of big data technology and visualization tools, enterprises' demand for data visualization is constantly increasing. Traditional static or polled data reports can no longer meet the high requirements of modern enterprises for real-time performance, interactivity, and decision support. BI (Business Intelligence) visualization dashboard technology has emerged to address this need. Fulicode RBI Business Intelligence Designer is a domestically developed online BI tool that enables business data visualization and analysis through low-code drag-and-drop operations. It helps users quickly build industrial internet application scenarios such as management dashboards, digital dashboards, and smart cities, achieving rapid and accurate decision-making and improving enterprise management efficiency.

[0003] In existing technologies, BI dashboard data updates are mostly implemented through a timed polling mechanism: users set the refresh cycle for each chart during the configuration phase, and the system sends requests to the backend to update the chart data according to this cycle. This mechanism has obvious drawbacks:

[0004] (1) Serious waste of resources: Regardless of whether the data changes, the system continues to send requests, resulting in a large amount of redundant data interaction and wasting server resources and bandwidth;

[0005] (2) Untimely data response: The fixed refresh cycle makes data updates lagging, which affects the response to emergencies;

[0006] (3) Security risks: Frequent requests may be maliciously exploited, increasing the risk of system attacks (such as DDoS) or data leaks;

[0007] (4) Significant performance bottlenecks: In environments with large data volumes and high connection concurrency, the traditional WebSocket implementation is highly coupled to specific charts and data sources, resulting in complex system structures, poor maintainability, and difficulty in supporting large-scale concurrent access and dynamic business needs.

[0008] The existing patent, CN111522565B, which discloses a method and system for real-time data visualization based on componentization, mentions that WebSocket technology is used to replace the polling mechanism. However, the design still requires binding a dedicated WebSocket connection to each chart and customizing the backend logic, resulting in extremely high system coupling and severely restricting scalability and maintainability.

[0009] Therefore, there is an urgent need for a real-time data push mechanism that has a clear structure, strong decoupling, and can support high-concurrency access. Summary of the Invention

[0010] The purpose of this invention is to address the following problems existing in the WebSocket application in BI visualization dashboards: firstly, high coupling and high concurrency; secondly, system performance degradation due to high concurrency access during data updates; and thirdly, the poor timeliness, resource waste, and low security of traditional polling update methods. The proposed method is a WebSocket-based approach for real-time high-concurrency data updates in BI visualization dashboards. This method effectively reduces system coupling, ensures system security and stability, improves data update efficiency, and enhances user experience, providing an efficient and reliable solution for real-time data updates in high-concurrency scenarios for BI visualization dashboards.

[0011] To achieve the above objectives, the present invention adopts the following technical solution:

[0012] A method for implementing high-concurrency real-time data updates for BI visualization dashboards based on WebSocket includes the following specific steps:

[0013] S1: Drag and drop components to build a BI dashboard: Create a new BI dashboard scene, and add components to the scene by dragging and dropping in the scene editor;

[0014] S11: Component location calculation:

[0015] When a user drags a component in the scene editor, the component's position on the main screen needs to be calculated. Assuming the origin of the main screen's coordinates is at the top left corner, with the x-axis horizontal and the y-axis vertical, the component's position is determined by the coordinates (x, y) of its top left corner. The calculation formula is as follows:

[0016] x = x-coordinate of the starting point of the drag + lateral displacement during the drag process

[0017] y = the ordinate of the starting point of the drag + the longitudinal displacement during the dragging process

[0018] The horizontal and vertical coordinates of the drag start point are the mouse position coordinates on the large screen when the user starts dragging the component; the horizontal and vertical displacements are the distances the mouse moves relative to the start point during the dragging process.

[0019] S12: Component size adjustment calculation:

[0020] Users can adjust the size of components while dragging or after placing them. The size of a component is represented by its width and height, calculated using the following formula:

[0021] Width = Original width × Scaling ratio

[0022] Height = Original Height × Scaling Ratio

[0023] The original width and original height are the initial dimensions of the component; the scaling factor is the scaling factor when the user adjusts the size of the component, with a value greater than 1 indicating enlargement and a value less than 1 indicating shrinkage.

[0024] S13: Component Index ID Generation Algorithm (UUID Algorithm):

[0025] When adding a component, a universally unique identifier is generated by calling the UUID library as the component's index ID. The UUID generation algorithm is as follows:

[0026] A UUID is a 128-bit value, usually represented in binary. The generation process involves multiple steps, including obtaining the current timestamp and a random number. Specifically, UUID generation can be divided into the following steps:

[0027] a. Obtain the current timestamp (accurate to the microsecond or nanosecond level) and use it as the seed for the time portion;

[0028] The timestamp portion (60 bits) = (current time - UUID epoch time) × 10^6 (or 10^9, depending on the time precision).

[0029] The UUID epoch time is a fixed starting point, such as October 15, 1582 in the Gregorian calendar, 00:00:00.000.

[0030] b. Generate a random number or use a node identifier (such as a MAC address) as a seed for the node part;

[0031] The node part (48 bits) = node identifier (such as MAC address) or random number.

[0032] c. Generate a 14-bit portion of the clock sequence using a random number.

[0033] The clock sequence portion is a random number.

[0034] d. Combine the above parts according to the UUID format, and add a version number (4 digits, representing the UUID version, such as 1 indicating time-based version) and a variant (2 digits, representing a variant of the UUID).

[0035] For example, a version 1 UUID format is as follows:

[0036] High-order bits of the timestamp (4-digit version number) - Middle-order bits of the timestamp - Low-order bits of the timestamp - Clock sequence - Node;

[0037] The final generated UUID serves as the component's index ID, used to accurately match messages applicable to this component.

[0038] The above steps ensure that the components are correctly placed, resized, and uniquely identified on the BI dashboard, laying the foundation for subsequent WebSocket communication and data updates.

[0039] S2: Configure and connect to the scenario's WebSocket service: Configure the URL and heartbeat interval of the WebSocket connection pool, dynamically adjust the allocation and use of WebSocket connections; monitor the WebSocket connection status; and maintain the connection by sending heartbeat messages.

[0040] S3: Set the component data source type to WebSocket and configure the interface name agreed upon with the server to achieve bidirectional communication between the component and the server through the middle-layer data management module.

[0041] This mechanism ensures that the component rendering process is not interrupted by new data updates, thus guaranteeing the integrity of data updates and the stability of component display. Simultaneously, through a polling mechanism, the system can promptly resume data updates after component rendering is complete, ensuring the real-time nature and accuracy of the data.

[0042] As a further technical solution of the present invention, in S1, the components include various types such as charts, pictures, videos, and text.

[0043] As a further technical solution of the present invention, the added components in S1 specifically include:

[0044] When adding a component, a universally unique identifier is generated by calling the UUID library as the component's index ID. The generated UUID is a value represented by a 128-bit binary number. The main function of the UUID library is to generate a unique identifier with a theoretically extremely low probability of repetition. This index ID will be used to accurately match messages applicable to this component.

[0045] As a further technical solution of the present invention, S2 specifically includes:

[0046] S21: Configure the URL and heartbeat time of the WebSocket connection pool, and combine dynamic load balancing strategy and dynamic connection management to dynamically adjust the allocation and use of WebSocket connections according to the client's access frequency, data requirements, etc.

[0047] S22: Monitor the WebSocket connection status;

[0048] S23: To prevent the connection from being closed due to no messages being sent for a period of time, heartbeat messages are sent to maintain the connection.

[0049] As a further technical solution of the present invention, S21 specifically includes:

[0050] S211: Initialize the client, receiving a list containing multiple server node URLs and the maximum number of connections (default is 5). Multiple server node URLs are used for load balancing to ensure that connection requests are evenly distributed to different server nodes.

[0051] The maximum number of connections can be dynamically adjusted according to the actual situation, using the following formula:

[0052] Maximum number of connections = number of concurrent connections that the client can handle ÷ average request processing capacity of each server node;

[0053] The number of concurrent connections a client can handle depends on the client's hardware resources and network bandwidth; the average request processing capacity of each server node is determined by the server's performance and network conditions.

[0054] S212: Create a connection pool to store all established WebSocket connections, instantiate a load balancer to dynamically select the next available server node, and create WebSocket connections in a loop based on the maximum number of connections. In each loop, obtain the URL of the next available server node through the load balancer, create a new WebSocket instance using the obtained URL, and add it to the connection pool.

[0055] S213: Dynamically adjust load balancing strategy:

[0056] To ensure load balancing of WebSocket connections, it is necessary to dynamically adjust the allocation and use of connections. A simple round-robin algorithm can be used to select the next available server node.

[0057] As a further technical solution of the present invention, S22 specifically includes:

[0058] S221: When a connection is successfully established, the WebSocket's onopen event is triggered, indicating that communication with the WebSocket server has been successfully established, and is used to send and receive messages from the WebSocket server; similarly, the WebSocket's onclose event is triggered when the connection is closed, and the WebSocket's onerror event is triggered when a connection error occurs;

[0059] S222: To prevent the WebSocket service from suddenly disconnecting during the scenario's duration, the socket.onclose method is used to listen for its close event, and a reconnection is initiated when the service is closed.

[0060] As a further technical solution of the present invention, S3 specifically includes:

[0061] S31: When loading the scene, extract the component configurations of all WebSocket data types, integrate the index ID and interface name configurations, serialize them into a JSON string, and send it to the server through the configured WebSocket connection. In this way, the server can know the interface name and data source information corresponding to each component. In addition, it is agreed with the server that the message returned to the client needs to carry the original index ID and interface name so that the client can accurately update the data to the corresponding component.

[0062] S32: The middle-layer data management module includes message queue management functionality to ensure that data can be updated normally in WebSocket high-concurrency scenarios;

[0063] S33: The intermediate layer data management module includes data processor functionality;

[0064] S34: Get the current state of the component, which is either "Rendering" or "Rendering Complete". If the component is currently in the "Rendering" state, start a timer to check the component state every 100 milliseconds until the state changes to "Rendering Complete". During this period, the system will not process new data updates to avoid conflicts. If the component is currently in the "Rendering Complete" state, the system will update the component state to "Rendering", retrieve the first piece of data from the queue, call the updateComponentData method, use the retrieved data to update the component's display content, and after the update is complete, the system will reset the component state to "Rendering Complete" and continue processing the next piece of data in the queue.

[0065] As a further technical solution of the present invention, in S31, the JSON string is a compact message format, which can reduce the amount of data transmitted, and all component configurations are sent together, which can reduce protocol overhead.

[0066] As a further technical solution of the present invention, S32 specifically includes:

[0067] Upon receiving a message, it is stored in a queue. The component index ID is extracted from the received message, and a queue is created for each component to receive it. For example, a dictionary can be used to store the message queues of each component, with the component index ID as the key and the queue as the value. The message is compared with the previous data in the queue. If the data is the same, the message is discarded to avoid invalid rendering. Suppose the previous data in the queue is data_prev and the currently received data is data_current. The decision to discard data_current is made by comparing whether the two data are the same.

[0068] As a further technical solution of the present invention, S33 specifically includes:

[0069] When the received data is determined to be inconsistent, the system first enters the data processor. The data processor has an open code editor for writing data processing scripts. For example, if an interface returns data with the structure {"temperature":25,"humidity":60}, and component A needs to directly display the temperature and humidity values, while component B needs to display the temperature status (e.g., "normal temperature" or "too high"), then the corresponding scripts can be written in the data processor to process the data, converting the raw data into the data format required by each component. If multiple components share the same interface, but each component needs to perform data differentiation processing, the data processor can process various structured and unstructured data into data usable by the components. For example, for the shared interface mentioned above, the data processor can generate different data formats according to the needs of different components to meet the personalized display requirements of each component. Then, the component data is updated.

[0070] The beneficial effects of this invention are as follows:

[0071] 1. Significantly reduces system coupling and improves system scalability and maintainability: Communication between the client and server is achieved through a WebSocket connection pool, avoiding the complexity of binding a dedicated WebSocket connection to each component; this design greatly simplifies the system structure, reduces the coupling between components and data sources, and enables the system to respond more flexibly to changes in dynamic business requirements, while also facilitating subsequent expansion and maintenance.

[0072] 2. Significantly improves data update efficiency and ensures data real-time performance and accuracy: The server-side proactive data push mechanism replaces the traditional timed polling mechanism, effectively reducing redundant client requests; the system only communicates when data changes, avoiding a large amount of invalid data interaction and significantly reducing the waste of server resources and bandwidth; at the same time, real-time two-way communication ensures that data can be updated in a timely manner, improves the ability to respond to emergencies, and provides users with more accurate and real-time data support.

[0073] 3. Effectively improve the system's concurrent processing capabilities and ensure the efficiency and timeliness of data updates: By introducing dynamic connection management and load balancing technologies, the allocation and use of WebSocket connections are dynamically adjusted according to the client's access frequency and data requirements, ensuring the rational utilization of connection resources. At the same time, by evenly distributing data requests to different servers or service nodes, the system performance bottleneck caused by high-concurrency access is effectively avoided, ensuring the efficiency and timeliness of data updates in large-scale concurrent access scenarios.

[0074] 4. Optimize data processing flow and improve overall system performance: The middle-layer data management module enables unified management and scheduling of data sent from the server; it generates component data queues and discards redundant data before rendering, avoiding invalid rendering and further improving system performance; in addition, the data processor function allows for differentiated processing of data from common interfaces of multiple components, ensuring that each component can receive data that meets its own needs, thus improving the system's flexibility and adaptability.

[0075] 5. Enhanced system security: Compared to the frequent requests under the traditional polling mechanism, this invention reduces unnecessary network communication and lowers the risk of malicious exploitation; at the same time, by reasonably managing WebSocket connections and data transmission, it further enhances system security and effectively prevents security threats such as DDoS attacks or data leaks.

[0076] 6. By optimizing data update efficiency and ensuring data real-time performance, the BI visualization dashboard can quickly and accurately display the latest data information, providing users with a smoother and more intuitive user experience. This helps enterprises quickly obtain key data, achieve rapid and accurate decision-making, and thus improve the management efficiency and competitiveness of enterprises. Attached Figure Description

[0077] Figure 1 A schematic diagram of the WebSocket connection pool configuration panel;

[0078] Figure 2 A schematic diagram of the panel for configuring component data sources and data processors;

[0079] Figure 3 Flowchart for server-side data update and push;

[0080] Figure 4 This is a flowchart of a method for real-time updating of high-concurrency data on a BI visualization dashboard based on WebSocket, as proposed in this invention. Detailed Implementation

[0081] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0082] Please see the appendix Figure 1 - Appendix Figure 4 A method for implementing high-concurrency real-time data updates for BI visualization dashboards based on WebSocket;

[0083] WebSocket: A network communication protocol that provides a way to communicate in full-duplex over a single TCP connection, allowing real-time, bidirectional data transmission between clients and servers;

[0084] High concurrency: The ability of a system to process a large number of requests simultaneously in a short period of time. It is usually used to describe the performance of a server, application, or network under high load.

[0085] The specific steps include the following:

[0086] S1: Drag and drop components to build a BI dashboard: Create a new BI dashboard scene. In the scene editor, add components (including charts, images, videos, text, and other types) to the scene by dragging and dropping. When adding a component, a universally unique identifier is generated as the component's index ID by calling the UUID library. The generated UUID is a value represented by a 128-bit binary number. The main function of the UUID library is to generate a unique identifier with a theoretically very low probability of repetition. This index ID will be used to accurately match messages applicable to this component.

[0087] The BI dashboard setup is achieved using the Fulicode RBI Business Intelligence Designer.

[0088] Fulicode RBI Business Intelligence Designer is a domestically developed online BI tool, mainly featuring the following characteristics and functions:

[0089] Core Functions

[0090] Low-code development: Through low-code drag-and-drop operations, users can quickly build industrial internet application scenarios such as management dashboards, digital dashboards, and smart cities, enabling rapid and accurate decision-making and improving enterprise management efficiency.

[0091] A rich component library: It has more than 100 built-in components, covering various chart types such as bar charts, line charts, radar charts, and pie charts, as well as text, media, maps, and controls, to meet users' various data visualization needs;

[0092] Drag-and-drop operation: Supports drag-and-drop report creation. Users can easily add components to the canvas and configure component properties, such as component size, color, labels, legends, and tooltips, through visual selection and input.

[0093] Animation configuration: Supports basic animation effects such as entrance and exit, and also provides timeline-based animation configuration functions, which can realize transformations such as translation, scaling, rotation, transparency, width, and height to improve the display effect;

[0094] Layer layout: Users can move components, change and lock layers, and hide components by dragging and dropping, and it also supports one-click alignment and one-click grouping.

[0095] iframe embedding: Built-in web page components support embedding in other iframes, and also support embedding RBI in other iframes, and configuring message functions to realize data transmission and interactive operations with other web pages.

[0096] Product Functions

[0097] Large screen design: Provides a wealth of large screen design templates and materials, allowing users to quickly create large screen display effects that meet their needs;

[0098] Data editing: Supports editing and processing of data to meet users' personalized data needs;

[0099] Project publishing: This feature allows users to publish and share their designed projects, making them suitable for use in different scenarios.

[0100] S11: Component location calculation:

[0101] When a user drags a component in the scene editor, the component's position on the main screen needs to be calculated. Assuming the origin of the main screen's coordinates is at the top left corner, with the x-axis horizontal and the y-axis vertical, the component's position is determined by the coordinates (x, y) of its top left corner. The calculation formula is as follows:

[0102] x = x-coordinate of the starting point of the drag + lateral displacement during the drag process

[0103] y = the ordinate of the starting point of the drag + the longitudinal displacement during the dragging process

[0104] The horizontal and vertical coordinates of the drag start point are the mouse position coordinates on the large screen when the user starts dragging the component; the horizontal and vertical displacements are the distances the mouse moves relative to the start point during the dragging process.

[0105] S12: Component size adjustment calculation:

[0106] Users can adjust the size of components while dragging or after placing them. The size of a component is represented by its width and height, calculated using the following formula:

[0107] Width = Original width × Scaling ratio

[0108] Height = Original Height × Scaling Ratio

[0109] The original width and original height are the initial dimensions of the component; the scaling factor is the scaling factor when the user adjusts the size of the component, with a value greater than 1 indicating enlargement and a value less than 1 indicating shrinkage.

[0110] S13: Component Index ID Generation Algorithm (UUID Algorithm):

[0111] When adding a component, a universally unique identifier is generated by calling the UUID library as the component's index ID. The UUID generation algorithm is as follows:

[0112] A UUID is a 128-bit value, usually represented in binary. The generation process involves multiple steps, including obtaining the current timestamp and a random number. Specifically, UUID generation can be divided into the following steps:

[0113] a. Obtain the current timestamp (accurate to the microsecond or nanosecond level) and use it as the seed for the time portion;

[0114] The timestamp portion (60 bits) = (current time - UUID epoch time) × 10^6 (or 10^9, depending on the time precision).

[0115] The UUID epoch time is a fixed starting point, such as October 15, 1582 in the Gregorian calendar, 00:00:00.000.

[0116] b. Generate a random number or use a node identifier (such as a MAC address) as a seed for the node part;

[0117] The node part (48 bits) = node identifier (such as MAC address) or random number.

[0118] c. Generate a 14-bit portion of the clock sequence using a random number.

[0119] The clock sequence portion is a random number.

[0120] d. Combine the above parts according to the UUID format, and add a version number (4 digits, representing the UUID version, such as 1 indicating time-based version) and a variant (2 digits, representing a variant of the UUID).

[0121] For example, a version 1 UUID format is as follows:

[0122] High-order bits of the timestamp (4-digit version number) - Middle-order bits of the timestamp - Low-order bits of the timestamp - Clock sequence - Node;

[0123] The final generated UUID serves as the component's index ID, used to accurately match messages applicable to this component.

[0124] The above steps ensure that the components are correctly placed, resized, and uniquely identified on the BI dashboard, laying the foundation for subsequent WebSocket communication and data updates.

[0125] S2: Configure and connect to the scenario's WebSocket service: Configure the URL and heartbeat interval of the WebSocket connection pool, dynamically adjust the allocation and use of WebSocket connections; monitor the WebSocket connection status; and maintain the connection by sending heartbeat messages.

[0126] S21: Configure the URL and heartbeat time of the WebSocket connection pool, and combine dynamic load balancing strategy and dynamic connection management to dynamically adjust the allocation and use of WebSocket connections according to the client's access frequency, data requirements, etc.

[0127] S211: Initialize the client, receiving a list containing multiple server node URLs and the maximum number of connections (default is 5). Multiple server node URLs are used for load balancing to ensure that connection requests are evenly distributed to different server nodes.

[0128] The maximum number of connections can be dynamically adjusted according to the actual situation, using the following formula:

[0129] Maximum number of connections = number of concurrent connections that the client can handle ÷ average request processing capacity of each server node;

[0130] The number of concurrent connections a client can handle depends on the client's hardware resources and network bandwidth; the average request processing capacity of each server node is determined by the server's performance and network conditions.

[0131] S212: Create a connection pool to store all established WebSocket connections, instantiate a load balancer to dynamically select the next available server node, and create WebSocket connections in a loop based on the maximum number of connections. In each loop, obtain the URL of the next available server node through the load balancer, create a new WebSocket instance using the obtained URL, and add it to the connection pool. The specific algorithm is as follows:

[0132] class WebSocketPool:

[0133] def__init__(self, server_urls, max_connections=5):

[0134] self.server_urls=server_urls

[0135] self.max_connections=max_connections

[0136] self.connection_pool=[]

[0137] self.load_balancer=LoadBalancer(server_urls)

[0138] def create_connections(self):

[0139] for_in range(self.max_connections):

[0140] server_url=self.load_balancer.get_next_server()

[0141] ws=WebSocket(server_url)

[0142] self.connection_pool.append(ws)

[0143] ws.connect();

[0144] Where: server_urls is a list of server node URLs; max_connections is the maximum number of connections, which defaults to 5.

[0145] S213: Dynamically adjust load balancing strategy:

[0146] To ensure load balancing of WebSocket connections, it is necessary to dynamically adjust the allocation and use of connections. A simple round-robin algorithm can be used to select the next available server node.

[0147] Algorithm description:

[0148] The load balancer maintains a list of server nodes and the current connection count;

[0149] Use a polling algorithm to select the next available server node;

[0150] The specific algorithm is as follows:

[0151] class LoadBalancer:

[0152] def __init__(self, server_urls):

[0153] self.server_urls=server_urls

[0154] self.current_index=0

[0155] def get_next_server(self):

[0156] server=self.server_urls[self.current_index]

[0157] self.current_index=(self.current_index+1)% len(self.server_urls)

[0158] return server.

[0159] S22: Monitor the WebSocket connection status;

[0160] S221: When a connection is successfully established, the WebSocket's onopen event is triggered, indicating that communication with the WebSocket server has been successfully established, and is used to send and receive messages from the WebSocket server; similarly, the WebSocket's onclose event is triggered when the connection is closed, and the WebSocket's onerror event is triggered when a connection error occurs;

[0161] S222: To prevent the WebSocket service from suddenly disconnecting during the scenario's duration, the `socket.onclose` method is used to listen for its close event. When the service closes, a reconnection is initiated; the specific algorithm is as follows:

[0162] class WebSocket:

[0163] def __init__(self, url):

[0164] self.url=url

[0165] self.connection=None

[0166] def connect(self):

[0167] # Establish a WebSocket connection

[0168] self.connection=self._create_connection(self.url)

[0169] self._add_event_listeners()

[0170] def _add_event_listeners(self):

[0171] self.connection.onopen=self._on_open

[0172] self.connection.onclose=self._on_close

[0173] self.connection.onerror=self._on_error

[0174] def _on_open(self):

[0175] print("WebSocket connection opened")

[0176] def _on_close(self):

[0177] print("WebSocket connection closed")

[0178] self.reconnect()

[0179] def _on_error(self, error):

[0180] print(f"WebSocket error: {error}")

[0181] self.reconnect()

[0182] def reconnect(self):

[0183] # Reconnect to the server

[0184] self.connect().

[0185] S23: To prevent the connection from being closed due to no messages being sent for a period of time, heartbeat messages are sent to maintain the connection;

[0186] Algorithm description:

[0187] Set the heartbeat interval (e.g., 30 seconds);

[0188] Use a timer to send heartbeat messages periodically;

[0189] The specific algorithm is as follows:

[0190] class WebSocket:

[0191] def __init__(self, url):

[0192] self.url=url

[0193] self.connection=None

[0194] self.heartbeat_interval=30 # Heartbeat interval (seconds)

[0195] def connect(self):

[0196] # Establish a WebSocket connection

[0197] self.connection=self._create_connection(self.url)

[0198] self._add_event_listeners()

[0199] self.start_heartbeat()

[0200] defstart_heartbeat(self):

[0201] self.heartbeat_timer=threading.Timer(self.heartbeat_interval,self.send_heartbeat)

[0202] self.heartbeat_timer.start()

[0203] def send_heartbeat(self):

[0204] self.connection.send("heartbeat")

[0205] self.start_heartbeat();

[0206] Where: heartbeat_interval is the heartbeat interval, which defaults to 30 seconds.

[0207] S3: Set the component data source type to WebSocket and configure the interface name agreed upon with the server to achieve bidirectional communication between the component and the server through the middle-layer data management module;

[0208] S31: When loading a scenario in the Fulicode RBI Business Intelligence Designer, extract all component configurations of WebSocket data types, integrate the index ID and interface name configurations, and serialize them into a JSON string. The JSON string is a compact message format, which can reduce the amount of data transmitted. All component configurations are sent together to reduce protocol overhead. They are sent to the server through the configured WebSocket connection, so that the server can know the interface name and data source information corresponding to each component. In addition, it is agreed with the server that the message returned to the client needs to carry the original index ID and interface name so that the client can accurately update the data to the corresponding component.

[0209] S32: The middle-layer data management module includes message queue management functionality to ensure normal data updates in high-concurrency WebSocket scenarios. Upon receiving a message, it stores it in a queue, extracts the component index ID from the received message, and creates a queue for each component. For example, a dictionary can be used to store the message queues of each component, with the component index ID as the key and the queue as the value. The message is compared to the previous message in the queue; if the data is the same, the message is discarded to avoid invalid rendering. Assuming the previous message in the queue is `data_prev` and the currently received message is `data_current`, the decision to discard `data_current` is made by comparing whether these two messages are the same.

[0210] S33: The middle-layer data management module includes a data processor function. When it is determined that the received data is different, it first enters the data processor. The data processor has an open code editor for writing data processing scripts. For example, if an interface returns data with the structure {"temperature":25,"humidity":60}, component A needs to directly display the temperature and humidity values, while component B needs to display the temperature status (such as "normal temperature" or "too high temperature"). The corresponding scripts can be written in the data processor to process the data, converting the raw data into the data format required by each component. If multiple components share the same interface, but each component needs to perform data differentiation processing, the data processor can process various structured and unstructured data into data usable by the components. For example, for the shared interface mentioned above, the data processor can generate different data formats according to the needs of different components to meet the personalized display requirements of each component. Then, the component data is updated.

[0211] S34: Get the current state of the component, which is either "Rendering" or "Rendering Complete". If the component is currently in the "Rendering" state, start a timer to check the component state every 100 milliseconds until the state changes to "Rendering Complete". During this period, the system will not process new data updates to avoid conflicts. If the component is currently in the "Rendering Complete" state, the system will update the component state to "Rendering", retrieve the first piece of data from the queue, call the updateComponentData method, use the retrieved data to update the component's display content, and after the update is complete, the system will reset the component state to "Rendering Complete" and continue processing the next piece of data in the queue.

[0212] This mechanism ensures that the component rendering process is not interrupted by new data updates, thus guaranteeing the integrity of data updates and the stability of component display. Simultaneously, through a polling mechanism, the system can promptly resume data updates after component rendering is complete, ensuring the real-time nature and accuracy of the data.

[0213] Example 1

[0214] Real-time visualization updates of smart park monitoring data

[0215] In this embodiment, a smart park monitoring center screen is constructed based on the WebSocket-based BI visualization screen high-concurrency real-time data update method of the present invention. This embodiment will be deployed in a park with multiple sensors, cameras, and smart devices for real-time monitoring of environmental data, equipment status, and security alarm information. The specific steps are as follows:

[0216] (1) BI dashboard setup and multi-source data integration

[0217] In the Fulicode RBI Business Intelligence Designer, create a smart park monitoring dashboard by adding the following components via drag and drop:

[0218] The system includes an environmental monitoring chart (line graphs displaying data such as temperature, humidity, and PM2.5), an equipment status monitoring panel (real-time display of equipment power-on / off status, operating parameters, etc.), and a security alarm panel (displaying intrusion alarms, fire alarms, etc.). Each component generates a unique index ID using a UUID library upon addition, used for precise matching with real-time data pushes from the backend. For example, the index ID for the environmental monitoring chart is env_chart_uuid, the index ID for the equipment status monitoring panel is device_panel_uuid, and the index ID for the security alarm panel is alarm_panel_uuid.

[0219] (2) WebSocket service access and connection pool configuration

[0220] Since smart parks involve multiple types of sensors and devices, and data sources are distributed across different backend servers, when the client loads a large-screen scene, it is configured with a WebSocket connection pool containing URLs of multiple server nodes (such as environmental monitoring servers, device status servers, security alarm servers, etc.), and combined with the maximum number of connections and heartbeat detection configuration, to ensure the stability of the connection.

[0221] The load balancing module dynamically allocates connections based on data update frequency and priority, ensuring that the load pushed by each data source is reasonably distributed and avoiding single-point bottlenecks on the server.

[0222] (3) Two-way communication and component data configuration

[0223] In the scenario configuration, users set the data interface name for each component (e.g., the interface name for the environmental monitoring chart is env_data, the interface name for the device status monitoring panel is device_status, and the interface name for the security alarm panel is alarm_info), and specify the data source server node.

[0224] After the scene is loaded, the client serializes all component configurations (including index IDs, interface names, data source nodes, etc.) into JSON messages and sends them to the corresponding WebSocket servers to establish bidirectional communication. The JSON message is as follows:

[0225] { "components": [

[0226] {"id":"env_chart_uuid","interface":"env_data"},

[0227] {"id":"device_panel_uuid","interface":"device_status"},

[0228] {"id":"alarm_panel_uuid","interface":"alarm_info"}]}.

[0229] When the backend detects changes in temperature, humidity, or device status, or when a safety alarm is triggered, it immediately pushes data to the client to ensure real-time updates. The pushed data message carries the original component index ID and interface name so that the client can accurately update the data to the corresponding component.

[0230] (4) Intelligent processing of the intermediate layer data management module

[0231] Because smart parks involve multiple types of data with varying update frequencies, the client-side middleware data management module intelligently processes the received messages.

[0232] Queue Management: Create independent queues for each component to avoid data corruption or loss during high-concurrency pushes; store the same data only once to save storage and processing resources, for example:

[0233] queues = {

[0234] "env_chart_uuid": queue.Queue(),

[0235] "device_panel_uuid": queue.Queue(),

[0236] "alarm_panel_uuid": queue.Queue()};

[0237] Upon receiving a message, extract the component index ID and compare it with the previous data in the queue. If the data matches, discard the message. For example, for the message queue of environmental monitoring charts:

[0238] if not queues["env_chart_uuid"].empty():

[0239] prev_data = queues["env_chart_uuid"].queue[-1]

[0240] current_data = received_data

[0241] if prev_data == current_data:

[0242] # Discard the current message

[0243] pass

[0244] else:

[0245] queues["env_chart_uuid"].put(current_data)

[0246] else:

[0247] queues["env_chart_uuid"].put(current_data).

[0248] Data Processor: When multiple components share the same data interface, the data processor module performs differentiated processing to meet different display and business needs. For example, temperature data can be used for both environmental monitoring charts and alarm threshold determination. The specific algorithm is as follows:

[0249] def process_env_data(data):

[0250] # Processing data from environmental monitoring charts

[0251] env_chart_data = data

[0252] # Data for processing alarm threshold determination

[0253] alarm_data = {}

[0254] if data["temperature"] > 30:

[0255] alarm_data["alarm_type"] = "Temperature alarm"

[0256] alarm_data["alarm_info"] = "Temperature too high"

[0257] # Return the processed data

[0258] return{"env_chart":env_chart_data,"alarm":alarm_data}

[0259] The data processor supports custom script configurations, enabling flexible data adaptation and formatting.

[0260] Real-time status detection: During component rendering, the "rendering in progress / rendering complete" status management mechanism is used to avoid data updates interfering with the components being rendered, thus achieving a seamless update experience.

[0261] (5) Dynamic connection management and scenario expansion

[0262] During the operation of a smart park, there may be sudden increases in data update frequency (such as a sudden security alarm). The middle-layer data management module monitors the load of each connection in real time through a dynamic connection management mechanism. For example, a dictionary can be defined to store the load information of each connection.

[0263] connection_load = {

[0264] "ws: / / env_server.com": 0,

[0265] "ws: / / device_server.com": 0,

[0266] "ws: / / alarm_server.com": 0}

[0267] It can dynamically expand the number of connections in the connection pool or reallocate resources as needed to achieve elastic scaling. For example, when the load on a certain connection is too high, new connections can be added to share the load.

[0268] def adjust_connections():

[0269] for server_url in server_urls:

[0270] load = connection_load[server_url]

[0271] if load > threshold: # threshold is the load threshold

[0272] # Add new connection

[0273] new_ws = WebSocket(server_url)

[0274] new_ws.connect()

[0275] connection_pool.append(new_ws)

[0276] # Update load status

[0277] connection_load[server_url]=(connection_load[server_url] + 1) / 2.

[0278] By combining load balancing strategies, all data requests are evenly distributed across different server nodes, effectively preventing a single service node from being overwhelmed by high concurrency requests and improving the overall reliability and response speed of the system.

[0279] As can be seen from the above description, the embodiments of the present invention achieve the following technical effects:

[0280] Establishing long-term connections and real-time bidirectional communication: By providing scenario-based WebSocket services and configuring them through a visual panel, long-term stable connections are established between the client and the server, enabling real-time bidirectional communication. At the same time, regardless of the number of components or data types involved, the client and the server communicate through only one WebSocket connection pool, avoiding the complexity and resource waste caused by multiple connections, reducing system coupling, and improving the efficiency and stability of data transmission.

[0281] When the server updates data, it can proactively push the data to the client, eliminating the need for the client to frequently initiate requests, thereby effectively reducing the number of communications and lowering the system load.

[0282] A middle-layer data management module is set up to uniformly manage and schedule the data sent by the server, generate component data queues, discard redundant data before rendering to avoid invalid rendering, and provide data processors to ensure differentiated processing when multiple components share a common interface.

[0283] Dynamic connection management and load balancing technology: A dynamic connection management mechanism is introduced to dynamically adjust the allocation and use of WebSocket connections based on client access frequency and data requirements, ensuring the rational utilization of connection resources. At the same time, combined with load balancing technology, data requests are evenly distributed to different servers or service nodes, effectively improving the system's concurrent processing capabilities, avoiding system performance bottlenecks caused by high concurrency access, and ensuring the efficiency and timeliness of data updates.

[0284] It can effectively reduce system coupling, ensure system security and stability, improve data update efficiency, enhance user experience, and provide an efficient and reliable solution for real-time data updates in high-concurrency scenarios for BI visualization dashboards.

[0285] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity.

[0286] This invention is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this specification. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for implementing high-concurrency real-time data updates on a BI visualization dashboard based on WebSocket, characterized in that, The specific steps include the following: S1: Drag and drop components to build a BI dashboard: Create a new BI dashboard scene, and add components to the scene by dragging and dropping in the scene editor; S2: Configure and connect to the scenario's WebSocket service: Configure the URL and heartbeat interval of the WebSocket connection pool, and dynamically adjust the allocation and use of WebSocket connections; It also monitors the WebSocket connection status and maintains the connection by sending heartbeat messages; specifically including: S21: Configure the URL and heartbeat interval of the WebSocket connection pool, and combine dynamic load balancing strategies and dynamic connection management to dynamically adjust the allocation and use of WebSocket connections based on client access frequency and data requirements; specifically: S211: Initialize the client and receive a list containing multiple server node URLs and the maximum number of connections. The multiple server node URLs are used for load balancing to ensure that connection requests are evenly distributed to different server nodes. S212: Create a connection pool to store all established WebSocket connections, instantiate a load balancer to dynamically select the next available server node, and create WebSocket connections in a loop based on the maximum number of connections; in each loop, obtain the URL of the next available server node through the load balancer, and create a new WebSocket instance using the obtained URL. S22: Monitor the WebSocket connection status; specifically: S221: When the connection is successful, the WebSocket onopen event will be triggered, indicating that communication with the WebSocket server has been successfully established, and is used to send and receive messages from the WebSocket server; S222: To prevent the WebSocket service from suddenly disconnecting during the scenario's duration, the socket.onclose method is used to listen for its close event, and a reconnection is initiated when the service is closed; S23: To prevent the connection from being closed due to no messages being sent for a period of time, heartbeat messages are sent to maintain the connection; S3: Set the component data source type to WebSocket and configure the interface name agreed upon with the server to achieve bidirectional communication between the component and the server through the middle-layer data management module.

2. The method for implementing high-concurrency real-time data updates for BI visualization dashboards based on WebSocket, as described in claim 1, is characterized in that... In S1, the components include various types such as charts, images, videos, and text.

3. The method for implementing high-concurrency real-time data updates for BI visualization dashboards based on WebSocket, as described in claim 1, is characterized in that... In step S1, adding components specifically includes: When adding a component, a universally unique identifier is generated by calling the UUID library as the component's index ID. The generated UUID is a value represented by a 128-bit binary number. The main function of the UUID library is to generate a unique identifier with a theoretically extremely low probability of repetition. This index ID will be used to accurately match messages applicable to this component.

4. The method for implementing high-concurrency real-time data updates for BI visualization dashboards based on WebSocket, as described in claim 1, is characterized in that... S3 specifically includes: S31: When loading the scene, extract the component configurations of all WebSocket data types, integrate the index ID and interface name configurations, serialize them into a JSON string, and send it to the server through the configured WebSocket connection; in addition, it is agreed with the server that the message returned to the client needs to carry the original index ID and interface name; S32: The middle-layer data management module includes message queue management functionality to ensure that data can be updated normally in WebSocket high-concurrency scenarios; S33: The intermediate layer data management module includes data processor functionality; S34: Get the current state of the component, which is either "Rendering" or "Rendering Complete". If the component is currently in the "Rendering" state, start a timer to check the component state every 100 milliseconds until the state changes to "Rendering Complete". During this period, the system will not process new data updates to avoid conflicts. If the component is currently in the "Rendering Complete" state, the system will update the component state to "Rendering", retrieve the first piece of data from the queue, call the updateComponentData method, use the retrieved data to update the component's display content, and after the update is complete, the system will reset the component state to "Rendering Complete" and continue processing the next piece of data in the queue.

5. The method for implementing high-concurrency real-time data updates for BI visualization dashboards based on WebSocket, as described in claim 4, is characterized in that... In S31, the JSON string is a compact message format, which reduces the amount of data transmitted. All component configurations are merged and sent, reducing protocol overhead.

6. The method for implementing high-concurrency real-time data updates for BI visualization dashboards based on WebSocket, as described in claim 5, is characterized in that... Specifically, S32 includes: Upon receiving a message, it is stored in a queue. The component index ID is extracted from the received message, and a queue is created for each component to receive the message. The message is compared with the previous message in the queue. If the data is the same, the message is discarded.

7. The method for implementing high-concurrency real-time data updates for BI visualization dashboards based on WebSocket according to claim 6, characterized in that, Specifically, S33 includes: When it is determined that the received data is different, the system first enters the data processor. The data processor has an open code editor for writing data processing scripts. If multiple components share the same interface, but each component needs to perform data differentiation processing, the data processor processes various structured and unstructured data into data that can be used by the components, and then starts updating the component data.

Images