A mobile big data platform collaboration system and method based on libuv and dbus

Abstract

The application discloses a mobile big data platform coordination system and method based on libuv and DBus, and specifically comprises the following steps: the mobile big data platform coordination system comprises a data plane module, a control plane module and a DBus interaction intermediate layer, and the three modules communicate through a DBus bus; the control plane module is started and listens to a state signal; the data plane module is started and initializes a listening port and a connection management component; the data plane module periodically reports state data; the control plane module generates a control instruction based on the state data and sends the control instruction to the data plane module through the DBus interaction intermediate layer; and the data plane module adjusts a running behavior after executing the instruction and continues to report the state, so that a closed-loop control is formed.

Description

Technical Field

[0001] This invention relates to the field of mobile big data processing, and mainly to a collaborative system and method for mobile big data platforms based on libuv and DBus. Background Technology

[0002] Existing mobile big data platforms generally suffer from high decoupling between the data plane and the control plane, and low interaction efficiency. The data plane only focuses on the collection and processing of massive amounts of mobile data, lacks standardized client connection management, message receiving and parsing capabilities, and cannot uniformly connect to multiple types of clients (such as mobile terminals and base station collection nodes). The client message processing latency is high and the compatibility is poor. The control plane can only issue coarse-grained global configurations, and cannot accurately control program behavior (such as limiting the number of client connections and adjusting the priority of message processing) based on fine-grained data such as client connection status and message processing load on the data plane. It also cannot obtain the program running status (such as the number of client connections, the amount of message backlog, and the processing failure rate) in real time and completely, resulting in poor command adaptability and lagging management.

[0003] In traditional technologies, the data plane and control plane often interact using hard-coded or general network protocols. The mobile data processing solution based on libuv does not combine with the inter-process communication advantages of DBus. It cannot support the standardized processing of client connections / messages by the data plane, nor can it enable the control plane to accurately perceive the client interaction status of the data plane. The overall platform has significant shortcomings in client adaptability, program control accuracy, and runtime status perception capabilities, and cannot meet the business needs of high concurrency, low latency, and dynamic management of mobile big data. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a mobile big data platform collaborative system based on libuv and DBus. This system includes a data plane module, a control plane module, and a DBus interaction middleware layer, all three communicating via the DBus bus. Specifically, it includes: The data plane module is built on the libuv event-driven library and is used to complete the collection, processing and client connection management of mobile big data. The control plane module is used for global policy control and state analysis, specifically including: deploying the DBus server component, receiving state signals sent by the data plane module, and parsing the messages; extracting libuv runtime state fields and client state fields from the state signals, and calculating the message processing failure rate; generating a maximum connection limit instruction when the current number of connections exceeds a preset threshold, and sending it to the data plane module via a DBus method call; generating a failure message retransmission instruction and sending it when the message processing failure rate exceeds a preset threshold; and constructing runtime state analysis logic to continuously monitor the data plane's operation and output analysis results. The DBus interaction middleware layer is used to define a unified interface specification, specifically including: defining the interface from the control plane to the data plane; defining the status interface from the data plane to the control plane; encapsulating all instruction messages and status messages using a unified data structure and transmitting them through the DBus bus.

[0005] Furthermore, the data plane module specifically includes: A client connection management component is constructed, which listens to the network port and accepts client connections under the event loop mechanism. The client connections support both TCP and UDP protocols. When a client connection request is received, the current number of connections is obtained and compared with the preset maximum number of client connections. When the limit is reached, the connection is rejected and the corresponding file descriptor is closed. Create a connection information structure for each access client. The structure includes the client IP address, connection file descriptor, connection status, received message count, successful processing count, and failed processing count. Add the structure to the global connection list. Based on the event callback mechanism, continuously listen for client message reading events, and after reading the message, call the libuv processing logic to process the data, and count the success and failure of processing respectively. Deploy the DBus client component to receive management instructions issued by the control plane. The management instructions include instructions to adjust the maximum number of client connections and instructions to kick out clients. After parsing, update local operating parameters or disconnect the specified client connection. Real-time collection of operational status data and client interaction status data, including data throughput, processing latency and resource utilization, and client interaction status data including the total number of current connections, the cumulative number of received messages and the cumulative number of processing failures. Status messages are generated at fixed time intervals and sent to the control plane module via the DBus signal.

[0006] Furthermore, the client connection management component employs a file descriptor-based multiplexing mechanism in the event loop to bind client connection objects to event callback functions one by one. It also completes connection state initialization, message counter initialization, and read event registration when the connection is established, enabling each client to achieve non-blocking concurrent processing in a single event loop.

[0007] Furthermore, when the data plane module implements the maximum number of client connections limit, it adopts a dual threshold control mechanism, including a connection access threshold and a connection recycling threshold. When the current number of connections reaches the connection access threshold, it refuses to establish new connections. When the current number of connections is higher than the connection recycling threshold, it actively disconnects high-failure-rate client connections based on the client's historical failure rate.

[0008] Furthermore, the client message processing process includes three stages: message parsing, classification and scheduling, and processing execution. In the message parsing stage, the message type field is extracted. In the classification and scheduling stage, messages are allocated to the corresponding processing queues according to the message priority parameters. In the processing execution stage, the libuv thread pool schedules and executes messages in priority order, and maintains a message backlog count for each queue.

[0009] Furthermore, when performing state analysis, the control plane module constructs a time window statistical model to calculate the rate of change of the number of connections and the rate of change of the message failure rate within multiple consecutive state reporting cycles. When the rate of change exceeds a preset growth threshold, an early warning level control instruction is triggered, and when the absolute value exceeds the threshold, a mandatory level control instruction is triggered.

[0010] Furthermore, the DBus interaction middleware layer encodes control commands and status data using a unified serialization structure. The message body includes a command type field, a parameter field, and a verification field. After receiving the command, the data plane module performs integrity verification and type matching verification. Only after the verification is passed can the execution process begin.

[0011] Furthermore, when reporting status, the data plane module not only sends aggregated statistical data, but also synchronously sends detailed data at the client level. The detailed data includes the client IP, connection status, last interaction time, and corresponding message processing statistics. On the control plane module side, an index structure based on the client dimension is constructed to quickly locate abnormal clients.

[0012] Furthermore, when the control plane module issues a client kick-out command, it locates the target client based on the client-dimensional index structure and carries the execution reason identifier and policy identifier in the same command. After the data plane module performs the disconnection operation, it sends back the execution result and the reason identifier together, thereby forming a closed-loop association mechanism between control commands and execution feedback.

[0013] According to a second aspect of the present invention, a collaborative method for a mobile big data platform based on libuv and DBus is proposed, comprising: a control plane module starting up and listening to status signals; a data plane module starting up and initializing the listening port and connection management component; the data plane module periodically reporting status data; the control plane module generating control instructions based on the status data and sending them to the data plane module through the DBus interaction middleware layer; and the data plane module adjusting its operating behavior and continuing to report status after executing the instructions, thereby forming a closed-loop control.

[0014] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects: 1. Improved client compatibility: Standardized client connection / message processing capabilities on the data plane support access from multiple protocols and types of clients, allowing for adaptation to new client types without modifying the core logic; 2. Improved precision in program behavior control: The control plane can issue control commands for single clients / single message types, replacing the traditional global coarse configuration; 3. Enhanced state awareness completeness: The control plane can acquire the full-dimensional state of the data plane client interaction and libuv operation, with no blind spots in awareness; 4. Improved anomaly response speed: Abnormal status is pushed to the control plane in real time, automatically triggering linkage control commands, reducing anomaly handling delay to the second level. Attached Figure Description

[0015] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many anticipated advantages of the embodiments will be readily recognized as they become better understood through reference to the following detailed description. Elements in the drawings are not necessarily to scale. The same reference numerals refer to corresponding similar parts.

[0016] Figure 1 A flowchart of a mobile big data platform collaborative system based on libuv and DBus according to an embodiment of the present invention is shown.

[0017] Figure 2 A flowchart illustrating the module interaction of a mobile big data platform collaborative system based on libuv and DBus according to an embodiment of the present invention is shown.

[0018] Figure 3 A data structure flowchart of the DBus interface specification of a mobile big data platform collaborative system based on libuv and DBus according to an embodiment of the present invention is shown.

[0019] Figure 4A flowchart illustrating the abnormal linkage control process of a mobile big data platform collaborative system based on libuv and DBus according to an embodiment of the present invention is shown.

[0020] Figure 5 This is a schematic diagram of the structure of a computer system suitable for implementing the electronic devices of the present application embodiments. Detailed Implementation

[0021] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] Figure 1 A flowchart illustrating an overall process of a mobile big data platform collaborative system based on libuv and DBus according to an embodiment of the present invention is shown. The mobile big data platform collaborative system includes a data plane module, a control plane module, and a DBus interaction middleware layer, and the three communicate via the DBus bus. The module interaction flowchart is shown below. Figure 2 As shown, it specifically includes: The data plane module is built on the libuv event-driven library and is used to complete data acquisition, data processing, and client connection management for mobile big data. The data plane module runs on a server node and uses an event loop mechanism to handle high-concurrency network connections, while communicating with the control plane module via the DBus bus.

[0024] 1. Client connection listening and access; The data plane module initializes the libuv event loop object and creates a network listener component upon startup. The listener component creates a listening socket by calling network interface functions provided by libuv and binds it to a specified IP address and port. The listener component is configured to listen on all network interface card addresses and enables port multiplexing to improve connection efficiency.

[0025] After the listening socket is initialized, the client connection receive callback function is registered to the event loop. When a new client connection request arrives, the event loop triggers the connection receive callback function and passes the client connection file descriptor and client address information to the callback function.

[0026] 2. Control of the number of client connections; In the connection reception callback function, the data plane module counts the number of currently online client connections. This is done either by iterating through the global client connection list or by maintaining a connection counter to obtain the current connection count.

[0027] Furthermore, the current number of connections is compared with the preset maximum number of client connections. The maximum number of client connections is a runtime parameter, set to a default value during system initialization, and can be dynamically adjusted via the control plane module.

[0028] When the current number of connections reaches the maximum number of client connections, the data plane module directly rejects new connection requests, calls the close interface to close the corresponding client file descriptor, and logs the connection rejection. When the current number of connections has not reached the maximum number of client connections, clients are allowed to establish connections.

[0029] 3. Client connection information management; Once the client connection is successfully established, the data plane module creates a client connection information structure for the client. This structure is used to record the client connection status and message statistics, and its fields include: [client IP address, connection file descriptor, connection status, received message count, successfully processed message count, and failed message count].

[0030] After the structure is created, add it to the global client connection list. The global client connection list is used to centrally manage all currently online client connections, providing a data source for subsequent status statistics and handling of abnormal connections.

[0031] 4. Client message reading and processing; After the client connection is established, the data plane module registers readable event callback functions for the client file descriptors through the libuv event loop mechanism. When the client sends data, the event loop triggers the message read callback function.

[0032] In the message read callback function, message data is read from the client file descriptor, and the read result is checked. If the length of the read data is less than or equal to zero, it indicates that the client connection has been broken or an exception has occurred. At this time, the client connection status is marked as broken, and the corresponding file descriptor is closed.

[0033] When valid message data is read, the data plane module first increments the received message count, and then calls the libuv processing logic to execute the data processing task. The processing logic may include message parsing, data preprocessing, and data forwarding.

[0034] After message processing is complete, the statistics fields in the client connection information structure are updated based on the processing result. A success count is incremented when processing is successful; a failure count is incremented when processing fails, and an error log is logged.

[0035] 5. DBus control command reception; The data plane module also deploys the DBus client component and connects to the system DBus bus. By registering DBus message processing callback functions, the data plane module can receive management commands from the control plane module.

[0036] The control commands issued by the control plane module include: commands to adjust the maximum number of client connections and commands to kick out clients.

[0037] When a command to adjust the maximum number of client connections is received, the data plane module parses the parameter values ​​in the command and updates the local operating parameters, thereby changing the restrictions on subsequent client connections.

[0038] When a client kickout command is received, the data plane module searches for the target client connection in the global client connection list based on the client IP address carried in the command, performs a disconnection operation, and updates the corresponding client connection status.

[0039] 6. Operational status acquisition; During system operation, the data plane module periodically collects operational status data and client interaction status data.

[0040] The operational status data includes: data processing throughput, data processing latency, and system resource utilization.

[0041] Client interaction status data includes: the total number of current client connections, the cumulative number of messages received, and the cumulative number of failed messages processed.

[0042] The runtime status data and client interaction status data are obtained by statistically analyzing the global client connection list and acquiring resource information through the system monitoring interface.

[0043] 7. Status data reporting; The data plane module generates status messages at fixed time intervals. In this embodiment, the status reporting period is set to 1 second.

[0044] When generating a status message, a DBus signal message object is constructed, and the runtime status data and client interaction status data are encapsulated according to a predefined data structure. This status signal is then sent to the control plane module via the DBus bus.

[0045] After receiving the status signal, the control plane module can analyze the operating status of the data plane and issue new control commands to the data plane module when necessary.

[0046] The data plane module of this invention can realize client connection management, data processing and real-time reporting of operating status, and form a collaborative control mechanism with the control plane module.

[0047] Furthermore, the client connection management component in the data plane module of this invention implements high-concurrency network connection management based on the libuv event-driven model. This component employs a file descriptor-based multiplexing mechanism at runtime and uses an event loop to uniformly schedule network connection events and message processing tasks.

[0048] During the initialization phase, the data plane module creates a libuv event loop object and starts the event loop thread. All client connections are uniformly scheduled within this event loop.

[0049] When the listening port receives a new client connection request, the file descriptor corresponding to the client connection is obtained through the listening callback function. A client connection object is then created for that client, and a mapping relationship is established between this object and the file descriptor.

[0050] The client connection object contains the following information fields: [client IP address, connection file descriptor, connection status identifier, received message counter, successful processing counter, failed processing counter, last interaction time].

[0051] After the client connection object is created, perform the following initialization operations: Initialize the connection status to online; Initialize the message receive counter, the successful processing counter, and the failed processing counter to zero; Record the current system time as the last interaction time of the client; Add the client connection object to the global client connection management list; Register readable event callback functions for the file descriptor in the libuv event loop.

[0052] In this way, each client connection object is bound to an independent event callback function. When the client sends data, the event loop detects a file descriptor readable event and triggers the corresponding callback function to execute the message reading logic.

[0053] Since all connection events are uniformly scheduled through the event loop, the system can handle a large number of client connections concurrently in a single thread, thereby avoiding the performance overhead caused by frequent thread creation and context switching in the traditional blocking network model and achieving non-blocking concurrent connection management.

[0054] To prevent the system from exhausting resources due to an excessive number of connections under high concurrency, this implementation introduces a dual-threshold connection control mechanism in the data plane module.

[0055] The mechanism includes two parameters: connection access threshold and connection recycling threshold.

[0056] The connection access threshold is used to limit the number of new client connections. When the current number of connections in the system reaches the connection access threshold, the client connection reception callback function will reject new connection requests, immediately close the corresponding file descriptor, and log the connection rejection.

[0057] The connection recycling threshold is used to trigger the cleanup of abnormal connections. When the current number of connections in the system exceeds the connection recycling threshold, the data plane module initiates the connection recycling process.

[0058] During the connection recycling process, all client connection objects are retrieved from the global client connection management list, and the historical message processing failure rate for each client is calculated. The failure rate is calculated as: processing failure count. Message count received.

[0059] Client connections are sorted according to their failure rate, with clients having higher failure rates at the top of the list. Connections to clients with the highest failure rates are then selected and disconnected sequentially, continuing until the current number of connections falls below the connection recycling threshold.

[0060] The dual threshold control mechanism of this invention can both limit the growth of the number of connections and prioritize the cleanup of abnormal client connections, thus ensuring the stable operation of system resources.

[0061] When the client sends data, the libuv event loop triggers the corresponding message read callback function. This callback function is responsible for reading data from the client file descriptor and submitting it to the message processing flow.

[0062] In this embodiment, the client message processing flow includes three stages: (1) Message parsing phase; During the message parsing phase, the received data is first parsed to extract the message type field and related business parameters. The message type field identifies the business category of the message, such as a business data message or a status report message.

[0063] After parsing, the message is encapsulated into an internal message object, along with the parsed message type and priority information.

[0064] (2) Classification and scheduling phase; During the classification and scheduling phase, message objects are assigned to the corresponding message processing queues based on message priority parameters.

[0065] Internally, multiple message processing queues are maintained, each with a different priority level. For example, high-priority queues are used to process critical business messages, while ordinary-priority queues are used to process general business messages.

[0066] When a message enters the corresponding queue, the message backlog count of that queue is updated to record the number of messages currently waiting to be processed.

[0067] (3) Processing execution phase; During the execution phase, the libuv thread pool retrieves messages to be processed from various message processing queues. When scheduling tasks, the thread pool selects queues according to priority, with messages in higher-priority queues being executed first.

[0068] Once the message processing task is completed, update the statistics in the client connection object based on the processing result: if the processing is successful, increment the success count; if the processing fails, increment the failure count and log the error.

[0069] At the same time, update the last interaction time in the client connection object to record the time of the most recent message interaction.

[0070] During system operation, the data plane module periodically reports operational status information to the control plane module. This status data reporting is triggered by a timed task, and in this implementation, the reporting period is set to a fixed time interval.

[0071] During each status report, the data plane module generates two types of status data: The first category is aggregated statistical data, including: the total number of current client connections, the cumulative number of messages received, the cumulative number of failed messages processed, the current message processing throughput, data processing latency, and system resource utilization.

[0072] The second category is client-level detailed data. This data generates a status record for each client connection object, and its fields include: client IP address, current connection status, most recent interaction time, received message count, successful processing count, and failed processing count.

[0073] After generating the above data, it is encapsulated into a DBus signal message according to a predefined data structure and sent to the control plane module via the DBus bus.

[0074] After receiving the status data sent by the data plane module, the control plane module parses the client-level data and constructs an index structure based on the client dimension.

[0075] The index structure uses the client's IP address as the index key and stores the corresponding client status information in the index nodes. Through this index structure, the control plane module can quickly locate a specific client from a large amount of client status data and analyze its historical interaction status.

[0076] When abnormal client connections, excessively high message failure rates, or prolonged periods of inactivity are detected, the control plane module can quickly locate the abnormal client through an index and issue corresponding control instructions to the data plane module, thereby achieving precise management of the abnormal client.

[0077] The control plane module is used to implement global policy control and operational status analysis. It establishes a communication connection with the data plane module through the DBus communication mechanism, monitors the operational status of the data plane module in real time, and issues control commands to the data plane module when abnormal states are detected, thereby achieving dynamic adjustment of the system's operational status. Specifically, this includes: 1. Deployment of DBus server-side components; When the control plane module starts up, it first deploys the DBus server component and registers the corresponding service name, object path and interface name, so that the data plane module can send status signals to the control plane module through the DBus bus.

[0078] During the system initialization phase, the control plane module performs the following operations: Initialize the DBus server connection and connect it to the system DBus bus; Register the status signal listener callback function; Establish a state data parsing module to parse state signals from the data plane module; Initialize the state data cache structure to store state data for consecutive periods.

[0079] When the data plane module sends a status signal through DBus, the DBus server component receives the signal and triggers the status signal handling function.

[0080] 2. State signal analysis; Upon receiving the status signal, the control plane module parses the message content. The status signal contains two core types of information: The first category consists of libuv runtime status fields, including: [data processing throughput, data processing latency, system resource utilization].

[0081] The second category is client status fields, including: [current total number of client connections, cumulative number of messages received, and cumulative number of failed messages processed].

[0082] The control plane module obtains the current running status of the data plane module by parsing the above fields, and writes the parsed data into the status cache structure.

[0083] Furthermore, the control plane module calculates the message processing failure rate based on the client status field. The failure rate is calculated as follows: Message processing failure rate = cumulative number of failed messages Total number of messages received.

[0084] When the cumulative number of received messages is zero, the failure rate is set to zero to avoid division by zero errors.

[0085] After the calculation is completed, the control plane module records the current running status to the status analysis module and continues to execute the subsequent status judgment process.

[0086] 3. Connection number threshold detection and connection limit control; During the state analysis process, the control plane module monitors the total number of current client connections and compares it with a preset connection threshold.

[0087] The preset connection threshold is a system operating parameter that is configured during the system initialization phase.

[0088] When the number of current client connections is detected to exceed the threshold, the control plane module generates a maximum connection limit instruction. When generating the instruction, a DBus method call message is constructed, and the maximum connection parameter is encapsulated within the message.

[0089] This instruction is sent to the data plane module via the DBus method call interface. Upon receiving the instruction, the data plane module adjusts its maximum client connection limit, thereby reducing the number of new client connections.

[0090] 4. Message failure rate detection and failed message retransmission control; The control plane module also monitors the message processing failure rate. When the calculated message processing failure rate exceeds a preset failure rate threshold, it is determined that there is a message processing anomaly in the data plane module.

[0091] In the event of a message processing exception, the control plane module generates a retransmission command for the failed message and sends it to the data plane module via the DBus method. Upon receiving the command, the data plane module performs the retransmission process to reduce message loss.

[0092] 5. Operational status analysis logic; To achieve continuous monitoring of the system's operational status, the control plane module constructs operational status analysis logic. This logic takes periodic status data as input and comprehensively evaluates the operational status of the data plane module by analyzing changes in the number of connections, message processing status, and system resource usage.

[0093] The control plane module updates the status analysis results and generates a current system operating status report each time it receives a status signal. This report can be used for system operation monitoring and anomaly diagnosis.

[0094] 6. Time Window Statistical Model In this embodiment, in order to more accurately identify the system operation trend, the control plane module constructs a time window statistical model during the state analysis process.

[0095] The time window statistical model performs trend analysis on the system's operating status by storing data from multiple consecutive status reporting periods. The specific implementation method is as follows: The state cache structure stores state data for multiple consecutive cycles. Calculate the rate of change in the number of connections and the rate of change in the message failure rate; The calculation results are compared with the preset growth threshold.

[0096] The rate of change of the number of connections is calculated as follows: Connection change rate = (Current cycle connection count - Previous cycle connection count) Number of connections in the previous cycle.

[0097] The calculation method for the message failure rate change rate is as follows: Failure rate change rate = (Current period failure rate - Previous period failure rate) Failure rate in the previous cycle.

[0098] When the rate of change exceeds a preset growth threshold, the control module determines that the system has an abnormal growth trend and triggers an early warning-level control command. This early warning-level control command is used to adjust system operating parameters in advance.

[0099] When the absolute value of the number of connections or the absolute value of the failure rate exceeds a preset threshold, the control plane module triggers a mandatory control command. Mandatory control commands are used to quickly intervene in the system's operating status, such as limiting the number of connections or triggering abnormal connection handling.

[0100] Time window statistical models can identify potential risks before anomalies occur, thereby improving system stability.

[0101] 7. Client-side kick-out command and closed-loop control mechanism; When an abnormal client is detected, the control plane module can send a client kick-out command to the data plane module.

[0102] After receiving client-level status data from the data plane module, the control plane module constructs a client-level index structure locally. This index structure uses the client's IP address as the index key and stores the corresponding client status information in the index nodes.

[0103] When a client kick-out operation needs to be performed, the control plane module locates the target client through the client-dimensional index structure and generates a client kick-out instruction.

[0104] When generating instructions, the control plane module carries the following information in the same DBus instruction message: client IP address, execution reason identifier, and policy identifier.

[0105] The instruction is sent to the data plane module via the DBus method call interface.

[0106] Upon receiving the instruction, the data plane module locates the target client connection based on the client's IP address and performs a disconnection operation. After completing the disconnection process, the data plane module generates an execution result message and returns the execution result along with the execution reason identifier to the control plane module via DBus.

[0107] After receiving the execution feedback, the control plane module associates and records the feedback information with the original control command, thereby forming a closed-loop association mechanism between the control command and the execution feedback, and realizing the traceability and verifiability of the execution results of the control strategy.

[0108] The DBus interaction middleware layer is used to establish a unified communication mechanism and interface specification between the control plane module and the data plane module. According to one embodiment of the present invention, the interface names are expanded as follows: Control plane adds interfaces: com.mobile.bigdata.controlplane.ClientCtrl (client management), com.mobile.bigdata.controlplane.MsgCtrl (message processing management); Data plane adds interface: com.mobile.bigdata.dataplane.ClientStatus (client status), realizing standardized management of control command issuance, status information reporting, and system operation information exchange. Specifically, it includes: The DBus interaction middleware layer builds a unified communication interface system based on the DBus bus mechanism. This includes a control interface for the control plane module to issue control policies and management commands to the data plane module, and a status interface for the data plane module to report runtime status information and execution results to the control plane module. Through this interface system, the control plane module can send policy-based control commands such as maximum connection limit commands, failure message retransmission commands, and client kick-out commands to the data plane module via a unified interface. Conversely, the data plane module can report runtime status data such as libuv runtime status fields, client status fields, connection statistics, and message processing failure rates to the control plane module via a unified interface structure. All of these interfaces are registered and invoked as DBus services. The control plane module, as the DBus service caller, initiates method calls, and the data plane module, as the service provider, receives the calls and executes the corresponding logic. Simultaneously, the data plane module periodically sends runtime status signals to the control plane module through the status reporting interface, enabling bidirectional flow of control and status information on the DBus bus.

[0109] The DBus interactive middleware layer encapsulates all control command messages and status messages using a unified data structure, ensuring structural and parsing consistency across different message types during transmission. For example... Figure 3 As shown, the unified data structure includes at least an instruction type field (instruction format), a parameter field, and a verification field in the message body. The instruction type field identifies the specific instruction or status category to which the current message belongs. The parameter field carries the strategy parameters or status reporting data required to execute the instruction. The verification field, combined with the interaction protocol, ensures the integrity and correctness of the message during transmission. For various control instructions generated by the control plane module, after being encapsulated into a unified data structure, they are transmitted via the DBus bus. The DBus interaction middleware layer is responsible for message encapsulation, transmission, and bus scheduling, ensuring that control instructions reach the data plane module according to the predetermined interface specifications. Status information generated by the data plane module is also encapsulated according to the unified data structure and sent to the control plane module via the DBus bus, achieving standardized transmission of system status information.

[0110] Furthermore, the DBus interaction middleware layer encodes control commands and status data using a unified serialization structure during message transmission. In the message sending phase, the DBus interaction middleware layer first organizes the command type field, parameter field, and verification field according to a predefined data structure. Then, it encodes the message body using the serialization structure to adapt it to the DBus bus transmission format. After encoding, the message is written to the DBus bus for transmission. When the data plane module receives a control command from the DBus bus, the DBus interaction middleware layer first decodes the message to recover the original unified data structure and then proceeds to the command verification phase. In this phase, the message body is checked for integrity based on the verification field to confirm that no data corruption or structural abnormalities have occurred during transmission. After the integrity check passes, a type matching check is performed based on the command type field to confirm that the current data plane module has the capability to execute the command type and that the interface matching relationship is correct. The control command is only allowed to enter the execution flow if both integrity and type matching checks pass, and the data plane module calls the corresponding execution logic to complete the specific control operation. If either check fails, the DBus interaction middleware will terminate the execution flow and generate an exception handling result, such as... Figure 4 As shown, this prevents erroneous instructions from entering the system execution path.

[0111] Through the above structural design, the DBus interaction middleware not only unifies the interface specifications between the control plane module and the data plane module, but also ensures the structural consistency, parsing consistency, and data integrity of control commands and status data during transmission through a unified data structure encapsulation mechanism and a unified serialization encoding mechanism. Simultaneously, by introducing integrity verification and type matching verification mechanisms on the data plane module side, the system can verify the legality of messages before executing control commands, thereby improving the reliability and security of the entire control link and ensuring that control plane policies can be executed accurately and stably.

[0112] In summary, this invention discloses a collaborative system and method for a mobile big data platform based on libuv and DBus. Building upon existing technologies, it significantly enhances the data plane's ability to receive client connections / messages, as well as the control plane's ability to precisely control program behavior and acquire full-dimensional operational status. The platform achieves standardized access for multi-protocol clients through a data plane client connection management component, and enables fine-grained control of client connections, message processing, and libuv configuration through an enhanced DBus interaction middleware layer. This solves the problems of poor client compatibility, low program control accuracy, and incomplete status awareness found in existing platforms.

[0113] The following is for reference. Figure 5It shows a schematic diagram of the structure of a computer system 500 suitable for implementing electronic devices according to embodiments of the present application. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0114] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0115] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a liquid crystal display (LCD) and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card and a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0116] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined in the methods of this application. It should be noted that the computer-readable storage medium of this application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0117] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0118] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0119] The modules described in the embodiments of this application can be implemented in software or in hardware.

[0120] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable storage medium carries one or more programs. When the electronic device executes one or more of these programs, the electronic device causes the following: a control plane module to start and listen for status signals; a data plane module to start and initialize the listening port and connection management components; a data plane module to periodically report status data; a control plane module to generate control commands based on the status data and send them to the data plane module through the DBus interaction middleware layer; and a data plane module to execute the commands, adjust its operating behavior, and continue to report the status, thereby forming a closed-loop control.

[0121] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A collaborative system for a mobile big data platform based on libuv and DBus, characterized in that, The mobile big data platform collaborative system includes a data plane module, a control plane module, and a DBus interaction middleware layer, and the three communicate through the DBus bus, specifically including: The data plane module is built on the libuv event-driven library and is used to complete the collection, processing and client connection management of mobile big data. The control plane module is used for global policy control and state analysis, specifically including: deploying the DBus server component, receiving state signals sent by the data plane module, and parsing the messages; extracting libuv runtime state fields and client state fields from the state signals, and calculating the message processing failure rate; generating a maximum connection limit instruction when the current number of connections exceeds a preset threshold, and sending it to the data plane module via a DBus method call; generating a failure message retransmission instruction and sending it when the message processing failure rate exceeds a preset threshold; and constructing runtime state analysis logic to continuously monitor the data plane's operation and output analysis results. The DBus interaction middleware layer is used to define a unified interface specification, specifically including: defining the interface from the control plane to the data plane; defining the status interface from the data plane to the control plane; encapsulating all instruction messages and status messages using a unified data structure and transmitting them through the DBus bus.

2. The mobile big data platform collaborative system according to claim 1, characterized in that, The data plane module specifically also includes: A client connection management component is constructed, which listens to the network port and accepts client connections under the event loop mechanism. The client connections support both TCP and UDP protocols. When a client connection request is received, the current number of connections is obtained and compared with the preset maximum number of client connections. When the limit is reached, the connection is rejected and the corresponding file descriptor is closed. Create a connection information structure for each access client. The structure includes the client IP address, connection file descriptor, connection status, received message count, successful processing count, and failed processing count. Add the structure to the global connection list. Based on the event callback mechanism, continuously listen for client message reading events, and after reading the message, call the libuv processing logic to process the data, and count the success and failure of processing respectively. Deploy the DBus client component to receive management instructions issued by the control plane. The management instructions include instructions to adjust the maximum number of client connections and instructions to kick out clients. After parsing, update local operating parameters or disconnect the specified client connection. Real-time collection of operational status data and client interaction status data, including data throughput, processing latency and resource utilization, and client interaction status data including the total number of current connections, the cumulative number of received messages and the cumulative number of processing failures. Status messages are generated at fixed time intervals and sent to the control plane module via the DBus signal.

3. The mobile big data platform collaborative system according to claim 2, characterized in that, The client connection management component employs a file descriptor-based multiplexing mechanism in the event loop to bind client connection objects to event callback functions one by one. It also completes connection state initialization, message counter initialization, and read event registration when the connection is established, enabling each client to achieve non-blocking concurrent processing in a single event loop.

4. The mobile big data platform collaborative system according to claim 1 or 2, characterized in that, When the data plane module enforces the maximum number of client connections, it adopts a dual threshold control mechanism, including a connection access threshold and a connection recycling threshold. When the current number of connections reaches the connection access threshold, it refuses to establish new connections. When the current number of connections exceeds the connection recycling threshold, it actively disconnects high-failure-rate client connections based on the client's historical failure rate.

5. The mobile big data platform collaborative system according to claim 2, characterized in that, The client message processing process includes three stages: message parsing, classification and scheduling, and processing execution. In the message parsing stage, the message type field is extracted. In the classification and scheduling stage, messages are allocated to the corresponding processing queues according to the message priority parameters. In the processing execution stage, the libuv thread pool schedules the messages for execution in priority order and maintains a message backlog count for each queue.

6. The mobile big data platform collaborative system according to claim 1, characterized in that, When performing state analysis, the control plane module constructs a time window statistical model to calculate the rate of change of the number of connections and the rate of change of the message failure rate within multiple consecutive state reporting cycles. When the rate of change exceeds a preset growth threshold, an early warning level control instruction is triggered, and when the absolute value exceeds the threshold, a mandatory level control instruction is triggered.

7. The mobile big data platform collaborative system according to claim 1, characterized in that, The DBus interaction middleware uses a unified serialization structure to encode control commands and status data. The message body includes command type field, parameter field and verification field. After receiving the command, the data plane module performs integrity verification and type matching verification. Only after the verification is passed will the execution process begin.

8. The mobile big data platform collaborative system according to claim 1, characterized in that, In addition to sending aggregated statistical data, the data plane module also sends detailed data at the client level when reporting status. The detailed data includes the client IP, connection status, last interaction time, and corresponding message processing statistics. Furthermore, an index structure based on the client dimension is constructed on the control plane module side to quickly locate abnormal clients.

9. The mobile big data platform collaborative system according to claim 1, characterized in that, When the control plane module issues a client kick-out command, it locates the target client based on the client-dimensional index structure and carries the execution reason identifier and policy identifier in the same command. After the data plane module performs the disconnection operation, it sends back the execution result and the reason identifier together, thus forming a closed-loop association mechanism between control commands and execution feedback.

10. A collaborative method for mobile big data platforms based on libuv and DBus, characterized in that, include: The collaborative method executed by the system according to any one of claims 1 to 9 specifically includes: the control plane module starting up and listening to status signals; the data plane module starting up and initializing the listening port and connection management component; the data plane module periodically reporting status data; the control plane module generating control instructions based on the status data and sending them to the data plane module through the DBus interaction middleware layer; and the data plane module adjusting its operating behavior and continuing to report status after executing the instructions, thereby forming a closed-loop control.

Images