A communication mode dynamic decision method and device based on multi-objective optimization and online learning
By employing a dynamic decision-making method for communication modes that combines multi-objective optimization and online learning, system parameters are collected in real time, future performance is predicted, and selection actions are generated. This solves the problems of adaptability and balance in communication mode selection in existing technologies and enables intelligent optimization and allocation of communication resources.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIAMEN XINGZONG DIGITAL TECH CO LTD
- Filing Date
- 2025-12-05
- Publication Date
- 2026-06-09
Smart Images
Figure CN122179398A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer network communication and cloud computing resource scheduling technology, and in particular to a dynamic decision-making method and apparatus for communication modes based on multi-objective optimization and online learning. Background Technology
[0002] In modern distributed systems and IoT applications, the choice of communication mode directly affects system performance and cost. Currently, the mainstream communication modes include WebSocket persistent connection mode and Webhook on-demand reporting mode. WebSocket mode achieves real-time communication by establishing persistent connections, but it requires continuous consumption of server resources; Webhook mode uses on-demand requests, which has the advantage of low resource consumption, but cannot guarantee real-time performance.
[0003] Existing communication mode selection methods have the following limitations: First, most solutions adopt a static configuration strategy, which cannot adaptively adjust according to the dynamic changes in the system's operating status; second, existing dynamic decision-making methods often only consider a single objective (such as cost or delay), making it difficult to achieve a balance among multiple conflicting objectives; third, traditional methods lack online learning capabilities and cannot continuously optimize decision quality from historical decision effects.
[0004] Especially in scenarios with large fluctuations in business load and unstable network environment, a fixed communication mode selection strategy often leads to the following problems: maintaining WebSocket long connections during low load periods will result in resource waste; using Webhook mode during high load periods may lead to increased communication latency; and the inability to adjust the communication strategy in a timely manner when network conditions deteriorate will affect system reliability.
[0005] Therefore, there is an urgent need for a dynamic decision-making method for communication modes that can comprehensively consider factors such as cost, latency, and reliability, and has online learning capabilities, in order to achieve intelligent optimization of communication resource allocation. Summary of the Invention
[0006] This invention provides a method and apparatus for dynamic decision-making on communication modes based on multi-objective optimization and online learning. It can realize intelligent dynamic switching of communication modes and effectively balance multiple requirements such as communication cost, transmission delay and system reliability while ensuring communication reliability.
[0007] The first aspect of this invention provides a dynamic decision-making method for communication patterns based on multi-objective optimization and online learning, comprising: Real-time acquisition of system operating parameters, and construction of multi-dimensional state vectors based on system operating parameters; Based on historical performance data, and using a time series prediction model, we output the performance prediction values of WebSocket long connection mode and Webhook on-demand reporting mode in future decision-making cycles. Based on the multidimensional state vector and performance prediction values, a communication mode selection action is generated by calculating a multi-objective reward function; Perform the communication mode selection action. If the action indicates a mode switch, complete the switch while ensuring no data loss. Collect actual system performance data after the action is executed, calculate the reward value based on the actual system performance data, and update the decision-making strategy according to the reward system.
[0008] Optionally, system operating parameters are collected in real time, and a multi-dimensional state vector is constructed based on the system operating parameters, including: the multi-dimensional state vector contains business load characteristics, network environment indicators, resource cost measurements, and historical communication patterns; Collect instantaneous data arrival rate and average data packet size to characterize service load features; Collect network latency and packet loss rate and calculate them together as a network status indicator; Collect server resource consumption data to quantify the maintenance cost of a WebSocket connection and the processing cost of a single Webhook request, thereby obtaining a resource cost metric. Record the abnormal disconnection rate of WebSocket connections to assess their connection health, and introduce business priority weights defined by business logic, as well as the communication mode used in the previous moment, to obtain the historical communication mode.
[0009] Optionally, using a time series forecasting model, the performance prediction values for WebSocket long-connection mode and Webhook on-demand reporting mode are output for future decision-making cycles, including... Using time series forecasting models, rolling predictions are made for the connection maintenance cost and average transmission latency of WebSocket mode, and the single request cost and average transmission latency of Webhook mode, respectively. The performance prediction values of WebSocket long connection mode and Webhook on-demand reporting mode in future decision-making cycles are output.
[0010] Optionally, a multi-objective reward function is calculated, including: The multi-objective reward function is constructed as a weighted sum of cost, latency, and reliability penalty terms. The cost term is calculated by combining the performance prediction value with the mode switching overhead, the latency term is calculated by combining the performance prediction value with the business priority, and the reliability penalty term is activated when the WebSocket connection health is lower than a preset threshold.
[0011] Optionally, the weight coefficients of the weighted sum are determined by the analytic hierarchy process, which includes: constructing a judgment matrix to characterize the relative importance of cost, delay and reliability, and calculating the eigenvectors of the judgment matrix to determine the weight coefficients of each objective.
[0012] Optionally, in the step of generating communication mode selection actions, the Q-Learning algorithm is used as the decision core, and a hybrid exploration strategy is applied to select actions. The hybrid exploration strategy introduces an upper confidence bound term related to the number of times the action is selected to guide the exploration, based on the ε-greedy strategy.
[0013] Optionally, if the action indication mode is switched, the switch is completed while ensuring no data loss, including: When the decision action is to switch from Webhook mode to WebSocket mode, the data reporting will continue to use Webhook mode until the WebSocket connection is fully established and confirmed to be available.
[0014] A second aspect of the present invention provides a dynamic decision-making device for communication modes based on multi-objective optimization and online learning, comprising: The state-aware unit is used to collect system operating parameters in real time and construct a multi-dimensional state vector based on the system operating parameters; The performance prediction unit is used to output the performance prediction values of WebSocket long connection mode and Webhook on-demand reporting mode in the future decision-making cycle based on historical performance data and time series prediction models. The decision generation unit is used to generate communication mode selection actions by calculating a multi-objective reward function based on a multi-dimensional state vector and performance prediction values. The action execution unit is used to perform the communication mode selection action. If the action indicates a mode switch, the switch is completed while ensuring that no data is lost. The feedback learning unit is used to collect actual system performance data after the action is executed, calculate the reward value based on the actual system performance data, and update the decision-making strategy according to the reward system.
[0015] A third aspect of the present invention provides an electronic device, comprising: One or more processors; A memory on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the communication mode dynamic decision-making method based on multi-objective optimization and online learning as described in any of the above.
[0016] A fourth aspect of the present invention provides a computer storage medium for storing a program, which, when executed, is used to implement the communication mode dynamic decision-making method based on multi-objective optimization and online learning as described in any of the preceding claims.
[0017] Beneficial Effects: This invention discloses a method and apparatus for dynamic decision-making on communication modes based on multi-objective optimization and online learning. The method includes: real-time acquisition of system operating parameters to construct a multi-dimensional state vector; prediction of performance indicators for WebSocket and Webhook communication modes in future periods based on historical data; calculation of a communication mode selection action using a multi-objective reward function based on the state vector and performance prediction values; execution of the selection action while ensuring no data loss during mode switching; acquisition of actual performance data to calculate reward values; and online updating of the decision-making strategy based on the reward values. This invention, through the combination of multi-objective optimization and online learning, achieves intelligent dynamic switching of communication modes, effectively balancing communication cost, transmission latency, and system reliability while ensuring communication reliability. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 A flowchart illustrating a dynamic decision-making method for communication modes based on multi-objective optimization and online learning, provided in an embodiment of the present invention; Figure 2 A diagram illustrating a dynamic decision-making system architecture for a communication mode, provided in an embodiment of the present invention. Figure 3 A schematic diagram of a communication mode dynamic decision-making device based on multi-objective optimization and online learning provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0020] This invention provides a method and apparatus for dynamic decision-making on communication modes based on multi-objective optimization and online learning. It can realize intelligent dynamic switching of communication modes and effectively balance multiple requirements such as communication cost, transmission delay and system reliability while ensuring communication reliability.
[0021] To facilitate understanding, the application scenarios of the embodiments of the present invention will be introduced first.
[0022] In microservice architectures and cloud-native applications, data reporting is an extremely common communication requirement, such as device telemetry data collection, user behavior log uploading, and application performance monitoring (APM) metric reporting. Currently, the mainstream data reporting methods in the industry are mainly divided into two categories: WebSocket (long connection) mode and Webhook (HTTP callback / on-demand reporting) mode.
[0023] WebSocket (long-lived connection) mode establishes a TCP connection between the client and server and maintains this connection for a relatively long period of time, allowing data to be transmitted continuously and with low latency in full-duplex mode. Webhook (HTTP callback / on-demand reporting) mode does not maintain a persistent connection. Instead, the client sends an HTTP (usually POST) request to a URL specified by the server only when data needs to be reported, and closes the connection immediately after the request is completed.
[0024] These two models each have their significant advantages and disadvantages, forming a typical engineering trade-off: Once a WebSocket connection is established, data transmission latency is extremely low; it is suitable for high-frequency, small-batch real-time data streams; the server can proactively push messages. However, regardless of whether data is being transmitted, the connection needs to be maintained, consuming server-side resources such as memory and CPU (connection cost); in unstable network environments, the connection may be unexpectedly interrupted, requiring a reconnection mechanism, which increases complexity.
[0025] The Webhook pattern has no connection maintenance overhead, and resource consumption is proportional to the number of requests, making it more economical in scenarios with intermittent bursts of data or low frequency. It conforms to the RESTful architectural style and is easy to understand and implement. However, each request requires a TCP three-way handshake and a TLS handshake (if using HTTPS), resulting in significantly higher latency than WebSocket. Under high-frequency requests, handshake overhead and connection reuse issues may become performance bottlenecks.
[0026] Existing technical solutions and their shortcomings: Static configuration strategy: During the system design phase, developers manually select a communication mode based on the expected load and embed it in the code. The drawback of this approach is significant: it cannot adapt to dynamic growth or cyclical fluctuations in business volume. A system that chooses Webhooks in the early stages of business may experience a sharp performance drop due to excessive HTTP request overhead after a surge in business volume; conversely, a WebSocket system designed for peak periods will waste a large amount of server resources during off-peak periods.
[0027] Dynamic switching based on simple thresholds: Some systems attempt to implement dynamic switching, for example, switching from Webhook to WebSocket when the data reporting frequency exceeds a certain preset threshold. However, this approach has the following problems: Lag: Threshold triggering always occurs when a performance problem has already occurred or is about to occur, and is a passive "firefighting" behavior.
[0028] Single-dimensional approach: It only considers the data frequency (QPS) dimension, ignoring multiple key factors such as network latency, current server load, business priority, and communication cost, resulting in one-sided decision-making.
[0029] Hysteresis configuration is complex: To avoid frequent jittering of the mode near the threshold, a hysteresis loop needs to be set up, which increases the complexity of the configuration and makes the parameters difficult to optimize.
[0030] Forecasts based on historical averages: Slightly more advanced systems make forecasts based on historical averages, but their forecasting models are too simple (such as moving averages) and cannot capture complex time series patterns (such as trends and periods), resulting in limited forecast accuracy and inaccurate decision-making.
[0031] Therefore, this invention proposes a dynamic decision-making method for communication patterns based on multi-objective optimization and online learning. It models communication pattern selection as a sequential decision problem and solves it using a reinforcement learning framework that integrates online prediction and multi-objective optimization. See also... Figure 2 This figure is an architecture diagram of a communication mode dynamic decision-making system provided by an embodiment of the present invention, illustrating the communication mode dynamic decision-making system based on multi-objective optimization and online learning and its workflow. Figure 2 The system's core components, data flow, and decision-making loop are clearly described, including three main levels and each core component, as detailed below: Data acquisition and monitoring layer: The client (data producer) is the source of the data, generating the data that needs to be reported.
[0032] The communication mode executor receives decision instructions and performs the actual execution by reporting data through WebSocket or Webhook modes.
[0033] The system monitor is responsible for collecting real-time performance metrics streams and recording monitoring data, including but not limited to QPS (queries per second), latency, success rate, and cost. The system monitor will also record historical data.
[0034] Knowledge base and feedback: The historical performance dataset stores historical data recorded by the system monitor.
[0035] Intelligent Decision Engine: The feature engineering module reads monitoring data from the system monitor, processes it, and outputs a state vector. .
[0036] The online performance predictor receives a state vector from the feature engineering module. Based on historical data from the historical performance dataset, output predicted performance metrics. , .
[0037] Multi-objective reward calculator based on predictive performance metrics , Calculate immediate reward .
[0038] The reinforcement learning decision-maker receives a state vector from the feature engineering module. And immediate rewards from the multi-objective reward calculator This leads to the output of actions. (Switch instruction) to the communication mode executor.
[0039] The state-action value function (Q-table) serves as the core of the reinforcement learning decision-maker, receiving instructions from the decision-maker and updating the Q-value.
[0040] This closed-loop process continues to run, enabling the system to adapt to changes in business models and converge to the long-term optimal communication strategy.
[0041] See Figure 1 This figure is a flowchart illustrating a dynamic decision-making method for communication modes based on multi-objective optimization and online learning, provided by an embodiment of the present invention. The dynamic decision-making method for communication modes based on multi-objective optimization and online learning provided by this embodiment of the present invention can be implemented, for example, through the following steps S101-105.
[0042] S101: Real-time acquisition of system operating parameters, and construction of a multi-dimensional state vector based on the system operating parameters.
[0043] In this embodiment of the invention, the multidimensional state vector includes service load characteristics, network environment indicators, resource cost metrics, and historical communication patterns; instantaneous data arrival rate and average data packet size are collected to characterize service load characteristics; network latency and packet loss rate are collected and comprehensively calculated as network condition indicators; server resource consumption is collected to quantify the maintenance cost of WebSocket connections and the processing cost of a single Webhook request, respectively, to obtain resource cost metrics; abnormal disconnection rate of WebSocket connections is recorded to assess its connection health, and business priority weights defined by business logic and the communication pattern used in the previous moment are introduced to obtain historical communication patterns.
[0044] Specifically, system status It is a multi-dimensional vector that comprehensively describes the business characteristics, system load, and network environment at time t.
[0045] ; The instantaneous data arrival rate represents the number of data packets that need to be reported per unit time (second); it is calculated using an exponentially weighted sliding window counter, as shown in the following formula: ; in, It is a smoothing factor; Indicates within the time window The total number of successfully transmitted data packets. "Successful transmission" means that the data packet has been sent and received an application layer confirmation from the receiving end. It is used to calculate the data arrival rate and is a core indicator of business activity. Indicates the length of the time window for data collection and statistics. Δt is a configurable parameter, typically set to a short time interval (such as 1 second or 5 seconds) to enable rapid response to changes in traffic volume. The choice of Δt reflects the trade-off between system response sensitivity and computational stability.
[0046] The average packet size represents the average number of bytes in recently reported packets; packets of different sizes have different impacts on network transmission and serialization / deserialization costs.
[0047] This is a comprehensive network condition indicator, calculated based on the latency and packet loss rate of the most recent N communications, using the following formula: ; in, Packet loss rate represents the percentage of data packets lost out of the total number of packets sent during network transmission. It is a key indicator for measuring network stability and reliability. A high packet loss rate can directly lead to audio and video stuttering and data retransmission. The average round-trip time (RTD) represents the average time required for a data packet to travel from the sender to the receiver and back to the acknowledgment signal. It is a core indicator for measuring network latency and congestion, and directly affects the interactive experience of real-time communication; The closer the value is to 1, the better the network condition.
[0048] The WebSocket connection resource cost is represented by a comprehensive score, calculated using the following formula: ; in, and It is a weighting coefficient, which can be configured according to server resources; This refers to the memory usage per connection, representing the average memory space required to maintain a single WebSocket long connection on the server side. It includes the kernel memory allocated by the operating system to maintain the TCP connection and control block (TCB), as well as the user space memory allocated by the application to manage the connection session (such as storing user state and message buffers). The unit is usually kilobytes (KB). CPU usage per connection represents the percentage of CPU processing time consumed on the server side to maintain a single WebSocket long-lived connection. It mainly includes the overhead of handling connection heartbeat keep-alive, internal state maintenance, and basic scheduling when idle. It is a dimensionless percentage value that represents the proportion of a single connection's processing power used by a single CPU core per unit of time.
[0049] The cost per request for a WebHook is calculated using the following formula: ; in, and These are the weighting coefficients. This refers to the memory usage per request, which represents the average memory consumed by the server in processing a complete Webhook HTTP request-response cycle. This includes memory allocation during operations such as parsing HTTP requests, constructing responses, and executing related business logic. The unit is usually kilobytes (KB). CPU usage per request represents the percentage of CPU time consumed by the server in processing a complete Webhook HTTP request-response cycle; it is a dimensionless percentage value. This is the bandwidth weighting coefficient, an adjustment parameter used in the cost function to balance the importance of bandwidth costs with other resource costs (CPU, memory). It is a constant greater than 0, set by the system administrator based on the ratio of actual network bandwidth costs to computing resource costs. For example, if bandwidth costs are very high, it can be increased accordingly. Avg_Packet_Size is the average packet size, representing the average volume of data reported via Webhook each time, usually in bytes. This value is used to estimate the network bandwidth consumption of a single request.
[0050] The formula for determining the health of a WebSocket connection is as follows: ; Where β is the attenuation coefficient. This represents the recent abnormal disconnection rate, indicating the frequency of abnormal interruptions to the WebSocket connection within a recent period (e.g., the past hour). "Abnormal disconnection" refers to all disconnections other than normal closures initiated by the client or server (carrying a standard close frame), including but not limited to: connection loss due to network timeout; client crashes; and server-initiated disconnections due to excessive load. This metric directly reflects the robustness and stability of the WebSocket connection channel and is a key indicator for evaluating its reliability. The more stable the connection, the closer N_t is to 1.
[0051] Business priority weights represent static parameters specified by business logic, such as financial transaction data. Operation log .
[0052] The pattern of the previous moment is represented as This state is introduced to impose a cost on mode switching itself, thus avoiding frequent switching.
[0053] S102: Based on historical performance data, using a time series prediction model, output the performance prediction values of WebSocket long connection mode and Webhook on-demand reporting mode in future decision-making cycles.
[0054] In this embodiment of the invention, a time series prediction model is used to perform rolling predictions on the connection maintenance cost and average transmission latency of WebSocket mode and the single request cost and average transmission latency of Webhook mode, respectively, and output the performance prediction values of WebSocket long connection mode and Webhook on-demand reporting mode in the future decision-making cycle.
[0055] Specifically, using historical performance data, predictions are made for the two communication modes in the next decision-making cycle. The core performance indicators within the model. The time series prediction model in this embodiment of the invention can be an ARIMA model.
[0056] The ARIMA(p, d, q) model is a classic method for time series forecasting, where p is the number of autoregressive terms, d is the difference order, and q is the number of moving average terms. Its mathematical expression is: ; in, It is a shift operator. and These are model parameters. It is a white noise sequence, and c is a constant term.
[0057] The prediction targets are as follows: For the WebSocket model, predict its connection maintenance cost. and average transmission delay .
[0058]
[0059]
[0060] For the Webhook pattern, predict its average cost per request. and average delay .
[0061]
[0062]
[0063] Model parameters (p, d, q) are selected using the AIC / BIC criteria and are periodically refitted with the latest data to adapt to changes in the data distribution.
[0064] S103: Based on the multidimensional state vector and performance prediction value, generate the communication mode selection action by calculating the multi-objective reward function.
[0065] In this embodiment of the invention, the multi-objective reward function is constructed as a weighted sum of cost, latency, and reliability penalty terms. The cost term is calculated by combining the predicted performance value with mode switching overhead; the latency term is calculated by combining the predicted performance value with business priority; and the reliability penalty term is activated when the WebSocket connection health is below a preset threshold. The weight coefficients of the weighted sum are determined using the analytic hierarchy process (AHP). Specifically, this involves constructing a judgment matrix to characterize the relative importance of cost, latency, and reliability, and calculating the eigenvectors of the judgment matrix to determine the weight coefficients of each objective. The Q-Learning algorithm is used as the decision core, and a hybrid exploration strategy is applied to select actions. This hybrid exploration strategy, based on the ε-greedy strategy, introduces an upper confidence bound term related to the number of times an action is selected to guide the exploration.
[0066] Specifically, the reward function is key to unifying multiple optimization objectives into a single scalar value, and is defined as:
[0067] .
[0068] Cost item: if (Switch to / maintain WebSocket) Represented as: ; if (Switch to / maintain Webhook), then Represented as: ; in, It is the cost of mode switching itself (such as the overhead of establishing / tearing up a connection). Indicates an indicator function.
[0069] Latency: if ,but ; if ,but .
[0070] Multiplying the delay factor by the business priority means that the penalty for delay is amplified for high-priority business processes.
[0071] Penalty (primarily used to address reliability issues): if And the predicted connection health , but ;otherwise .
[0072] Weighting coefficient , , : The weighting coefficients are determined using the analytic hierarchy process (AHP) combined with business requirements. First, decision-makers compare the importance of each objective pairwise, constructing a judgment matrix. Then, the eigenvector corresponding to the largest eigenvalue of the matrix is calculated, and after normalization, the weighting coefficients are obtained. , , This method ensures the scientific nature and consistency of weight settings.
[0073] The reinforcement learning decision-maker (Q-Learning algorithm) employs a model-free Q-Learning algorithm because it does not require transition probabilities from the environment.
[0074] Q-function and update rules: The Q-function Q(S, A) stores the long-term value of performing action A in state S.
[0075] The update rules are as follows: .
[0076] Learning rate Adaptive learning rate It involves rapid learning in the initial stage and tends to stabilize later.
[0077] Discount factor The value is usually 0.9, indicating a high degree of importance attached to future rewards.
[0078] Exploration-Exploitation Strategy (UCBHybrid): To balance exploration and exploitation, a hybrid strategy of upper confidence bound (UCB) and ε-greedy is adopted.
[0079] The expression for the action selection probability is as follows:
[0080] ; in It is a state Number of visits Is Select action Number of times, It is the exploration coefficient. The UCB item encourages exploration of actions that require fewer visits.
[0081] S104: Perform a communication mode selection action. If the action indicates a mode switch, the switch will be completed while ensuring that no data is lost.
[0082] In this embodiment of the invention, when the decision action is to switch from Webhook mode to WebSocket mode, the data reporting is continued to use Webhook mode until the WebSocket connection is fully established and confirmed to be available.
[0083] Specifically, the communication mode executor receives instructions. If necessary, perform a mode switch and ensure that no data is lost during the switch. For example, temporarily use a Webhook to report before the WebSocket connection is successfully established.
[0084] S105: Collect actual system performance data after the action is executed, calculate the reward value based on the actual system performance data, and update the decision-making strategy according to the reward system.
[0085] In this embodiment of the invention, at the next decision point, the measurement is performed during the action. The actual costs, delays, and reliability are considered. The reward calculator calculates immediate rewards based on actual results. .
[0086] The decision-maker observes a new state And based on the Q-Learning update rules, iterate its strategy and update the Q table C1.
[0087] Beneficial effects: Multi-dimensional dynamic adaptation: This invention no longer relies on a single threshold, but makes decisions based on multi-dimensional information such as data volume, network, cost, and business priority, and can dynamically adjust with changes in the environment to achieve precise elasticity.
[0088] Forward-looking decision-making: This invention, through online ARIMA prediction, has the ability to "foresee the future" and can make decisions based on the predicted performance indicators, avoiding the lag of traditional methods.
[0089] Multi-objective global optimization: This invention uses a scientifically quantified multi-objective reward function and weight determination method (AHP) to find the optimal balance point that aligns with business priorities among competing objectives (cost, latency, reliability).
[0090] Possesses online learning capabilities: This invention, through a reinforcement learning framework, can learn on its own in a production environment through trial and error without requiring a large amount of historical labeled data. Its strategies continuously evolve and it has strong robustness to unforeseen business models.
[0091] Deep integration of high-order algorithms: This invention creatively integrates algorithms from different mathematical fields, such as Time Series Prediction (ARIMA), Multi-Attribute Decision Theory (AHP), and Reinforcement Learning (Q-Learning), to provide a brand-new, highly intelligent solution for solving complex engineering selection problems.
[0092] Now combined Figure 2 For example, ...
[0093] Initialization: The system starts, initializes the Q table, and loads the ARIMA model. It begins with a default mode.
[0094] State awareness and feature engineering: The monitor continuously collects raw metrics, and the feature engineering module calculates the state vector. .
[0095] Forecasting and Decision Making: The online performance predictor runs an ARIMA model based on data from the historical database C2 and outputs predicted values. , , , .
[0096] Reinforcement learning decision makers based on the current state Using Q-table C1 and a hybrid exploration strategy, select an action. .
[0097] Execution: Communication mode executor receives instructions If necessary, perform a mode switch and ensure that no data is lost during the switch (for example, temporarily use Webhook to report before the WebSocket connection is successfully established).
[0098] Measurement and Learning: At the next decision point, the system measures the action. The actual costs, delays, and reliability incurred.
[0099] The reward calculator calculates immediate rewards based on actual results. .
[0100] The decision-maker observes a new state And based on the Q-Learning update rules, iterate its strategy and update the Q table C1.
[0101] Continuous optimization: This closed-loop process runs continuously, enabling the system to adapt to changes in business models and converge to the long-term optimal communication strategy.
[0102] Based on the methods provided in the above embodiments, this invention also provides a dynamic decision-making device for communication modes based on multi-objective optimization and online learning. The following describes the dynamic decision-making device for communication modes based on multi-objective optimization and online learning in conjunction with the accompanying drawings.
[0103] See Figure 3 The figure is a schematic diagram of the structure of a communication mode dynamic decision-making device based on multi-objective optimization and online learning provided in an embodiment of the present invention.
[0104] The communication mode dynamic decision-making device 300 based on multi-objective optimization and online learning provided in this embodiment of the invention includes: a state perception unit 301, a performance prediction unit 302, a decision generation unit 303, an action execution unit 304, and a feedback learning unit 305.
[0105] The state sensing unit 301 is used to collect system operating parameters in real time and construct a multi-dimensional state vector based on the system operating parameters; The performance prediction unit 302 is used to output the performance prediction values of WebSocket long connection mode and Webhook on-demand reporting mode in the future decision-making period based on historical performance data and using a time series prediction model. The decision generation unit 303 is used to generate a communication mode selection action by calculating a multi-objective reward function based on a multi-dimensional state vector and performance prediction value. The action execution unit 304 is used to perform the communication mode selection action. If the action indicates a mode switch, the switch is completed while ensuring that no data is lost. Feedback learning unit 305 is used to collect actual system performance data after the action is executed, calculate reward value based on actual system performance data, and update decision-making strategy according to reward system.
[0106] In one possible implementation, the state-aware unit 301 is specifically used for: The multidimensional state vector includes service load characteristics, network environment indicators, resource cost metrics, and historical communication patterns; Collect instantaneous data arrival rate and average data packet size to characterize service load features; Collect network latency and packet loss rate and calculate them together as a network status indicator; Collect server resource consumption data to quantify the maintenance cost of a WebSocket connection and the processing cost of a single Webhook request, thereby obtaining a resource cost metric. Record the abnormal disconnection rate of WebSocket connections to assess their connection health, and introduce business priority weights defined by business logic, as well as the communication mode used in the previous moment, to obtain the historical communication mode.
[0107] In one possible implementation, the performance prediction unit 302 is specifically used for: Using time series forecasting models, rolling predictions are made for the connection maintenance cost and average transmission latency of WebSocket mode, and the single request cost and average transmission latency of Webhook mode, respectively. The performance prediction values of WebSocket long connection mode and Webhook on-demand reporting mode in future decision-making cycles are output.
[0108] In one possible implementation, the decision generation unit 303 has the following functions: The multi-objective reward function is constructed as a weighted sum of cost, latency, and reliability penalty terms. The cost term is calculated by combining the performance prediction value with the mode switching overhead, the latency term is calculated by combining the performance prediction value with the business priority, and the reliability penalty term is activated when the WebSocket connection health is lower than a preset threshold.
[0109] In one possible implementation, the decision generation unit 303 has the following functions: The weight coefficients of the weighted sum are determined by the analytic hierarchy process, which includes: constructing a judgment matrix to characterize the relative importance of cost, delay and reliability, and calculating the eigenvectors of the judgment matrix to determine the weight coefficients of each objective.
[0110] In one possible implementation, the decision generation unit 303 has the following functions: The Q-Learning algorithm is used as the decision core, and a hybrid exploration strategy is applied to select actions. The hybrid exploration strategy introduces an upper confidence bound term related to the number of times an action is selected to guide the exploration, based on the ε-greedy strategy.
[0111] In one possible implementation, the feedback learning unit 305 is specifically used for: When the decision action is to switch from Webhook mode to WebSocket mode, the data reporting will continue to use Webhook mode until the WebSocket connection is fully established and confirmed to be available.
[0112] Since the device 300 is a device corresponding to the communication mode dynamic decision-making method based on multi-objective optimization and online learning provided in the above method embodiments, the specific implementation of each unit of the device 300 is based on the same concept as in the above method embodiments. Therefore, for the specific implementation of each unit of the device 300, please refer to the description of the communication mode dynamic decision-making based on multi-objective optimization and online learning in the above method embodiments, which will not be repeated here.
[0113] This invention also provides an electronic device, the device comprising: a processor and a memory; The memory is used to store instructions; The processor is configured to execute the instructions in the memory and perform the communication mode dynamic decision-making method based on multi-objective optimization and online learning mentioned in the above embodiments.
[0114] It should be noted that the hardware structure of the devices provided in the embodiments of the present invention can all be as follows. Figure 4 The structure shown, Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.
[0115] Please see Figure 4 As shown, device 400 includes: processor 410, communication interface 420, and memory 430. The number of processors 410 in device 400 can be one or more. Figure 4 Taking a processor as an example, in this embodiment of the invention, the processor 410, communication interface 420, and memory 430 can be connected via a bus system or other means. Figure 4 Taking the connection between China and Israel via the bus system 440 as an example.
[0116] Processor 410 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. Processor 410 may further include hardware chips. These hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.
[0117] The memory 430 may include volatile memory, such as random-access memory (RAM); the memory 430 may also include non-volatile memory, such as flash memory, hard disk drive (HDD) or solid-state drive (SSD); the memory 430 may also include a combination of the above types of memory.
[0118] Optionally, the memory 430 stores an operating system and programs, executable modules, or data structures, or subsets thereof, or extended sets thereof. The programs may include various operation instructions for implementing various operations. The operating system may include various system programs for implementing various basic services and handling hardware-based tasks. The processor 410 can read the programs from the memory 430 to implement the dynamic decision-making method for communication modes based on multi-objective optimization and online learning provided in this embodiment of the invention.
[0119] The bus system 440 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus system 440 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0120] This invention also provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the communication mode dynamic decision-making method based on multi-objective optimization and online learning mentioned in the above embodiments.
[0121] This invention also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the dynamic decision-making method for communication modes based on multi-objective optimization and online learning mentioned in the above embodiments.
[0122] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.
Claims
1. A dynamic decision-making method for communication modes based on multi-objective optimization and online learning, characterized in that, Includes the following steps: The system's operating parameters are collected in real time, and a multi-dimensional state vector is constructed based on these parameters. Based on historical performance data, and using a time series prediction model, we output the performance prediction values of WebSocket long connection mode and Webhook on-demand reporting mode in future decision-making cycles. Based on the multidimensional state vector and the performance prediction value, a communication mode selection action is generated by calculating a multi-objective reward function. Execute the communication mode selection action. If the action indicates mode switching, the switching is completed while ensuring that no data is lost. Collect actual system performance data after the action is executed, calculate the reward value based on the actual system performance data, and update the decision-making strategy according to the reward system.
2. The method according to claim 1, characterized in that, The system collects operating parameters in real time and constructs a multi-dimensional state vector based on the operating parameters. The multi-dimensional state vector includes service load characteristics, network environment indicators, resource cost measurements, and historical communication patterns. The instantaneous data arrival rate and average data packet size are collected to characterize the service load characteristics; Network latency and packet loss rate are collected and calculated together to form the network condition index. The resource consumption of the server is collected to quantify the maintenance cost of the WebSocket connection and the processing cost of a single Webhook request, respectively, to obtain the resource cost metric. Record the abnormal disconnection rate of WebSocket connections to assess their connection health, and introduce the business priority weight defined by business logic, as well as the communication mode used in the previous moment, to obtain the historical communication mode.
3. The method according to claim 1, characterized in that, The time series prediction model is used to output performance prediction values for WebSocket long-connection mode and Webhook on-demand reporting mode in future decision-making cycles, including The time series prediction model is used to perform rolling predictions on the connection maintenance cost and average transmission latency of WebSocket mode and the single request cost and average transmission latency of Webhook mode, respectively, and outputs the performance prediction values of WebSocket long connection mode and Webhook on-demand reporting mode in future decision-making cycles.
4. The method according to claim 1, characterized in that, The calculation of the multi-objective reward function includes: The multi-objective reward function is constructed as a weighted sum of a cost term, a latency term, and a reliability penalty term. The calculation of the cost term incorporates the performance prediction value and the mode switching overhead. The calculation of the latency term incorporates the performance prediction value and the business priority. The reliability penalty term is activated when the WebSocket connection health is lower than a preset threshold.
5. The method according to claim 4, characterized in that, The weight coefficients of the weighted sum are determined by the analytic hierarchy process, which includes: constructing a judgment matrix to characterize the relative importance of cost, delay and reliability, and calculating the eigenvectors of the judgment matrix to determine the weight coefficients of each objective.
6. The method as described in claim 1, characterized in that, In the step of generating communication mode selection actions, the Q-Learning algorithm is used as the decision core, and a hybrid exploration strategy is applied to select actions. The hybrid exploration strategy introduces an upper confidence bound term related to the number of times the action is selected to guide the exploration, based on the ε-greedy strategy.
7. The method as described in claim 1, characterized in that, If the action indication mode is switched, the switch will be completed while ensuring that no data is lost, including: When the decision action is to switch from Webhook mode to WebSocket mode, the data reporting will continue to use Webhook mode until the WebSocket connection is fully established and confirmed to be available.
8. A dynamic decision-making device based on multi-objective optimization and online learning communication modes, characterized in that, include: A state-aware unit is used to collect system operating parameters in real time and construct a multi-dimensional state vector based on the system operating parameters. The performance prediction unit is used to output the performance prediction values of WebSocket long connection mode and Webhook on-demand reporting mode in the future decision-making cycle based on historical performance data and time series prediction models. The decision generation unit is used to generate a communication mode selection action by calculating a multi-objective reward function based on the multi-dimensional state vector and the performance prediction value. The action execution unit is used to execute the communication mode selection action. If the action indicates a mode switch, the switch is completed while ensuring that no data is lost. The feedback learning unit is used to collect actual system performance data after the action is executed, calculate the reward value based on the actual system performance data, and update the decision-making strategy according to the reward system.
9. An electronic device, characterized in that, The device includes: a processor and a memory; The memory is used to store instructions; The processor is configured to execute the instructions in the memory to perform the method according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, Including instructions that, when run on a computer, cause the computer to perform the method described in any one of claims 1-7 above.