Cloud control system multi-scene grading signaling interaction and evaluation method and related equipment

Abstract

The invention provides a multi-scene hierarchical signaling interaction and evaluation method and related equipment for a cloud control system, and the method comprises the steps: segmenting different signaling interaction services in the cloud control system according to multiple scenes, and defining a plurality of levels of network control modes; the method comprises the following steps: applying a sliding window algorithm and a Markov decision process to automatic switching of multi-level network control modes under a signaling interaction service, and determining an automatic switching strategy which should be adopted by each network control mode; collecting real-time monitoring data of a plurality of latest continuous windows in a current network state, and automatically executing an automatic switching strategy according to the real-time monitoring data; and carrying out real-time monitoring on performance indexes in the signaling interaction process so as to carry out quantitative evaluation on the graded cloud control signaling interaction result. According to the method, the system can automatically execute the optimal automatic switching strategy under different network states and workloads, the communication performance maximization and the stable operation of the system in the signaling interaction process are ensured, and the robustness of the multilevel signaling interaction of the cloud control system is improved.

Description

Technical Field

[0001] The present invention relates to the field of network communication technology, and in particular to a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system and related equipment. Background Art

[0002] The cloud control platform is an intelligent transportation system that integrates vehicles, roads and cloud computing. Through the vehicle-mounted platform, roadside sensors and other equipment, real-time data of vehicles and roads are collected, and these data are uploaded to the cloud control platform for processing and analysis, so as to realize the sharing of traffic information and traffic management, and the intelligence of networked vehicles. The cloud control platform can realize functions such as real-time traffic monitoring, intelligent road condition prediction, intelligent scheduling and route planning, which can improve traffic safety, reduce congestion, and also provide support for the development of urban traffic planning and intelligent transportation systems.

[0003] In the context of autonomous driving based on a cloud control platform, the control link of the cloud control system is often interfered by factors such as unstable network latency and network packet loss, resulting in delayed or failed signaling interactions, and the inability to guarantee the effectiveness of interactions in various business scenarios. In order to ensure the effectiveness of signaling interactions in various business scenarios, a set of related methods is needed to plan various scenarios and corresponding actions to ensure the continuous and stable operation of the system. Summary of the invention

[0004] The present invention provides a multi-scenario hierarchical signaling interaction and evaluation method and related equipment for a cloud control system, aiming to solve the problems existing in the prior art.

[0005] The purpose of the present invention is achieved by the following technical solutions:

[0006] In a first aspect, the present invention provides a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system, comprising:

[0007] Different signaling interaction services in the cloud control system are divided into multiple scenarios, and multiple levels of network control modes under the signaling interaction services are defined;

[0008] Applying a sliding window algorithm and a Markov decision process to the automatic switching of multiple levels of network control modes under the signaling interaction service to determine the automatic switching strategy that each of the network control modes should adopt;

[0009] Collecting the real-time monitoring data of the most recent continuous multiple windows under the current network state, the cloud control system automatically executes the automatic switching strategy according to the real-time monitoring data, wherein the real-time monitoring data includes: the packet loss rate and delay time of the heartbeat signal;

[0010] The performance indicators of the cloud control system adopting the automatic switching strategy during the signaling interaction process are monitored in real time to quantitatively evaluate the hierarchical cloud control signaling interaction results.

[0011] According to a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system provided by the present invention, the step of applying a sliding window algorithm and a Markov decision process to the automatic switching of multiple levels of network control modes under the signaling interaction service and determining the automatic switching strategy to be adopted by each of the network control modes includes:

[0012] Modeling the automatic switching strategy problem of multi-level network control modes using the Markov decision process;

[0013] Using the network control modes at multiple levels as the state set of the Markov decision process, and using the control instructions for switching the network control modes as the action set of the Markov decision process;

[0014] Use the sliding window algorithm to collect state transition data within a period of time, and count the state transition probabilities of transferring to the corresponding state after performing different actions in different states;

[0015] According to the state transition probability and in combination with the reward function defined for each state, a value iteration algorithm is used to calculate the optimal automatic switching strategy for each of the network control modes.

[0016] According to a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system provided by the present invention, the step of calculating the optimal automatic switching strategy for each network control mode using a value iteration algorithm based on the state transition probability and in combination with a reward function defined for each state includes:

[0017] Establish a state value function and initialize the initial value of each state, wherein the state value function represents the expected reward value after performing an action for a given state;

[0018] For each state of the state set, using the state transition probability to predict the next predicted state corresponding to executing each action from the action set;

[0019] Calculate the expected reward value of each predicted state in combination with the reward function, select the action with the maximum expected reward value through the ε-greedy strategy, and update the state value function;

[0020] The iterative process is repeated until the state value function converges, and the optimal action for each state is determined according to the updated state value function to serve as the optimal automatic switching strategy for each of the network control modes.

[0021] According to a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system provided by the present invention, the action set includes: an upgrade switching instruction, a downgrade switching instruction, a keep unchanged instruction and a same-level switching instruction.

[0022] According to a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system provided by the present invention, the performance indicators include: response speed, control accuracy, stability and reliability, wherein:

[0023] The response speed is the average time from sending control signaling to receiving the response;

[0024] Control accuracy is the deviation between the expected response and the actual response.

[0025] Stability is the inverse frequency of the system state changes;

[0026] Reliability is the proportion of instructions that the system successfully executes.

[0027] According to a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system provided by the present invention, after the step of real-time monitoring of the performance indicators of the cloud control system adopting the automatic switching strategy in the signaling interaction process to quantitatively evaluate the hierarchical cloud control signaling interaction results, the method further includes: in response to changes in each of the performance indicators, the cloud control system automatically adjusts the network performance, specifically including:

[0028] In response to the slow response speed, the cloud control system automatically reduces or shuts down non-critical functions to reduce unnecessary signaling interactions;

[0029] In response to the control accuracy being reduced, the cloud control system automatically improves the signaling confirmation mechanism and increases redundancy;

[0030] In response to the stability drop, the cloud control system automatically adds a state filtering mechanism to reduce frequent strategy switching;

[0031] In response to the reliability degradation, the cloud control system automatically adds a fault detection and recovery mechanism to improve system redundancy.

[0032] According to a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system provided by the present invention, after the step of automatically adjusting network performance in the cloud control system, the method further includes:

[0033] According to the latest acquired performance indicator, the state transition probability and reward function used in the Markov decision process are updated using a sliding window algorithm to dynamically adjust the automatic switching strategy.

[0034] In a second aspect, the present invention provides an electronic device, the electronic device comprising:

[0035] one or more processors;

[0036] And a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the multi-scenario hierarchical signaling interaction and evaluation method of the cloud control system as described in any of the above-mentioned methods.

[0037] In a third aspect, the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system as described in any one of the above.

[0038] In a fourth aspect, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the multi-scenario hierarchical signaling interaction and evaluation method of a cloud control system as described in any one of the above-mentioned methods.

[0039] The present invention provides a multi-scenario hierarchical signaling interaction and evaluation method and related equipment for a cloud control system. By dividing different signaling interaction services in the cloud control system according to multiple scenarios, multiple levels of network control modes under the signaling interaction services are defined; a sliding window algorithm and a Markov decision process are applied to the automatic switching of multiple levels of network control modes under the signaling interaction services, and an automatic switching strategy to be adopted for each network control mode is determined; real-time monitoring data of multiple recent consecutive windows under the current network state are collected, and the cloud control system automatically executes the automatic switching strategy according to the real-time monitoring data, wherein the real-time monitoring data includes: the packet loss rate and delay time of the heartbeat signal; the performance indicators of the cloud control system adopting the automatic switching strategy in the signaling interaction process are monitored in real time, so as to quantitatively evaluate the hierarchical cloud control signaling interaction results. The present invention combines the actual state space in the signaling interaction process of the cloud control system, obtains multiple levels of network control modes through multi-scenario segmentation, and determines the automatic switching strategy of each network control mode in combination with the sliding window algorithm and the Markov decision process to ensure that the system can automatically determine the optimal automatic switching strategy to be adopted under different network states and workloads, ensure the maximization of communication performance and the stable operation of the system during the signaling interaction process, and improve the robustness of the multi-level signaling interaction of the cloud control system; and through the monitoring of performance indicators in the signaling interaction process, it provides quantitative evaluation performance indicators for the daily operation of the system, and provides a scientific basis for system design and improvement. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other implementation methods can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 It is a flowchart of a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system provided by an embodiment of the present invention;

[0042] Figure 2 is a schematic diagram of switching between multi-level network control modes provided by an embodiment of the present invention;

[0043] Figure 3 It is a schematic diagram of a simulation environment for simulating multi-scenario hierarchical signaling interaction provided by an embodiment of the present invention;

[0044] Figure 4 It is a schematic diagram of the change of each state of a multi-level network control mode over time in a simulation environment provided by an embodiment of the present invention;

[0045] Figure 5 is a schematic diagram of the state transition probability between states in a Markov decision process drawn according to simulation results provided by an embodiment of the present invention;

[0046] Figure 6 It is a comparison chart of the performance indicators monitored by using the hierarchical signaling interaction method for up-down switching and the performance indicators monitored without using the up-down switching provided by the embodiment of the present invention;

[0047] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0048] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following specifically cites a preferred embodiment and describes it in detail with the accompanying drawings as follows.

[0049] It should be noted that it is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in the present invention can be combined with other embodiments without conflict. Unless otherwise defined, the technical terms or scientific terms involved in the present invention should be the usual meanings understood by people with ordinary skills in the technical field to which the present invention belongs. The words "one", "a", "a", "the" and the like involved in the present invention do not indicate a quantitative limitation and may represent the singular or plural. The terms "including", "comprising", "having" and any of their variations involved in the present invention are intended to cover non-exclusive inclusions; the terms "first", "second", "third", etc. involved in the present invention are merely to distinguish similar objects and do not represent a specific ordering of objects.

[0050] When it comes to method steps, the sequence illustrated in this document represents an exemplary solution, but does not mean a limitation on the sequence.

[0051] Reference Figure 1 As shown, an embodiment of the present invention provides a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system, including:

[0052] Step S1: Different signaling interaction services in the cloud control system are divided into multiple scenarios, and multiple levels of network control modes under the signaling interaction services are defined.

[0053] In this step, for different signaling interaction services of the cloud control system, in order to illustrate that different signaling interaction services are divided according to multiple scenarios, this embodiment selects two dimensions of packet loss rate and delay, and explains from the perspective of high and low packet loss rate and high and low delay. The high and low packet loss rate and high and low delay are combined, and four scenarios are used as the basis for multi-scenario segmentation. Combined with specific services, the network control mode is defined for the four scenarios respectively:

[0054] 1) Low packet loss and low latency (LL network control mode): When communication conditions are good, the system should remain fully functional to achieve optimal performance.

[0055] 2) Low packet loss, high latency (LH network control mode): When faced with slight delays, the system may reduce certain non-critical performance to ensure the continuity of core operations.

[0056] 3) High packet loss, low latency (HL network control mode): Even in the case of high packet loss rate, if the latency remains within an acceptable range, the system should prioritize the most critical tasks.

[0057] 4) High packet loss, high latency (HH network control mode): In the case of severe communication restrictions, the system enters a safe mode to ensure the most basic operation and security.

[0058] Step S2: Apply the sliding window algorithm and Markov decision process to the automatic switching of multiple levels of network control modes under the signaling interaction service, and determine the automatic switching strategy that each network control mode should adopt.

[0059] In this embodiment, step S2 specifically includes:

[0060] Step S21: Modeling the automatic switching strategy problem of multi-level network control mode using Markov decision process;

[0061] Step S22: using the multiple levels of network control modes as a state set of a Markov decision process, and using the control instructions for switching the network control modes as an action set of the Markov decision process;

[0062] Step S23: using a sliding window algorithm to collect state transition data within a period of time, and to count the state transition probabilities of transferring to corresponding states after performing different actions in different states;

[0063] Step S24: Based on the state transition probability and the reward function defined for each state, a value iteration algorithm is used to calculate the optimal automatic switching strategy for each network control mode.

[0064] In this embodiment, step S24 specifically includes:

[0065] Step S241: Establish a state value function and initialize the initial value of each state, wherein the state value function represents the expected reward value after performing an action for a given state;

[0066] Step S242: for each state in the state set, using the state transition probability to predict the next predicted state corresponding to executing each action in the action set;

[0067] Step S243: Calculate the expected reward value of each predicted state in combination with the reward function, select the action with the largest expected reward value through the ε-greedy strategy, and update the state value function;

[0068] Step S244: Repeat the iterative process until the state value function converges, and determine the optimal action for each state based on the updated state value function as the optimal automatic switching strategy for each network control mode.

[0069] Specifically, the Markov decision process can be represented by a quintuple (S, A, P(s′|s, a), R, γ).

[0070] Among them, S is a state set, including four network control modes: LL (low packet loss, low latency), LH (low packet loss, high latency), HL (high packet loss, low latency), and HH (high packet loss, high latency).

[0071] A is an action set, including: upgrade switching instructions, downgrade switching instructions, keep unchanged instructions and same-level switching instructions.

[0072] For s∈S, s'∈S, P(s'|s, a) is the state transition probability of the state from s to s' after action a∈A is executed. For example, if there is a 70% probability of transitioning to the state of LH network control mode after executing the downgrade switch instruction action in the state of LL network control mode, then P(LH|LL, downgrade switch instruction) = 0.7. For each state in the state set, the sum of its transition probabilities to other states is 1, which is expressed as:

[0073] R(s, a) is the reward obtained after executing action a in state s. The reward is divided into immediate reward and cumulative reward. According to the system goal, the reward value of each state and action combination is defined. If certain actions (such as upgrade instructions or downgrade instructions) can improve system performance, these actions are given higher reward values. Usually, a higher reward will be set when entering a good communication environment (such as LL network control mode), while a lower reward will be set when entering a poor communication environment (such as HH network control mode).

[0074] γ represents a discount factor, which is used to balance the importance of current rewards and future rewards. In the above representation of calculating cumulative rewards, γ∈[0,1] is equivalent to discounting future rewards. If γ approaches 0, only the current immediate reward is concerned, and actions that maximize the current immediate reward are often performed, which is essentially a greedy behavior; if γ approaches 1, more consideration will be given to future rewards.

[0075] The steps of Markov decision process training automatic switching strategy are as follows:

[0076] (1) Establish a five-tuple of Markov decision process (S, A, P(s′|s, a), R, γ);

[0077] (2) Establish a state value function V and set the initial value of each state V(s) = 0;

[0078] (3) randomly selecting an initial network control mode S1 from the state set S, for example, selecting the LL network control mode as the initial state;

[0079] (4) Based on the ε-greedy strategy, action A1 is selected for learning in state S1. After learning A1, the next state S2 can be observed according to the environment, and the immediate reward R2 is obtained. At this time, the maximum Q value in the current state is selected to update the V function:

[0080]

[0081] (5) Repeat step (4) and use the value iteration algorithm to repeat the iteration process until the state value function converges and reaches the termination state, that is, the value change of the state is less than the preset threshold. Repeat step (3), select other states as the initial state to start learning, and determine the optimal action for each state, that is, the optimal automatic switching strategy (strategy π represents the optimal action to be taken in each state) according to the state value function V after the iteration is terminated (indicates the expected total reward obtained by acting according to the automatic switching strategy π starting from state s), as shown below:

[0082]

[0083] (6) Store the optimal strategy for each state in the form of a dictionary.

[0084] Among them, the specific steps of the ε-greedy strategy are as follows:

[0085] (1) Specify the value of ε∈[0,1] and generate a random number between 0 and 1;

[0086] (2) If the random number is less than ε, then randomly select the action that can be selected in the current state for learning (the probability of each action being selected is ε / n, where n is the number of actions that can be selected in the current state);

[0087] (3) If the random number is greater than or equal to ε, the action with the maximum state-action Q value in the current state is selected for learning.

[0088] Through the above steps, the optimal automatic switching strategy π is calculated based on the sliding window algorithm and the Markov decision process. The cloud control system automatically determines the optimal control strategy to be adopted under different network conditions, so that it can perform corresponding actions in each state and decide when the system should be upgraded or downgraded. Based on the probability transfer estimation, the action preparation of related resources can be made in advance, making the switching process smoother and ensuring the maximization of communication performance and the stable operation of the system.

[0089] Step S3: Collect the real-time monitoring data of multiple consecutive windows in the current network state, and the cloud control system automatically executes the automatic switching strategy according to the real-time monitoring data, wherein the real-time monitoring data includes: the packet loss rate and delay time of the heartbeat signal.

[0090] The sliding window algorithm is used to collect the real-time monitoring data of the most recent continuous windows under the current network status, such as the success rate and delay time of the heartbeat signal, to evaluate the current network status as the input of the system's automatic switching strategy control logic. Based on the real-time monitoring data, the system can automatically decide to downgrade to adapt to the poor communication environment, or automatically upgrade to restore signaling interaction performance when the communication environment improves.

[0091] like Figure 2 As shown, the LL network control mode is used as the initial state of the signaling interaction process for explanation. For example, when the current network state is monitored to be an increase in delay, the system sends a downgrade switching instruction to automatically switch the LL network control mode to the LH network control mode; when the current network state is monitored to be an increase in packet loss rate, the system sends a downgrade switching instruction to automatically switch the LL network control mode to the LH network control mode; when the current network state is monitored to be an increase in delay and an increase in packet loss rate, the system sends a downgrade switching instruction to automatically switch the LL network control mode to the HH network control mode.

[0092] When switching to the LH network control mode, the success rate and delay time of the heartbeat signal are checked regularly. When it is detected that the current network status is a delay reduction, the system sends an upgrade switching command to automatically switch the LH network control mode to the LL network control mode; when it is detected that the current network status is an increase in packet loss rate and a decrease in delay, the system sends a same-level switching command to automatically switch the LH network control mode to the HL network control mode; when it is detected that the current network status is an increase in packet loss rate, the system sends a downgrade switching command to automatically switch the LH network control mode to the HH network control mode.

[0093] When switching to the HL network control mode, the success rate and delay time of the heartbeat signal are checked regularly. When it is detected that the current network status is a decrease in packet loss rate, the system sends an upgrade switching instruction to automatically switch the HL network control mode to the LL network control mode; when it is detected that the current network status is an increase in delay and a decrease in packet loss rate, the system sends a same-level switching instruction to automatically switch the HL network control mode to the LH network control mode; when it is detected that the current network status is an increase in delay, the system sends a downgrade switching instruction to automatically switch the HL network control mode to the HH network control mode.

[0094] When switching to the HH network control mode, the success rate and delay time of the heartbeat signal are checked regularly. When it is detected that the current network status is a reduced packet loss rate, the system sends an upgrade switching instruction to automatically switch the HH network control mode to the LH network control mode; when it is detected that the current network status is a reduced delay, the system sends an upgrade switching instruction to automatically switch the HH network control mode to the HL network control mode; when it is detected that the current network status is a reduced packet loss rate and a reduced delay, the system sends an upgrade switching instruction to automatically switch the HH network control mode to the LL network control mode.

[0095] Furthermore, this embodiment also sets emergency response measures, such as switching to manual control, to ensure that signaling interaction control is not completely lost even under the most unfavorable conditions.

[0096] Step S4: Real-time monitoring of the performance indicators of the cloud control system adopting the automatic switching strategy during the signaling interaction process is performed to quantitatively evaluate the hierarchical cloud control signaling interaction results.

[0097] In this step, the design of the evaluation method is crucial to ensure that the system can operate effectively. This evaluation method usually involves the calculation of multiple performance indicators, including: response speed, control accuracy, stability and reliability.

[0098] Response speed is a key indicator to measure the speed of system response. It can be used to evaluate the average time from sending control signaling to receiving a response. The mathematical representation of response speed is usually the average value of response time, and its calculation formula is:

[0099]

[0100] Where N is the number of interactions of the sliding window, t i is the i-th response time.

[0101] Control accuracy measures the accuracy of the system's execution of instructions. In the signaling interaction process of the cloud control system, control accuracy can be quantified by comparing the deviation between the expected response and the actual response. The commonly used mathematical expression of control accuracy is the mean square error (MSE), which is calculated as follows:

[0102]

[0103] Where N is the number of interactions of the sliding window, x i is the actual response of the ith time, is the expected response for the ith time.

[0104] Stability describes the ability of a system to maintain its performance in the face of external disturbances. Mathematically, stability can be measured by calculating the frequency of system state changes. If the number of state changes is defined as Δs, then stability can be expressed as the inverse frequency of state changes, and its calculation formula is:

[0105]

[0106] Where N is the number of interactions of the sliding window and ΔS is the number of state changes.

[0107] Reliability measures the ability of a system to run for a long time without failure. Mathematically, reliability is often defined by the proportion of instructions successfully executed by the system, and its calculation formula is:

[0108]

[0109] Where N is the number of interactions of the sliding window.

[0110] The above performance indicators not only provide quantitative evaluation data for the hierarchical cloud-control signaling interaction results, but also provide a scientific basis for system design and improvement.

[0111] By monitoring these performance indicators in real time, the cloud control system can automatically adjust performance in response to changes in various performance indicators to ensure that the best operating status can be maintained in various operating environments. Specifically:

[0112] In response to the slower response speed, indicating an increase in network latency, the cloud control system automatically reduces or shuts down non-critical functions to reduce unnecessary signaling interactions.

[0113] In response to the reduction in control accuracy, indicating that the accuracy of the system's execution of instructions has decreased, the cloud control system automatically improves the signaling confirmation mechanism and increases redundancy.

[0114] In response to the decrease in stability, indicating that the system state changes frequently, the cloud control system automatically adds a state filtering mechanism to reduce frequent strategy switching.

[0115] In response to the decrease in reliability, indicating an increase in the frequency of system failures, the cloud control system automatically adds fault detection and recovery mechanisms to improve system redundancy.

[0116] In this embodiment, after the cloud control system automatically performs the step of adjusting the network performance, the following steps are also included:

[0117] According to the latest performance indicators, the sliding window algorithm is used to update the state transition probability and reward function used in the Markov decision process to achieve dynamic adjustment of the automatic switching strategy.

[0118] Therefore, the present invention provides a multi-scenario hierarchical signaling interaction and evaluation method for a cloud control system. By dividing different signaling interaction services in the cloud control system according to multiple scenarios, multiple levels of network control modes under the signaling interaction services are defined; a sliding window algorithm and a Markov decision process are applied to the automatic switching of multiple levels of network control modes under the signaling interaction services, and an automatic switching strategy to be adopted for each network control mode is determined; real-time monitoring data of multiple consecutive windows recently under the current network state are collected, and the cloud control system automatically executes the automatic switching strategy according to the real-time monitoring data, wherein the real-time monitoring data includes: the packet loss rate and delay time of the heartbeat signal; the performance indicators of the cloud control system adopting the automatic switching strategy in the signaling interaction process are monitored in real time, so as to quantitatively evaluate the hierarchical cloud control signaling interaction results. The present invention combines the actual state space in the signaling interaction process of the cloud control system, obtains multiple levels of network control modes through multi-scenario segmentation, and determines the automatic switching strategy of each network control mode in combination with the sliding window algorithm and the Markov decision process to ensure that the system can automatically determine the optimal automatic switching strategy to be adopted under different network states and workloads, ensure the maximization of communication performance and the stable operation of the system during the signaling interaction process, and improve the robustness of the multi-level signaling interaction of the cloud control system; and through the monitoring of performance indicators in the signaling interaction process, it provides quantitative evaluation performance indicators for the daily operation of the system, and provides a scientific basis for system design and improvement.

[0119] For the signaling interaction process of the cloud control system using the automatic switching strategy, a simulation environment is constructed on a PC. The system is simulated with a time step of 100,000 and the control delay threshold is set to 0.1s. During the 100,000-step simulation process, the control delay changes with packet loss and time as shown in the following figure. Figure 3 The multi-scenario hierarchical signaling interaction method provided in the embodiment of the present invention is applied to schedule multiple levels of network control modes, and the changes of the system status over time are shown in FIG. Figure 4 As shown. According to the simulation results, the state transition probability between states in the Markov decision process is plotted, as shown in Figure 5 shown.

[0120] Reference Figure 6As shown, Graded Method represents the performance indicators monitored by using the graded signaling interaction method provided by the embodiment of the present invention to perform up-down switching, and No Degradation Method represents the performance indicators monitored by the network state without up-down switching. After comparing the performance indicators of the two, we can draw the following conclusions. First, the system average response time (Average Response Time) of the up-down switching scheme is slightly higher than that of the scheme without up-down switching, which are 0.1001 seconds and 0.0921 seconds respectively. This shows that in terms of ensuring system stability and reliability, the up-down switching scheme may sacrifice some response speed. However, from the comparison of control accuracy, it can be seen that the control accuracy (ControlPrecision) of the up-down switching scheme is much higher than that of the scheme without up-down switching, which are 0.9920 and 0.0079 respectively. This shows that the up-down switching scheme can maintain high operation accuracy and efficiency when the network fluctuates or other unstable factors exist.

[0121] Although the up-down and down-shift switching scheme is slightly lower than the non-up-down and down-shift switching scheme in terms of stability, at 0.7883 and 0.8170 respectively, the difference is not significant, which shows the effectiveness of the up-down and down-shift switching scheme in maintaining system stability. The most striking thing is that in terms of reliability, the up-down and down-shift switching scheme is significantly better than the non-up-down and down-shift switching scheme, with reliability indicators of 0.8991 and 0.4457 respectively. This significant advantage shows that the up-down and down-shift switching scheme is more efficient in dealing with factors that may cause system failures and can significantly improve the overall reliability of the system.

[0122] In general, although the up-down and down-shift switching scheme has a slight disadvantage in response time, its significant advantages in control accuracy and reliability prove its value, especially in the face of network instability and other external challenges. The up-down and down-shift automatic switching strategy can ensure the continuous and stable operation of the system.

[0123] In addition, some embodiments of the present application also provide an electronic device. The electronic device may be a digital computer in various forms, such as a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, etc. The electronic device may also be a mobile device in various forms, such as a personal digital processing, a cellular phone, a smart phone, a wearable device, and other similar computing devices.

[0124] The electronic device includes: one or more processors; and a memory storing computer program instructions, and when the computer program instructions are executed, the processor performs the steps of the cloud control system multi-scenario hierarchical signaling interaction and evaluation method provided in any one or more of the above embodiments. Figure 7An exemplary structural diagram of the electronic device is disclosed. Figure 7 As shown, the electronic device includes: one or more processors 1101, memory 1102, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Among them, the components shown in this article, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0125] The electronic device may further include: an input device 1103 and an output device 1104. The processor 1101, the memory 1102, the input device 1103 and the output device 1104 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0126] The input device 1103 can receive input digital or character information, and generate key signal input related to the user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 1104 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0127] To provide interaction with a user, the electronic device may be a computer. The computer has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0128] In an embodiment of the present application, a computer program / instruction is stored on a computer-readable medium, and when the computer program / instruction is executed by a processor, the steps of the multi-scenario hierarchical signaling interaction and evaluation method of the cloud control system provided by any one or more of the above embodiments are implemented. The computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist alone without being assembled into the device. The above computer-readable medium carries one or more computer-readable instructions.

[0129] The memory 1102 can be used as a non-transient computer-readable storage medium, which can be used to store non-transient software programs, non-transient computer executable programs and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 1102, so as to implement the program instructions / modules corresponding to the method provided by any one or more embodiments in the embodiments of the present application.

[0130] The memory 1102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1102 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 1102 may optionally include a memory remotely arranged relative to the processor 1101, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0131] It should be noted that more specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0132] Computer readable storage media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0133] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] In the above-described embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. For example, it can be implemented by using an application specific integrated circuit (ASIC), a general-purpose computer or any other similar hardware device. In certain embodiments, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including relevant data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented by hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0135] The computer program product provided in the embodiment of the present application includes one or more computer programs / instructions, and when the computer program / instructions are executed by the processor, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0136] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0137] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily mention changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims, and the above embodiments should be regarded as exemplary and non-restrictive.

Claims

1. A multi-scenario hierarchical signaling interaction and evaluation method for cloud control system, It is characterized in that include: Different signaling interaction services in the cloud control system are divided into multiple scenarios, and multiple levels of network control modes under the signaling interaction services are defined; Applying a sliding window algorithm and a Markov decision process to the automatic switching of multiple levels of network control modes under the signaling interaction service to determine the automatic switching strategy that each of the network control modes should adopt; Collecting the real-time monitoring data of the most recent continuous multiple windows under the current network state, the cloud control system automatically executes the automatic switching strategy according to the real-time monitoring data, wherein the real-time monitoring data includes: the packet loss rate and delay time of the heartbeat signal; The performance indicators of the cloud control system adopting the automatic switching strategy during the signaling interaction process are monitored in real time to quantitatively evaluate the hierarchical cloud control signaling interaction results.

2. According to the multi-scenario hierarchical signaling interaction and evaluation method of the cloud control system in claim 1, It is characterized in that The step of applying the sliding window algorithm and the Markov decision process to the automatic switching of multiple levels of network control modes under the signaling interaction service and determining the automatic switching strategy to be adopted by each of the network control modes includes: Modeling the automatic switching strategy problem of multi-level network control modes using the Markov decision process; Using the network control modes at multiple levels as the state set of the Markov decision process, and using the control instructions for switching the network control modes as the action set of the Markov decision process; Use the sliding window algorithm to collect state transition data within a period of time, and count the state transition probabilities of transferring to the corresponding state after performing different actions in different states; According to the state transition probability and in combination with the reward function defined for each state, a value iteration algorithm is used to calculate the optimal automatic switching strategy for each of the network control modes.

3. According to the multi-scenario hierarchical signaling interaction and evaluation method of the cloud control system in claim 2, It is characterized in that The step of calculating the optimal automatic switching strategy for each network control mode using a value iteration algorithm based on the state transition probability and in combination with a reward function defined for each state includes: Establish a state value function and initialize the initial value of each state, wherein the state value function represents the expected reward value after performing an action for a given state; For each state of the state set, using the state transition probability to predict the next predicted state corresponding to executing each action from the action set; Calculate the expected reward value of each predicted state in combination with the reward function, select the action with the maximum expected reward value through the ε-greedy strategy, and update the state value function; The iterative process is repeated until the state value function converges, and the optimal action for each state is determined according to the updated state value function to serve as the optimal automatic switching strategy for each of the network control modes.

4. According to claim 2, the multi-scenario hierarchical signaling interaction and evaluation method of the cloud control system, It is characterized in that The action set includes: an upgrade switching instruction, a downgrade switching instruction, a keep unchanged instruction and a same-level switching instruction.

5. According to claim 1, the multi-scenario hierarchical signaling interaction and evaluation method of the cloud control system, It is characterized in that The performance indicators include: response speed, control accuracy, stability and reliability, among which, The response speed is the average time from sending control signaling to receiving the response; Control accuracy is the deviation between the expected response and the actual response. Stability is the inverse frequency of the system state changes; Reliability is the proportion of instructions that the system successfully executes.

6. According to claim 5, the multi-scenario hierarchical signaling interaction and evaluation method of the cloud control system, It is characterized in that After the step of real-time monitoring of the performance indicators of the cloud control system adopting the automatic switching strategy during the signaling interaction process to quantitatively evaluate the hierarchical cloud control signaling interaction results, the method further includes: in response to changes in each of the performance indicators, the cloud control system automatically adjusts the network performance, specifically including: In response to the slow response speed, the cloud control system automatically reduces or shuts down non-critical functions to reduce unnecessary signaling interactions; In response to the control accuracy being reduced, the cloud control system automatically improves the signaling confirmation mechanism and increases redundancy; In response to the stability drop, the cloud control system automatically adds a state filtering mechanism to reduce frequent strategy switching; In response to the reliability degradation, the cloud control system automatically adds a fault detection and recovery mechanism to improve system redundancy.

7. According to claim 6, the multi-scenario hierarchical signaling interaction and evaluation method of the cloud control system, It is characterized in that After the step of automatically adjusting the network performance by the cloud control system, the method further includes: According to the latest acquired performance indicator, the state transition probability and reward function used in the Markov decision process are updated using a sliding window algorithm to dynamically adjust the automatic switching strategy.

8. An electronic device, It is characterized in that The electronic device comprises: one or more processors; And a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the multi-scenario hierarchical signaling interaction and evaluation method of the cloud control system as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program / instructions stored thereon, It is characterized in that When the computer program / instructions are executed by the processor, the steps of the cloud control system multi-scenario hierarchical signaling interaction and evaluation method as described in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, It is characterized in that When the computer program / instruction is executed by the processor, the steps of the cloud control system multi-scenario hierarchical signaling interaction and evaluation method as described in any one of claims 1 to 7 are implemented.

Images