Service efficiency regulation and control strategy design method for group intelligence

Abstract

The invention discloses a group intelligence-oriented service efficiency regulation and control strategy design method, and aims at optimizing behaviors of crowd intelligence groups in service ecology and an intervention strategy of a regulation and control algorithm at the same time. According to the method, through double-layer deep reinforcement learning, a crowd intelligence group and a regulation and control algorithm are promoted to interact and become an opposite environment, and a dynamic game state is formed. Dynamic balance is achieved through autonomous evolution and a cyclic feedback mechanism. Through continuous interactive learning between the two, the dynamic adjustment ability of the strategy is enhanced, and the learning ability and mode of the intelligent agent in the artificial society are optimized. In addition, the method not only can reproduce the historical state of the service ecology, but also can construct the future service ecology state by setting diversified configurations.

Description

Technical Field

[0001] The present invention relates to the fields of complex systems and computer technology, and more specifically, to a design method for a service efficiency control strategy oriented to swarm intelligence. Background Art

[0002] As a complex system influenced by multiple factors, the efficient operation of the service ecosystem requires a balanced approach: the service supply side constantly pursues high efficiency (efficiency = output / time spent) and the service demand side continuously provides valuable output (effectiveness = valuable output / time spent), preventing the entire system from falling into a vicious cycle of ineffective output. Regulation algorithms within the service ecosystem offer more sophisticated, precise, and comprehensive service operation models than traditional service systems, using a more autonomous and intelligent approach. For example, social media leverages recommendation algorithms to increase the speed at which people access information, credit institutions use credit scoring algorithms to determine loan amounts, and service platforms employ algorithms for route planning. Regulation algorithms play an increasingly important role in daily decision-making and processing, driving the optimization of service operation models and the development of the service ecosystem, leading to significant changes in service organizational models and operational approaches. However, due to inherent complexity, opacity, lack of interpretability, unpredictable behavior after interaction with intelligent agents, and a lack of effective human oversight, the use of regulation algorithms is increasingly risky. For example, platform algorithms have trapped food delivery drivers in a state of involution with diminishing marginal returns. Problems such as service network risks caused by intelligent control algorithms in the service ecosystem, information cocoons in service recommendations, and algorithmic discrimination in advertising delivery have posed new challenges to the performance analysis and optimization of the service ecosystem.

[0003] Currently, the literature on service ecosystem performance analysis in dynamic environments is fragmented and unsystematic, which limits in-depth and comprehensive assessments of the overall performance of service ecosystems. Existing literature primarily focuses on performance analysis in static environments, while performance analysis of service ecosystems in dynamically changing environments is relatively neglected. When analyzing the operational mechanisms of service ecosystems, researchers tend to employ quantitative analysis methods based on static data to analyze system performance. However, this approach fails to fully account for the complex interactions between agents in the service ecosystem and their dynamic nature. Consequently, this field lacks comprehensive analytical models addressing complexity and dynamics, hindering comprehensive, controllable, efficient, and reliable analysis. Furthermore, this also results in a lack of effective analytical tools for predictive research on potential risks introduced by control algorithms in specific application scenarios, making it difficult to thoroughly explore the negative phenomena that may be caused by intrinsic performance metrics in service ecosystems and identify appropriate intervention strategies.

[0004] Therefore, exploring how to systematically, quantitatively and efficiently analyze the performance of the service ecosystem, reveal the inherent mechanisms of negative phenomena caused by performance measurement, and formulate effective dynamic intervention strategies to optimize the performance of the service ecosystem has become a key research topic that needs to be addressed urgently. Summary of the Invention

[0005] In order to overcome the shortcomings of existing methods, the present invention provides a service efficiency regulation strategy design method for swarm intelligence, constructs an AI-driven service ecosystem in computational experiments, and provides a dynamic testing environment for various intervention strategies. This method uses deep reinforcement learning to train the intelligent agents and dynamic intervention strategy planners in the artificial society respectively, so that they can learn and evolve together in dynamic interactions. In addition, this method is repeatable, flexible and dynamic, and can dynamically provide appropriate intervention strategies according to different operating states, solving problems that static intervention strategies cannot solve. Next, this article will focus on how to improve the previous intervention strategies, the learning and evolution mechanism of intelligent agents in artificial societies, and the bilateral dynamic interaction process and evolutionary game mechanism.

[0006] To achieve the above-mentioned purpose, the technical solution of the present invention is a service efficiency control strategy design method for swarm intelligence, which is a flexible method for simulating the operating scenarios of the service ecology and optimizing its performance. In this method, two levels of deep reinforcement learning are instantiated, and the intelligent agents and dynamic intervention strategy planners in the artificial society are trained simultaneously. Through the architecture of two-layer deep reinforcement learning, the behavior of the intelligent agent and the intervention strategy of the planner can be optimized simultaneously, different application scenarios in the service ecology can be simulated, and the intervention strategy of the control algorithm can be optimized. This method allows the intelligent agent and the planner to adapt and optimize each other, thereby promoting the healthy development of the service ecology and improving the system performance. The specific steps are as follows:

[0007] 1) The overall architecture of the dynamic intervention strategy design method: The specific mechanisms and related connections between the planner operation mechanism, the agent operation mechanism, and the bilateral dynamic interaction process and evolutionary game mechanism are displayed in the form of an overall architecture diagram.

[0008] 2) The operating mechanism of intelligent agents in artificial societies: From the perspective of reinforcement learning, the optimization goals of numerous intelligent agents in artificial societies are given, and it is pointed out that the multi-agent environment here is trained in a shared weight manner.

[0009] 3) Operation mechanism of dynamic intervention strategy planner: The optimization goal of dynamic intervention planner is given from the perspective of reinforcement learning.

[0010] 4) Bilateral Dynamic Interaction Process and Evolutionary Game Mechanism: This paper presents the dynamic interaction and evolutionary game between intelligent agents and dynamic intervention planners in artificial societies from the perspective of reinforcement learning, and introduces how to conduct two-layer reinforcement learning training to obtain a stable solution space.

[0011] The beneficial effects of the present invention are as follows:

[0012] Compared with the prior art, the advantages of the present invention are:

[0013] 1) The present invention proposes a service efficiency regulation strategy design method for swarm intelligence, thereby establishing a service efficiency regulation strategy model with swarm intelligence.

[0014] 2) The technical solution proposed in this invention is general and allows for the migration of various swarm intelligence scenarios for modeling and design. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings;

[0016] Figure 1 This is the overall framework diagram of the service efficiency control strategy design method for swarm intelligence of the present invention;

[0017] Figure 2 It is an example of the change of service efficiency regulation strategy of swarm intelligence; DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments and drawings in the present invention, all other embodiments and drawings obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Those skilled in the art should understand that the content described specifically below is illustrative rather than restrictive, and should not be used to limit the scope of protection of the present invention.

[0019] The present invention provides a method for designing a service efficiency regulation strategy for swarm intelligence. The specific implementation process includes a planner operation mechanism, an intelligent agent operation mechanism, a bilateral dynamic interaction process, and an evolutionary game mechanism, thereby describing and explaining the negative phenomena caused by the deterioration of the service ecology.

[0020] Some optional implementations of this embodiment are as follows:

[0021] In artificial societies, agents use deep reinforcement learning to train their behavior policies to maximize their objective (the expected sum of future rewards). Each agent learns a value function that estimates this expectation based on observations. The agent learns by iteratively sampling from its current behavior policy and improving the model through training on empirical data. Deep reinforcement learning can optimize for any learning objective, relying on rewards generated by interactions with the environment and not requiring domain expertise. Furthermore, to improve data utilization, all agents in the service ecosystem share the same parameters during training. During the learning evolution process, when an agent learns a useful new behavior in a certain part of its state space, other agents in the service ecosystem can also learn this behavior. Furthermore, because agents have different observations and hidden states, their behaviors remain diverse. Generally speaking, in optimization problems, the aforementioned multi-agent behavior optimization problem is often formulated as a partially observable multi-agent Markov decision, with the optimal solution corresponding to an improvement on a Nash equilibrium. As long as no agent is willing to unilaterally deviate from its own policy, a set of Nash equilibria will emerge. To achieve improvements, rational non-equilibrium behaviors, including subgame perfect equilibrium, need to be considered. Although equilibrium is difficult to solve in complex computational environments, by leveraging two-layer deep reinforcement learning, it is still possible to achieve the co-evolution of learning between agents and planners in the service ecosystem (when the dynamic intervention strategy planner changes, the optimal behavior of many agents in the service ecosystem will also change), thereby promoting the performance of the service ecosystem.

[0022] This assumes a closed service ecosystem: total profit (i.e., distributable funds) is fixed, no funds leave the system, and the income structure of different agents redistributes as the planner's intervention strategy changes. Similar to the behavioral strategies of agents in artificial societies, the planner can use learned hidden states and the entire observed historical data to decide which intervention strategy to implement. The dynamic intervention strategy planner observes the agents' revenues and the global state of the service ecosystem, including their positions, the location of orders, and the competitive landscape in the region. The planner cannot directly observe the agents' capabilities or other intrinsic properties, such as the costs incurred and the rewards for completing orders. When formulating intervention strategies, the planner does not consider individual agents, but instead adopts a unified intervention strategy for all agents. However, the planner can fully learn from the complete historical observation data and learn from the hidden states to determine the intervention strategy to implement. In this way, the planner can select the appropriate intervention strategy based on the currently available information to optimize the performance of the entire service ecosystem. In the service ecosystem, agents must adapt to the intervention strategies set by the planner, and the planner must also adapt to the feedback from the agents. During this non-stationary learning evolution, the planner influences the agents' income by adjusting their intervention strategies, as these strategies alter the distribution function of service revenue, thereby affecting the agents' income. As intervention strategies change, the optimal strategies and behaviors of different agents also change. This instability exacerbates the challenging learning dynamics of mutual exploration, which is precisely the nested optimization problem that two-level reinforcement learning attempts to solve. The goal of this section is to find a set of dynamic intervention strategies that maximize the performance of the service ecosystem. Under this dynamic intervention strategy, the agents and planners will maximize their own objective functions. The dynamic intervention strategy planner (the outer layer) strives to maximize the performance of the service ecosystem, while the agents in the artificial society (the inner layer) strive to optimize their own behavior to adapt to the environment.

[0023] In the inner loop, agents in the artificial society learn to adjust their behavior to maximize their objective function through trial and error by completing service orders and earning revenue. Given a fixed intervention policy, the agents continuously explore and find the optimal behavior for their fixed payoff function while observing the intervention policy used by the planner. However, because the intervention policy and the behavior of other agents are constantly changing, the agents face a non-stationary Markov decision process. Specifically, changes in the intervention policy affect the agents' expected future revenue. Therefore, the agents need to adapt to the ever-changing and complex environment and adjust their behavior in response to changes in the planner's intervention policy. In the outer loop, the planner adjusts the intervention policy based on defined learning objectives to optimize the performance of the service ecosystem. Because the agents also change their behavior, the planner faces the problem of non-stationarity. To enable the agents and planner to simultaneously learn the optimal behavior, they must be trained jointly. Training the planner solely based on a fixed agent intervention policy is meaningless because it fails to account for the variability of agent behavior and is inconsistent with real-world service ecosystems. When learning intervention strategies, a multi-agent reinforcement learning process is used to train both the agent policy and the planner policy simultaneously. During the joint training process, the agent and planner update their weights simultaneously during each training process. Two-layer reinforcement learning methods can experience instability during the learning process. One reason is that during training, intervention strategies in extreme cases (without any constraints) may cause the agents in the system to lose their initial goals, and the agents cannot effectively learn their optimal behavior. To stabilize learning, this section uses two methods: staged learning and entropy regularization. By staggering the learning of the agent and planner, the agent can adapt to various intervention strategies before the planner begins learning. First, a group of agent models are trained in a non-intervention mode without intervention strategies. Then, in the second stage, intervention strategies are gradually introduced, and entropy regularization is applied to the planner policy to stabilize the learning process and promote collaborative adaptation between the agent and planner.

Claims

1. A method for designing service performance control strategies for swarm intelligence. This method learns from observational data without relying on modeling assumptions about the service system. It employs deep reinforcement learning to optimize the behavior of the crowd intelligence group within the service ecosystem and the dynamic intervention strategy of the control algorithm. In this method, the crowd intelligence group within the service ecosystem and the dynamic intervention strategy of the control algorithm serve as each other's operating environment, enabling dynamic interaction and mutual learning, ultimately achieving a dynamic balance between the two. This co-learning and evolutionary mechanism not only enhances the dynamic adjustment capabilities of the strategy but also grants it a high degree of flexibility and controllability. This method allows the historical states of the service ecosystem to be recreated in a virtual environment, and through diverse configurations, it can pre-imagine future virtual service ecosystem states, thereby better adapting to the ever-changing service ecosystem and improving the breadth and depth of learning and expression capabilities.

2. A swarm intelligence-oriented service efficiency control strategy design method according to claim 1. This method introduces two-layer reinforcement learning into a service ecology analysis method based on computational experiments, and proposes a dynamic intervention strategy design method based on two-layer reinforcement learning. This method includes the overall architecture of the dynamic intervention strategy design method, the operating mechanism of the agents in the artificial society, the operating mechanism of the dynamic intervention strategy planner, and the bilateral dynamic interaction process and evolutionary game mechanism. As a validation, the case study of the internalization of food delivery riders was selected to confirm the effectiveness of the proposed method. The overall architecture of the dynamic intervention strategy design method: The specific mechanisms and related connections of the planner operating mechanism, the agent operating mechanism, and the bilateral dynamic interaction process and evolutionary game mechanism are displayed in the form of an overall architecture diagram. The operating mechanism of the agents in the artificial society: The optimization objectives of the numerous agents in the artificial society are given from the perspective of reinforcement learning, and it is pointed out that the multi-agent environment here is trained in a shared weight manner. The operating mechanism of the dynamic intervention strategy planner: The optimization objectives of the dynamic intervention planner are given from the perspective of reinforcement learning. Bilateral dynamic interaction process and evolutionary game mechanism: From the perspective of reinforcement learning, the dynamic interaction and evolutionary game between intelligent agents and dynamic intervention planners in artificial societies are given, and how to conduct two-layer reinforcement learning training to obtain a stable solution space is introduced.

3. According to claim 1, a dynamic intervention strategy design method based on double-layer reinforcement learning is designed to simultaneously optimize the behavior of the crowd intelligence group and the intervention strategy of the control algorithm in the service ecosystem. Through double-layer deep reinforcement learning, this method promotes the interaction between the crowd intelligence group and the control algorithm, each becoming the other's environment, forming a dynamic game state. Through autonomous evolution and circular feedback mechanisms, the two achieve a dynamic balance. Through continuous interactive learning between the two, the dynamic adjustment ability of the strategy is enhanced, and the learning ability and pattern of the intelligent agents in the artificial society are optimized. In addition, this method can not only reproduce the historical state of the service ecosystem, but also construct the future state of the service ecosystem by setting a variety of configurations.

Images