Communication tariff scheme intelligent optimization method based on reinforcement learning and related products

Abstract

The invention discloses a communication tariff scheme intelligent optimization method based on reinforcement learning and a related product. A charge optimization model obtained by reinforcement learning training based on a DQN model can firstly obtain fusion features of each communication charge scheme to be processed, and the schemes are classified in combination with an elimination target to obtain eliminated and non-eliminated schemes. And then grouping the non-elimination schemes according to the similarity between the fusion features corresponding to the non-elimination schemes to form a plurality of tariff combination groups. And for each merging group, determining a pre-merging mode in combination with a merging optimization target, and evaluating a reward value of the mode by using a reward function in the optimization model. And when the reward value reaches a set threshold value, determining the pre-merging mode as a target merging mode. And defining a non-elimination scheme not included in the target merging mode as a reservation scheme, and finally outputting the target merging mode, the reservation scheme and the elimination scheme, thereby realizing intelligent and efficient optimization of the tariff scheme.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a method for intelligently optimizing communication tariff plans based on reinforcement learning and related products. Background Art

[0002] With the continuous development of telecom operators' businesses and fierce market competition, they frequently launch new packages to meet diverse market demands. This has led to a rapid increase in the number of packages in their product catalogs, placing significant pressure on product management and user experience. These packages are not only numerous and complex in structure, but also suffer from uneven user distribution. When streamlining and consolidating packages, operators typically use merging solutions based on predefined rules, such as combining packages based on the same price or similar service content. However, these rules are overly simplistic, unable to address the complex relationships between packages, and lack the ability to dynamically adjust, limiting the flexibility and efficiency of merging strategies.

[0003] Therefore, how to efficiently clean up and merge complex and unevenly distributed packages to improve product management and user experience, while overcoming the poor flexibility and low efficiency of merging solutions based on predefined rules, is a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0004] Based on the above problems, this application provides a reinforcement learning-based intelligent optimization method for communication tariff plans and related products, which can efficiently clean up and merge complex and unevenly distributed packages to improve product management and user experience, while overcoming the poor flexibility and low efficiency of merging solutions based on predefined rules.

[0005] The embodiments of this application disclose the following technical solutions:

[0006] A method for intelligently optimizing communication tariff plans based on reinforcement learning is applied to a tariff optimization model. The tariff optimization model is obtained by reinforcement learning training based on a deep Q network (DQN) model. The method includes:

[0007] Obtaining tariff fusion features of each communication tariff plan to be processed, and classifying the plurality of communication tariff plans to be processed in combination with the elimination target to obtain N elimination plans and M non-elimination plans; the tariff fusion features are obtained by fusing price structure features, service content features, user behavior features, and operation attribute features of the communication tariff plans; N and M are both non-negative integers;

[0008] Grouping the non-eliminated plans based on the similarity between the tariff fusion features of the non-eliminated plans to obtain multiple tariff fusion groups;

[0009] For each tariff merger group, determine a pre-merger mode for the tariff merger in combination with the merger optimization goal, and calculate the optimized reward value under the pre-merger mode using the reward function of the tariff optimization model;

[0010] If the optimization reward value is greater than or equal to the reward threshold, the pre-merger mode corresponding to the optimization reward value is determined as the target merging mode; the target merging mode includes a merging strategy of some or all of the M non-eliminated solutions;

[0011] Determine the non-elimination solutions that are not included in any of the target combination methods as the retained solutions;

[0012] Output each of the target merging methods, the retention plan and the elimination plan.

[0013] In a possible implementation, the non-eliminated solutions are grouped based on the similarity between the tariff fusion features of the non-eliminated solutions to obtain multiple tariff fusion groups, including:

[0014] Calculate the similarity between the tariff integration features of each non-elimination plan;

[0015] Establish a connection relationship between the non-eliminated solutions corresponding to two tariff fusion features whose similarity is greater than or equal to the similarity threshold;

[0016] All non-eliminated plans that are directly or indirectly connected through a connection relationship are grouped into the same group using the connectivity principle to obtain the multiple tariff combination groups.

[0017] In one possible implementation, the training process of the tariff optimization model includes:

[0018] Initializing the DQN model to obtain an initial model, and obtaining multiple historical communication tariff plans and historical tariff fusion features of each of the historical communication tariff plans; each of the historical communication tariff plans is marked with an optimization method label;

[0019] Combining the optimization method labels of each historical communication tariff plan, the historical tariff fusion characteristics, the reward function, and the reward weight staged adjustment strategy, the initial model is iteratively reinforced learning trained until the training cutoff condition is reached, thereby obtaining the tariff optimization model; the reward function is used to evaluate the feedback effect of the model on the tariff plan optimization goal, so as to guide the model's decision-making process for optimizing the merger, retention, and elimination of plans;

[0020] In each reinforcement learning training, P historical communication tariff plans are extracted, and the optimization method labels and historical tariff fusion features of the P historical communication tariff plans are used as model inputs, where P is a positive integer.

[0021] In a possible implementation, it is characterized in that the reward weight stage-by-stage adjustment strategy includes: dividing the model training rounds into a first stage round, a second stage round and a third stage round, and setting a corresponding reward function weight combination for each stage.

[0022] In a possible implementation, the reward function is:

[0023] R=W simp ·R simp +W user ·R user +W rev ·R rev +W comp ·R comp ;

[0024] Among them, R simp To streamline the rewards for the package, W simp is the weight of the reward; R user Influence rewards for users, W user is the weight of the reward; R rev is the income change reward, W rev is the weight of the reward; R comp is the operational complexity reward, W comp is the weight of the reward;

[0025] R simp =γ· , γ is the global effect reward coefficient, N before is the number of communication tariff plans before the merger, N after The number of communication tariff plans after the merger;

[0026] R rev =ε· , ε is the service improvement sensitivity coefficient, V is the number of users whose service is improved, i is the degree of improvement, W i is the weight of the degree of improvement;

[0027] R comp =ζ· ζ is the sensitivity coefficient of income change, Revenue before Revenue is the estimated revenue of the communication tariff plan before the merger. after Estimated revenue from the combined communications tariff plan.

[0028] In a possible implementation, before outputting the target merging schemes, the retention schemes, and the elimination schemes, the method further includes:

[0029] Combining the target merging method, the retention plan, and the elimination plan, a simulation evaluation is performed on user impact, business impact, and operational impact to obtain a simulation result;

[0030] If the simulation result meets the merging optimization target, the step of outputting each of the target merging methods, the retention scheme and the elimination scheme is performed.

[0031] In a possible implementation, the price structure characteristics include basic price characteristics, cost composition characteristics, and implicit value characteristics;

[0032] The service content features include service item coding features, service restriction condition features, and service combination features;

[0033] The user behavior characteristics include subscription characteristics, usage pattern characteristics and user attribute characteristics;

[0034] The operation attribute characteristics include time attribute characteristics, channel attribute characteristics and marketing strategy attribute characteristics.

[0035] A device for intelligently optimizing communication tariff plans based on reinforcement learning, comprising:

[0036] A first acquisition unit is configured to acquire a tariff fusion feature of each communication tariff plan to be processed; the tariff fusion feature is obtained by fusing the price structure feature, service content feature, user behavior feature, and operation attribute feature of the communication tariff plan;

[0037] A classification unit, configured to classify the plurality of communication tariff schemes to be processed in combination with an elimination target to obtain N eliminated schemes and M non-eliminated schemes; N and M are both non-negative integers;

[0038] A first grouping unit is configured to group the non-eliminated plans based on the similarity between the tariff fusion features of the non-eliminated plans to obtain a plurality of tariff fusion groups;

[0039] a determination calculation unit, configured to determine, for each tariff merging group, a pre-merger mode for the tariff merging in combination with a merger optimization objective, and calculate an optimized reward value under the pre-merger mode using a reward function of the tariff optimization model;

[0040] A first determining unit determines, if the optimization reward value is greater than or equal to a reward threshold, a pre-merger mode corresponding to the optimization reward value as a target merging mode; the target merging mode includes a merging strategy of some or all of the M non-eliminated solutions;

[0041] A second determining unit is configured to determine non-eliminated solutions that are not included in any of the target merging methods as retained solutions;

[0042] An output unit is used to output each of the target merging methods, the retention plan and the elimination plan.

[0043] A device for intelligent optimization of communication rate plans based on reinforcement learning, characterized in that it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the intelligent optimization method for communication rate plans based on reinforcement learning as described above.

[0044] A computer-readable storage medium, characterized in that instructions are stored in the computer-readable storage medium. When the instructions are executed on a terminal device, the terminal device executes the above-mentioned reinforcement learning-based intelligent optimization method for communication tariff plans.

[0045] Compared with the existing technology, this application has the following beneficial effects:

[0046] This application provides a reinforcement learning-based intelligent optimization method for communication rate plans and related products. Specifically, when implementing the reinforcement learning-based intelligent optimization method for communication rate plans provided in embodiments of this application, a rate optimization model, trained through reinforcement learning using a DQN (Deep Q-Network) model, first obtains the rate fusion features for each communication rate plan to be processed. These features are derived by fusing price structure, service content, user behavior, and operational attribute features. Multiple communication rate plans to be processed are then classified based on an elimination objective, resulting in N eliminated plans and M non-eliminated plans. Next, the non-eliminated plans are grouped based on the similarity between their rate fusion features to form multiple rate merging groups. For each rate merging group, a pre-merger method for the rate merging is determined based on the merger optimization objective, and the optimized reward value for this pre-merger method is calculated using the reward function of the rate optimization model. If the optimized reward value is greater than or equal to a reward threshold, the pre-merger method corresponding to the optimized reward value is determined as the target merging method. The non-eliminated plans not included in the target merging methods are then determined as retained plans. Finally, each target merging method, retention plan and elimination plan are output. The tariff optimization model obtained by training through reinforcement learning based on the DQN model in this application can dynamically adjust the merging strategy to cope with the complex relationship between packages, thereby improving the flexibility and efficiency of the merging strategy. At the same time, by obtaining and integrating the price structure characteristics, service content characteristics, user behavior characteristics and operation attribute characteristics of the communication tariff plan, it can more comprehensively and accurately reflect the characteristics of the tariff plan, which helps to more accurately classify and merge tariff plans. In addition, this application can not only group non-eliminated plans according to the degree of similarity, but also determine the pre-merger method in combination with the merging optimization goal, and use the reward function to calculate the optimization reward value, and then determine the target merging method, making the merging process more intelligent and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A method flow chart of a method for intelligent optimization of communication tariff plans based on reinforcement learning provided in an embodiment of the present application;

[0049] Figure 2 A method flow chart of a method for combining and grouping tariffs provided in an embodiment of the present application;

[0050] Figure 3 A method flow chart of a rate optimization model training method provided in an embodiment of the present application;

[0051] Figure 4 A structural diagram of a communication tariff scheme intelligent optimization device based on reinforcement learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.

[0053] With the development of telecom operators' businesses and intensified market competition, they are constantly launching new packages to meet diverse market demands, resulting in an explosive growth in the number of packages in their product catalogs. This not only poses significant challenges to product management but also impacts user experience. Specifically, these packages are massive: a large operator typically has thousands of different packages, with significant overlap and similarity between packages launched over time. Annual quarterly marketing campaigns continue to drive the addition of new packages. Furthermore, the packages are complex, covering a wide range of services, including voice, data, and SMS. Pricing varies, including base fees, overage fees, and tiered pricing. There are also complex dependencies and mutual exclusions between different packages. Furthermore, user distribution is uneven, with some legacy packages still only being used by a small number of users, and user usage patterns and preferences vary widely. Currently, solutions based on predefined rules are often used to streamline and consolidate packages, such as merging services based on the same price or similar services. However, these rules are overly simplistic, difficult to handle complex situations, and lack the ability to dynamically optimize and adjust, limiting the effectiveness and flexibility of consolidation strategies.

[0054] To address this problem, an embodiment of the present application provides a method and related products for intelligent optimization of communication tariff plans based on reinforcement learning. The tariff optimization model obtained by reinforcement learning training based on the deep Q network (DQN) model first obtains the fusion features of the communication tariff plan to be processed. These features integrate the price structure, service content, user behavior and operational attributes, and classify these plans into elimination plans and non-elimination plans based on the elimination target. Subsequently, the non-elimination plans are grouped according to the similarity between the tariff fusion features to form multiple tariff merging groups. For each merging group, a pre-merger plan is designed based on the merging optimization target, and the corresponding optimization reward value is calculated using the reward function in the tariff optimization model. When the reward value reaches or exceeds the set threshold, the pre-merger plan is selected as the target merging plan. At the same time, the non-elimination plans not included in the target merging plan are determined as retained plans. Finally, the target merging plan, retained plan and eliminated plan are output to achieve intelligent and efficient optimization of the tariff plan. This application uses a tariff optimization model obtained through training based on the DQN model to merge, eliminate and retain communication tariff plans. It can dynamically adjust the merging strategy to cope with the complex relationship between communication tariff plans, thereby improving the flexibility and efficiency of the merging strategy. By integrating the price structure characteristics, service content characteristics, user behavior characteristics and operational attribute characteristics of communication tariff plans, this application can more comprehensively and accurately reflect the characteristics of tariff plans, and help to more accurately classify and merge tariff plans. In addition, this application not only classifies non-eliminated plans based on the degree of similarity between tariff fusion characteristics, but also determines the pre-merger method in combination with the merger optimization goal, and uses the reward function to calculate the optimized reward value, and then determines the target merging method, making the merging process more intelligent and efficient.

[0055] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0056] See also Figure 1 , this figure is a flow chart of a method for intelligent optimization of communication tariff plans based on reinforcement learning provided in an embodiment of the present application. The method is applied to a tariff optimization model, and the tariff optimization model is obtained by reinforcement learning training based on a DQN model.

[0057] like Figure 1 As shown, the communication tariff intelligent optimization method based on reinforcement learning may include steps S101-S106:

[0058] S101: The tariff optimization model obtains tariff integration features of each communication tariff plan to be processed, and classifies a plurality of the communication tariff plans to be processed in combination with an elimination target to obtain N elimination plans and M non-elimination plans.

[0059] To intelligently optimize each pending communication rate plan, the rate optimization model (hereinafter referred to as this model) first obtains the rate fusion features of each pending communication rate plan. These fusion features are derived by integrating relevant data about the pending communication rate plans, including but not limited to price structure, service content, user behavior, and operational attributes. Next, based on a pre-defined elimination objective—such as reducing operating costs, improving user satisfaction, or improving market competitiveness—the rate optimization model classifies all communication rate plans. Rate plans that underperform or meet the elimination objective are classified as eliminated, while those that meet the optimization requirements and have value are classified as non-eliminated. For example, for a given operator, suppose that Package A is expensive and has low user activity, while Package B is reasonably priced and has strong user stickiness. The elimination objective is to eliminate packages with low user activity. In this case, Package A is classified as eliminated, while Package B is classified as non-eliminated. Both N and M are non-negative integers.

[0060] In one possible implementation, the sources of the communication tariff plan data to be processed may include but are not limited to: obtaining package definition data from the operator's product management system, or obtaining package subscription and usage data from the customer relationship management (CRM) system and billing system, or obtaining user information and behavior data from the CRM system.

[0061] In one possible implementation, elimination targets refer to a series of pre-defined criteria or strategies during the rate plan optimization process, used to identify and eliminate communications rate plans that do not meet operational requirements, lack market competitiveness, or provide poor user experience. The goal is to effectively screen and reduce resource waste, improve the rationality and profitability of the overall rate structure, and ensure user satisfaction. Specifically, elimination targets may include, but are not limited to, the following aspects:

[0062] Insufficient price competitiveness: The price of the tariff plan is significantly higher than that of similar products in the market, resulting in user churn or low subscription rates.

[0063] Low user activity: Users of relevant tariff plans have low usage frequency and stickiness, reflecting the lack of attractiveness of the plans.

[0064] Low profit margin: The revenue generated by the tariff plan cannot cover the costs or the profit margin does not meet the operator's expectations.

[0065] Service content does not match user needs: There is a significant deviation between the service content of the package configuration and the actual needs of the target user group.

[0066] Technical or operational limitations: Tariff plans are difficult to promote or upgrade continuously due to factors such as network coverage and compatibility.

[0067] By clarifying the elimination targets, the tariff optimization model can scientifically classify the tariff integration features to be optimized and classify poorly performing solutions into elimination categories, thereby providing a decision-making basis for subsequent optimization mergers and resource reconfiguration, and realizing refined management and intelligent upgrades of the communications tariff system.

[0068] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) and communication tariff plans involved in this application are all information, data and plans authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0069] S102: The tariff optimization model groups the non-eliminated solutions based on the similarity between the tariff fusion features of the non-eliminated solutions to obtain a plurality of tariff fusion groups.

[0070] After completing the initial screening and classification of the communication tariff plans to be processed, the tariff optimization model will focus on the tariff fusion features of non-eliminated plans. In order to further improve optimization efficiency and achieve reasonable integration of resources, the tariff optimization model will systematically group these non-eliminated tariff fusion features based on the similarity between them. Through this process, the originally scattered non-eliminated plans are divided into several similar tariff fusion groups. The plans within each fusion group tend to be consistent in their feature performance, which is conducive to the subsequent use of a unified fusion strategy for optimization and adjustment. This similarity-based grouping method can not only reduce redundant configurations and improve the overall coordination of tariff plans, but also effectively support the reinforcement learning model's accurate judgment in fusion decisions, ultimately promoting the evolution of communication tariff plans in a more scientific and reasonable direction.

[0071] S103: The tariff optimization model determines a pre-merger mode for each tariff merger group in combination with the merger optimization target, and calculates an optimized reward value under the pre-merger mode using the reward function of the tariff optimization model.

[0072] After grouping multiple non-eliminated plans into several rate consolidation groups, the rate optimization model then determines, for each rate consolidation group, a pre-merger method for the rate plans within that group, based on the specific consolidation optimization objectives. This pre-merger method refers to the process in which the model designs possible consolidation strategies based on the consolidation optimization objectives, such as merging Plan A into Plan B. This approach aims to achieve effective resource integration and enhance the competitiveness and user satisfaction of the overall rate plan. After determining the pre-merger method, the rate optimization model evaluates the consolidation plan using a built-in reward function. This reward function comprehensively considers multiple indicators, such as revenue variability, user impact, and operational complexity. By quantifying these benefits, it calculates the optimal reward value corresponding to the current pre-merger method. This reward value reflects the pros and cons of the consolidation plan in meeting the consolidation optimization objectives and serves as an important basis for subsequent decision-making.

[0073] It should be noted that merger optimization objectives are the core criteria guiding the rate plan merger process, clarifying the specific effects and direction the merger should achieve. In one possible implementation, these objectives could include the following: first, reducing operating costs by eliminating duplicate or inefficient rate plans and optimizing resource allocation; second, improving user experience by simplifying package structures to make them easier for users to understand and choose, while also increasing package attractiveness to enhance customer stickiness; third, improving overall profitability by increasing average revenue and profit through rational pricing and reasonable mergers; and fourth, strengthening market competitiveness by rapidly responding to market changes and launching differentiated rate packages that meet user needs. For example, an operator might set a merger optimization objective of reducing the number of package options by more than 10%, while ensuring a user churn rate of no more than 2%, and achieving a 5% increase in overall profit. Based on these objectives, the model, when designing and evaluating pre-merger plans, will focus on how to merge rate plans to achieve both product line simplification and user stability and profit growth, thereby promoting the intelligent optimization and continuous improvement of communication rate plans.

[0074] S104: If the optimization reward value is greater than or equal to the reward threshold, the tariff optimization model determines the pre-merger mode corresponding to the optimization reward value as the target merger mode.

[0075] After evaluating the pre-merger method for each rate consolidation group, the rate optimization model calculates a corresponding optimized reward value using a reward function. This value reflects the pre-merger method's overall effectiveness in achieving the consolidation optimization goal. When this optimized reward value reaches or exceeds the pre-set reward threshold, it indicates that the pre-merger method can effectively improve the overall performance of the rate plan and meet the operator's requirements for cost control, user experience, and market competitiveness. Based on this judgment, the rate optimization model officially determines the pre-merger method corresponding to this optimized reward value as the target consolidation method.

[0076] It's important to note that the targeted merger approach encompasses not only targeted mergers for selected non-elimination plans but also encompasses the entire set of non-elimination rate convergence features, effectively merging and adjusting multiple non-elimination plans in a coordinated and unified manner. This flexible and precise merger strategy helps optimize resource allocation, eliminate internal redundancies, and improve the overall efficiency and competitiveness of the rate system, thereby ensuring the continued strength and vitality of communications rate plans in a competitive market.

[0077] S105: The tariff optimization model determines the non-eliminated solutions that are not included in any of the target merging methods as retained solutions.

[0078] After the rate optimization model has completed grouping and determining the merging methods for the non-eliminated rate convergence characteristics, it will further clarify the final allocation of each communication rate plan. Specifically, the model will identify and retain those non-eliminated plans that are not included in any target merger method. This means that these plans meet the basic optimization criteria, but due to their uniqueness or stable market performance, are not included in the merger adjustment, allowing them to continue to serve users in their current state.

[0079] Through this dual screening mechanism, the rate optimization model achieves refined management of communication rate plans, maximizing the retention of plans with sustainable value and market demand, ensuring that users' diverse needs are met. This scientific classification and decision-making process effectively supports operators' marketing strategy goals of improving the rationality of their rate structures and profitability in a highly competitive environment.

[0080] S106: The tariff optimization model outputs the target merging methods, the retention scheme, and the elimination scheme.

[0081] After comprehensively analyzing, categorizing, and determining consolidation strategies for communication rate plans, the rate optimization model systematically outputs three key results: target consolidation methods, retention plans, and elimination plans. By simultaneously outputting these three types of solutions, the rate optimization model scientifically organizes and optimizes the entire rate structure, helping operators streamline and upgrade their rate systems while balancing user experience and market competitiveness, thereby promoting the sustainable and healthy development of their communications business.

[0082] Based on steps S101-S106, the rate optimization model first processes the converged features of multiple communication rate plans. These features are formed by integrating information from multiple dimensions, including price structure, service content, user behavior, and operational attributes. Subsequently, these plans are classified into "eliminated" and "non-eliminated" categories based on the preset elimination target. Next, the non-eliminated plans are grouped based on the similarity between their rate convergence features to form several rate merging groups. For each merging group, a pre-merger method is determined based on the merging optimization target, and the corresponding optimized reward value is calculated using the reward function in the rate optimization model. When the reward value reaches or exceeds the set threshold, the pre-merger method is considered to be a suitable target merging method. Finally, the non-eliminated plans not included in any target merging method are identified as retained plans. The determined target merging methods, retained plans, and eliminated plans are then output, achieving intelligent optimization management of communication rate plans. The rate optimization model trained using DQN reinforcement learning in this application can dynamically adjust merging strategies to effectively address complex relationships between plans, thereby improving the flexibility and efficiency of merging. By integrating multi-dimensional features such as price structure, service content, user behavior, and operational attributes, a more comprehensive and accurate representation of tariff plans is achieved, facilitating precise classification and merging. Furthermore, by combining similarity grouping with merging optimization objectives and using a reward function to evaluate pre-merger plans, the optimal merging method is intelligently determined, making the tariff merging process more efficient and intelligent.

[0083] In a possible implementation, the present application also provides a method for grouping tariff merging groups, see Figure 2 , Figure 2 A method flow chart of a method for grouping tariff merging groups provided in an embodiment of the present application. Accordingly, step S102 groups the non-eliminated plans based on the similarity between the tariff fusion characteristics of the non-eliminated plans to obtain multiple tariff merging groups, which can be specifically implemented through steps S201-S203:

[0084] S201: Calculate the similarity between the tariff fusion features of each non-elimination plan.

[0085] Calculating the similarity between the fusion features of each non-elimination plan is a key step in the rate optimization and management process. This process aims to quantify the similarity between features and provide data support for subsequent grouping. This process first requires defining an appropriate similarity calculation method to quantitatively measure the similarity between any two non-elimination plan fusion features. For example, Euclidean distance, cosine similarity, Manhattan distance, or correlation coefficient based on probability statistics can be used to measure the proximity between features at different angles and scales.

[0086] S202: Establish a connection relationship between the non-eliminated solutions corresponding to the two non-tariff fusion features whose similarity is greater than or equal to the similarity threshold.

[0087] In the rate optimization model, establishing connections between two non-eliminated plans with similarity greater than or equal to a preset similarity threshold is a key step in achieving efficient grouping and merging. This process compares the similarity between the fused features of the non-eliminated plans pair by pair, selecting those feature pairs whose similarity meets or exceeds the threshold. Setting the similarity threshold ensures that only features with sufficiently high similarity are considered closely associated, thereby avoiding invalid or misleading connections due to low similarity and ensuring accurate grouping and business relevance. Once feature pairs that meet the criteria are identified, the model establishes connections between their corresponding plans. By establishing these connections, the model effectively makes implicit connections between the non-eliminated plans explicit, forming a network structure consisting of nodes (features) and edges (connections). This threshold-based connection strategy not only enhances the model's understanding of logical connections within rate plans but also lays a solid foundation for subsequent clustering and grouping based on connectivity principles. More importantly, this mechanism can dynamically adapt to the strictness of similarity requirements in different business scenarios. By adjusting thresholds, it flexibly controls the density of connection relationships and the granularity of grouping, thereby enabling precise management and optimization of tariff plans. Therefore, establishing connection relationships is not only a technical means but also a key step in the model's implementation of intelligent decision-making and precise resource integration.

[0088] For example, assuming that there are four non-elimination plans corresponding to four tariff integration features A, B, C, and D, after similarity calculation, the following similarity values are obtained:

[0089] The similarity between A and B is 0.85; the similarity between A and C is 0.65; the similarity between B and C is 0.75; the similarity between B and D is 0.80; and the similarity between C and D is 0.72. If the similarity threshold is set to 0.70, then according to the above steps:

[0090] A connection is established between the non-eliminated solutions corresponding to A and B (because their similarity is 0.85), between B and C (because their similarity is 0.75), between B and D (because their similarity is 0.80), and between C and D (because their similarity is 0.72). However, no connection is established between the non-eliminated solutions corresponding to A and C because their similarity is 0.65, which is below the threshold.

[0091] This process effectively identifies and establishes connections between sufficiently similar non-elimination tariff convergence features. This not only helps clearly demonstrate the correlation between features, but also provides the necessary support for subsequent grouping based on the connectivity principle.

[0092] In short, building connection relationships is a key step in converting similarity-measured feature relationships into visual network structures, laying the foundation for the ultimate formation of an effective tariff merger group.

[0093] S203: All non-eliminated plans that are directly or indirectly connected through a connection relationship are grouped into the same group using a connectivity principle to obtain the multiple tariff combination groups.

[0094] In the rate optimization model's grouping process, utilizing the connectivity principle to group all non-eliminated options that are directly or indirectly connected through connections is a key step in achieving reasonable segmentation and effective integration. Specifically, after constructing a connectivity network based on the aforementioned similarity threshold, each non-eliminated option is considered a node in the network, and feature nodes that meet the similarity requirements are connected by edges. The connectivity principle states that in this network, whether two nodes are connected by a single direct edge or by a path consisting of multiple edges, as long as such a connected path exists, the nodes are considered to belong to the same connected subgraph. Based on this principle, the model traverses and analyzes the entire connectivity network, aggregating and merging all non-eliminated options within each connected subgraph into the same group. This approach not only ensures high similarity and intrinsic connections between features within a group, but also avoids grouping fragmentation or omissions caused by local judgments, ensuring the integrity and logical consistency of the grouping. Through this method, the model can scientifically identify multiple highly cohesive tariff merger groups. The non-eliminated plans within each merger group have strong correlations in terms of price structure, service content and user behavior, thus providing a solid data foundation and strategic basis for subsequent tariff plan mergers and optimization.

[0095] For example, assume the following non-elimination resource solutions and their connection relationships:

[0096] Plan A is connected to Plans B, C, and D; Plan B is connected to Plans A and C; Plan C is connected to Plans A and B; Plan D is connected to Plan A; Plan E is connected to Plan F; and Plan F is connected to Plan E. Based on the above connection relationships, the following structure can be constructed:

[0097] son Figure 1 :Contains plans A, B, C, D; Figure 2 : Includes plans E and F.

[0098] Through the principle of connectivity, the Figure 1Plans A, B, C, and D in the plan are grouped into the same tariff combination group. Figure 2 Plans E and F are grouped into another tariff combination group.

[0099] In a possible implementation, the present application also provides a method for training a rate optimization model, see Figure 3 , Figure 3 A method flow chart of a tariff optimization model training method provided in an embodiment of the present application can be specifically implemented through steps S301-S303:

[0100] S301: Initialize the DQN model to obtain an initial model.

[0101] Initializing the DQN model to obtain an initial model is the first step in the rate optimization training process, aiming to provide a reasonable starting point for subsequent reinforcement learning. This initialization process typically involves designing the structure and parameters of the deep Q network. Specifically, this involves determining hyperparameters such as the number of layers, the number of neurons in each layer, and the activation function. Initial values for the network weights and biases are assigned randomly or based on a set of rules to ensure the model has appropriate expressive power and learning potential. At this stage, the initial model has not yet undergone any training, and its behavior is relatively neutral, but it already possesses the basic ability to process input data and output action value estimates. The quality of the initial model directly affects the convergence speed and ultimate effectiveness of reinforcement learning. Therefore, the design should comprehensively consider the complexity of the task and the characteristics of the data, balancing the model's capacity and generalization ability. After initialization, the DQN model enters an iterative training phase based on historical rate fusion features and reward feedback, continuously updating parameters to achieve precise optimization decisions for communication rate plans.

[0102] In one possible implementation, the initialization of the DQN model includes:

[0103] The DQN model was initialized as a three-layer neural network structure, containing 256, 128, and 64 neurons, respectively. This layer configuration ensures strong expressiveness while effectively controlling model complexity and adapting to the high-dimensional input of the tariff-integrated feature. The learning rate of the DQN model was set to 0.001 to ensure stable and efficient parameter updates during training. The discount factor of the DQN model was set to 0.9 to balance the weights of current and future rewards, thereby maximizing long-term benefits. A mini-batch stochastic gradient descent strategy with a batch size of 32 was used during training to improve training stability and convergence speed. The entire training process was repeated for 500 rounds, with the reward function weights dynamically adjusted in stages to meet the optimization requirements of different training stages.

[0104] In terms of reward function configuration, package simplification, user impact, revenue change, and operational complexity are assigned initial weights of 0.4, 0.3, 0.2, and 0.1, respectively. They are then adjusted according to the training process in the first three stages: in the first round (e.g., rounds 1 to 200), package simplification, user impact, revenue change, and operational complexity are assigned corresponding weights of 0.5, 0.2, 0.2, and 0.1, respectively. The goal is to increase the weight of simplification rewards to quickly reduce obvious duplicate packages; in the second round (e.g., In rounds 201 to 400, package simplification, user impact, revenue change, and operational complexity were weighted at 0.4, 0.3, 0.2, and 0.1, respectively. The goal was to increase the weight of user impact rewards and focus on user experience. In the third round (e.g., rounds 401 to 500), package simplification, user impact, revenue change, and operational complexity were weighted at 0.3, 0.3, 0.3, and 0.1, respectively. The goal was to increase the weight of revenue and complexity rewards and optimize overall system performance. This phased adjustment strategy effectively guides the model to focus on different objectives at different training stages, ensuring that the DQN model has good basic performance and flexible adaptability after initialization. This lays a solid foundation for subsequent reinforcement learning training and enables intelligent optimization of communication tariff plans.

[0105] It should be noted that the DQN model is a reinforcement learning method that combines deep neural networks with Q-learning algorithms. Its core idea is to use deep neural networks to approximate the action-value function (Q function), so that the intelligent agent can learn the best decision-making strategy in a complex and high-dimensional state space.

[0106] S302: Acquire multiple historical communication tariff plans and historical tariff fusion features of each of the historical communication tariff plans.

[0107] Obtaining multiple historical communication tariff plans and the historical tariff fusion features of each historical communication tariff plan is an important data preparation step in training the tariff optimization model. Specifically, the multi-dimensional fusion features corresponding to each communication tariff plan are comprehensively extracted from the historical operating data. These features cover key indicators such as tariff structure, user usage behavior, service content, and market feedback, and can fully reflect the actual performance and potential value of the tariff plan. At the same time, in order to support the supervised learning process of the model, each historical communication tariff plan is given a clear optimization method label. The label indicates the specific optimization strategy adopted by the plan during actual optimization, such as merging with a certain plan, retaining or eliminating it. This labeled communication tariff plan not only provides an accurate learning goal for the model, but also helps to strengthen the design of the reward function and decision-making guidance in the learning process, thereby ensuring that the model can gradually master the reasonable handling principles of the tariff plan based on real business results and optimization experience.

[0108] S303: Combining the optimization method labels of each historical communication tariff plan, the historical tariff fusion characteristics, the reward function and the reward weight stage-by-stage adjustment strategy, the initial model is iteratively reinforced learning trained until the training cutoff condition is reached, thereby obtaining the tariff optimization model.

[0109] During the training process of the rate optimization model, the optimization method labels of each historical communication rate plan, the historical rate fusion features, a designed reward function, and a phased adjustment strategy for the reward weights are organically combined. The initial model is continuously optimized through an iterative reinforcement learning method. Specifically, the reward function plays a key role in evaluating the model's performance on the rate plan optimization objective, effectively guiding the model to continuously improve its strategy in decision-making stages such as merging, retaining, and eliminating, thereby enhancing the overall optimization effect. In each reinforcement learning training iteration, P samples are drawn from a rich historical communication rate plan database. The optimization method labels and historical rate fusion features corresponding to these P samples are used as model inputs, where P is a positive integer. This ensures that the training process covers a wide range of rate plans while maintaining reasonable computational efficiency. Training continues until a preset training cutoff condition (such as convergence criterion or maximum number of training rounds) is met, ultimately obtaining a fully tuned rate optimization model. Through this reinforcement learning training method based on historical data, reward mechanism and dynamic weight adjustment, the model can realize intelligent and adaptive tariff plan optimization decision-making, effectively supporting operators' refined management and innovative development in complex market environments.

[0110] In one possible implementation, the reward weight phased adjustment strategy divides the total rounds of model training into three stages: the first stage (e.g., rounds 1-200), the second stage (rounds 201-400), and the third stage (rounds 401-500). A different reward function weight combination is set for each stage, specifically the weight ratios of the four dimensions of package streamlining, user impact, revenue changes, and operational complexity.

[0111] In the first phase, the weight combination is [0.5, 0.2, 0.2, 0.1], focusing on increasing the reward weight for package simplification to quickly reduce obvious duplication and redundant tariff packages and improve system efficiency.

[0112] The weights for the second phase are adjusted to [0.4, 0.3, 0.2, 0.1], increasing the weight of user influence and emphasizing the optimization of user experience to ensure that tariff adjustments meet business goals while also taking into account customer satisfaction.

[0113] The third stage is adjusted to [0.3, 0.3, 0.3, 0.1], further increasing the weights of revenue changes and operational complexity, promoting the model to control operating costs while ensuring revenue growth, and optimizing the overall performance of the system.

[0114] Through this strategy of dynamically adjusting reward weights in stages, the initial model can optimize different indicators in a targeted manner according to the different focuses of the training process, thereby achieving balanced and efficient optimization of the tariff plan.

[0115] In one possible implementation, the reward function is:

[0116] R=W simp ·R simp +W user ·R user +W rev ·R rev +W comp ·R comp ;

[0117] Among them, R simp To streamline the rewards for the package, W simp is the weight of the reward; R user Influence rewards for users, W user is the weight of the reward; R rev is the income change reward, W rev is the weight of the reward; R comp is the operational complexity reward, W comp is the weight of the reward;

[0118] R simp =γ· , γ is the global effect reward coefficient, N before is the number of communication tariff plans before the merger, N after The number of communication tariff plans after the merger;

[0119] R rev =ε· , ε is the service improvement sensitivity coefficient, V is the number of users whose service is improved, i is the degree of improvement, W i is the weight of the degree of improvement;

[0120] R comp =ζ· ζ is the sensitivity coefficient of income change, Revenue before Revenue is the estimated revenue of the communication tariff plan before the merger. after Estimated revenue from the combined communications tariff plan.

[0121] In a possible implementation, before outputting the target merging schemes, the retention schemes, and the elimination schemes, the method further includes:

[0122] A simulation evaluation of user impact, business impact, and operational impact is performed in combination with the target merging method, the retention plan, and the elimination plan to obtain a simulation result. If the simulation result meets the merging optimization goal, the step of outputting each of the target merging method, the retention plan, and the elimination plan is performed.

[0123] Before outputting the target consolidation methods, retention options, and elimination options, a key step is to simulate and evaluate the user, business, and operational impacts of these combined methods, retention options, and elimination options to generate simulation results. If the simulation results show that these options meet the predetermined consolidation optimization goals, the step of outputting the target consolidation methods, retention options, and elimination options is executed.

[0124] Specifically, simulation evaluation involves inputting generated scenarios into a simulation system to assess their impact on users, business, and operations. This includes calculating metrics such as user satisfaction, business continuity, and operating costs.

[0125] Result verification: By comparing the simulation results with the preset combined optimization goals (such as improving user satisfaction, reducing costs, or simplifying business processes), it is determined whether these solutions meet expectations.

[0126] Conditional output: If the simulation results show that all solutions have achieved or exceeded the set goals, these solutions will be output; otherwise, the solutions need to be further adjusted until the conditions are met.

[0127] This process ensures that the output solution is not only feasible in theory but also achieves the expected results in practical applications, thereby improving the overall reliability and effectiveness of the tariff optimization strategy.

[0128] In one possible implementation, the user impact evaluation metrics include:

[0129] (1) User service change index: measures the degree of change in the service content of the tariff plan and reflects the differences in the services enjoyed by users.

[0130] (2) User cost change index: evaluates the proportion of cost changes caused by tariff adjustments to users.

[0131] (3) User migration cost index: measures the adaptation and conversion costs that users need to pay when switching from the original package to the new package.

[0132] (4) User satisfaction prediction: Based on historical data and behavioral analysis, predict the changing trend of user satisfaction with the new tariff plan.

[0133] Business impact assessment metrics include:

[0134] (1) Revenue change rate: The percentage increase or decrease in overall revenue after the forecast rate merger or adjustment.

[0135] (2) Changes in ARPU (average revenue per user): Evaluate changes in revenue contribution from individual users after the merger.

[0136] (3) User retention rate prediction: Estimate the likelihood that users will continue to use the service after the price adjustment.

[0137] (4) Market competitiveness index: a comprehensive assessment of the impact of the adjustment plan on the company’s competitive position and advantages in the market.

[0138] Evaluation indicators of operational impact include: Package simplification rate: measures the proportion of reduction in the number of packages after the tariff plan is adjusted, reflecting the simplification effect.

[0139] (1) Changes in management complexity: Assess the impact of tariff adjustments on internal management processes and difficulty.

[0140] (2) System implementation difficulty: Evaluate the technical complexity and challenges involved in the implementation of new solutions.

[0141] (3) Simplification of marketing communication: measures the degree of simplification and efficiency improvement of marketing promotion activities after rate adjustment.

[0142] The above indicators together constitute a multi-dimensional evaluation system for the impact of tariff plans, supporting scientific decision-making and the formulation of optimization plans.

[0143] In one possible implementation, the price structure features include the following three parts:

[0144] Basic price features: These cover the basic elements of a package, such as monthly base fee, contract length, minimum spending standard, discount factor, first-charge refund amount, etc. These features reflect the core pricing framework of the package.

[0145] Cost composition characteristics: Detailed description of the various cost components, including voice charges, data charges, SMS charges, over-package charges, and international roaming charges, etc., and a detailed description of the charging structure for users on different services.

[0146] Implicit value characteristics: These reflect the implicit value indicators of the tariff plan, such as unit traffic cost (yuan / GB), unit call cost (yuan / minute), package price-performance ratio, and deviation from the market average price, to help evaluate the economic rationality and market competitiveness of the package.

[0147] Among them, the vector of price structure characteristics is expressed as:

[0148] V price =[b1,b2,...,b m ,c1,c2,...,c k ,v1,v2,...,v f ]

[0149] Among them, b i is the basic price feature, i is 1, 2, ..., m; c j is the cost composition characteristic, j is 1, 2, ..., k; v h It is the implicit value feature, and the value of h is 1, 2, ..., f.

[0150] In one possible implementation, the service features include the following three aspects:

[0151] Service item features: covers specific service content indicators, such as call minutes, GB of traffic, number of text messages, as well as value-added services such as video rights and music memberships, comprehensively reflecting the various service resources included in the package.

[0152] Service restriction features: Describes various restrictions on service use, including traffic speed limit thresholds, directional traffic application scope, call region restrictions and validity period restrictions, etc., reflecting the constraints and regulations of the service.

[0153] Service portfolio characteristics: depict the combination relationship between different service items, such as the combination degree of basic communication and entertainment benefits, the ratio of data traffic and call duration, and the proportion of family package and personal package benefits, etc., reflecting the overall service matching strategy.

[0154] Among them, the vector of service content features is expressed as:

[0155] V servicee =[s1,s2,...,s p ,r1,r2,...,r q ,g1,g2,...,g s ]

[0156] Among them, s i is the service item feature, i is 1, 2, ..., p; r j is the service restriction condition, the value of j is 1, 2, ..., g; g h It is the service combination feature, and the value of h is 1, 2, ..., s.

[0157] In one possible implementation, the user behavior characteristics mainly include the following three dimensions:

[0158] Subscription characteristics: reflect users' subscription behavior and dynamic changes in tariff plans, including indicators such as the number of users, new addition rate, churn rate, average holding time, and renewal rate, which are used to evaluate the market acceptance and user stability of the plan.

[0159] Usage pattern characteristics: Describes users' actual usage habits of communication services, such as average monthly call duration, data usage rate, proportion of free resource consumption, communication time distribution, and cross-network call ratio, to help understand user needs and behavioral characteristics.

[0160] User attribute characteristics: Covers users' basic demographic information and value assessment indicators, including age distribution, geographical distribution, consumption level, user value level and brand loyalty, etc., supporting the construction of accurate user portraits and the formulation of differentiated operation strategies.

[0161] Among them, the vector of user behavior characteristics is expressed as:

[0162] V user =[u1,u2,...,u x ,m1,m2,...,m y ,a1,a2,...,a z ]

[0163] Among them, u i is the ordering feature, i is 1, 2, ..., x; m j It is the usage pattern feature, the value of j is 1, 2, ..., y; a h is the user attribute feature, and the value of h is 1, 2, ..., z.

[0164] In one possible implementation, the operational attribute characteristics mainly include the following three dimensions:

[0165] Time attribute characteristics: Covers the launch time, life cycle stage, last update time and promotion period of the tariff plan, reflecting the status and change pattern of the plan in the time dimension.

[0166] Channel attribute characteristics: Describes the sales and promotion channel distribution of the tariff plan, including the proportion of online channels, the proportion of offline channels, the type of main channels, and the difference in channel discounts, reflecting the impact of channel strategy on plan performance.

[0167] Marketing strategy attributes and characteristics: These include the market positioning and competitive environment of the tariff plan, such as target market positioning, competitive product benchmarking, marketing strategy importance level, and market promotion priority, supporting the effective integration of the plan with the company's overall marketing strategy.

[0168] Among them, the vector of operation attribute characteristics is expressed as:

[0169] V oper =[t1,t2,...,t a ,d1,d2,...,d b ,e1,e2,...,e c ]

[0170] Among them, t i It is a time attribute feature, and the value of i is 1, 2, ..., a; d j is the channel attribute feature, the value of j is 1, 2, ..., b; e h It is the attribute characteristic of marketing strategy, and the value of h is 1, 2, ..., c.

[0171] See also Figure 4 , Figure 4 This is a structural diagram of a communication tariff intelligent optimization device based on reinforcement learning provided in an embodiment of the present application. Figure 4 As shown, the communication tariff intelligent optimization device based on reinforcement learning includes:

[0172] The first acquisition unit 401 is configured to acquire a tariff fusion feature of each communication tariff plan to be processed; the tariff fusion feature is obtained by fusing the price structure feature, service content feature, user behavior feature, and operation attribute feature of the communication tariff plan;

[0173] A classification unit 402 is configured to classify the plurality of communication rate plans to be processed according to the elimination target to obtain N eliminated plans and M non-eliminated plans, wherein N and M are both non-negative integers;

[0174] A first grouping unit 403 is configured to group the non-eliminated plans based on the similarity between the tariff fusion features of the non-eliminated plans to obtain a plurality of tariff fusion groups;

[0175] a determination calculation unit 404 for determining, for each tariff merging group, a pre-merger mode for the tariff merging in combination with a merger optimization objective, and calculating an optimized reward value under the pre-merger mode using a reward function of the tariff optimization model;

[0176] The first determining unit 405 determines, if the optimization reward value is greater than or equal to the reward threshold, the pre-merger mode corresponding to the optimization reward value as the target merging mode; the target merging mode includes a merging strategy of some or all of the M non-eliminated solutions;

[0177] The second determining unit 406 determines the non-eliminated solutions that are not included in any of the target merging modes as retained solutions;

[0178] The output unit 407 is configured to output each of the target merging methods, the retention scheme, and the elimination scheme.

[0179] In a possible implementation, the first grouping unit 403 specifically includes:

[0180] A similarity calculation unit, used to calculate the similarity between the tariff fusion features of each non-elimination plan;

[0181] A connection relationship building unit, configured to build a connection relationship between non-eliminated solutions corresponding to two tariff fusion features whose similarity is greater than or equal to a similarity threshold;

[0182] The second grouping unit is configured to group all non-eliminated plans that are directly or indirectly connected via a connection relationship into the same group by utilizing a connectivity principle, to obtain the multiple tariff combination groups.

[0183] In a possible implementation, the apparatus further includes:

[0184] An initialization unit, used to initialize the DQN model to obtain an initial model;

[0185] A second acquisition unit is configured to acquire a plurality of historical communication tariff plans and a historical tariff fusion feature of each of the historical communication tariff plans; each of the historical communication tariff plans is marked with an optimization method label;

[0186] A training unit is configured to perform iterative reinforcement learning training on the initial model based on the optimization method labels of each historical communication tariff plan, the historical tariff fusion characteristics, the reward function, and the reward weight stage-by-stage adjustment strategy, and terminate the training until a training cutoff condition is met, thereby obtaining the tariff optimization model; the reward function is used to evaluate the feedback effect of the model on the tariff plan optimization goal, so as to guide the model's decision-making process for optimizing the merger, retention, and elimination of plans;

[0187] In each reinforcement learning training, P historical communication tariff plans are extracted, and the optimization method labels and historical tariff fusion features of the P historical communication tariff plans are used as model inputs, where P is a positive integer.

[0188] In one possible implementation, the reward weight adjustment strategy includes dividing the model training rounds into a first-stage round, a second-stage round, and a third-stage round, and setting a corresponding reward function weight combination for each stage.

[0189] In one possible implementation, the reward function is:

[0190] R=W simp ·Rsimp +W user ·R user +W rev ·R rev +W comp ·R comp ;

[0191] Among them, R simp To streamline the rewards for the package, W simp is the weight of the reward; R user Influence rewards for users, W user is the weight of the reward; R rev is the income change reward, W rev is the weight of the reward; R comp is the operational complexity reward, W comp is the weight of the reward;

[0192] R simp =γ· , γ is the global effect reward coefficient, N before is the number of communication tariff plans before the merger, N after The number of communication tariff plans after the merger;

[0193] R rev =ε· , ε is the service improvement sensitivity coefficient, V is the number of users whose service is improved, i is the degree of improvement, W i is the weight of the degree of improvement;

[0194] R comp =ζ· ζ is the sensitivity coefficient of income change, Revenue before Revenue is the estimated revenue of the communication tariff plan before the merger. after Estimated revenue from the combined communications tariff plan.

[0195] In a possible implementation, the apparatus further includes:

[0196] a simulation unit, configured to perform simulation evaluation of user impact, business impact, and operational impact in combination with the target merging method, the retention plan, and the elimination plan, and obtain simulation results;

[0197] An execution unit is used to execute the steps of outputting the target merging methods, the retention scheme and the elimination scheme if the simulation result meets the merging optimization target.

[0198] In a possible implementation, the price structure characteristics include basic price characteristics, cost composition characteristics, and implicit value characteristics;

[0199] The service content features include service item coding features, service restriction condition features, and service combination features;

[0200] The user behavior characteristics include subscription characteristics, usage pattern characteristics and user attribute characteristics;

[0201] The operation attribute characteristics include time attribute characteristics, channel attribute characteristics and marketing strategy attribute characteristics.

[0202] In addition, an embodiment of the present application also provides a communication rate scheme intelligent optimization device based on reinforcement learning, including: a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the processor executes the computer program, it implements the communication rate scheme intelligent optimization method based on reinforcement learning as described above.

[0203] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal device, the terminal device executes the above-mentioned reinforcement learning-based intelligent optimization method for communication tariff plans.

[0204] This application dynamically adjusts the merging strategy of tariff plans through a DQN-based reinforcement learning model, which can effectively cope with the complex and changing relationships between packages, thereby significantly improving the flexibility and execution efficiency of the merging strategy. By integrating multi-dimensional features such as the price structure, service content, user behavior and operational attributes of communication tariff plans, this application more comprehensively captures the essential characteristics of tariff plans and achieves more accurate classification and merging. In addition, this application not only reasonably groups non-eliminated plans based on feature similarity, but also designs a pre-merger method based on the merging optimization goal, and uses the reward function to evaluate the optimization effect of each plan, and finally intelligently determines the optimal merging plan, making the entire merging process more efficient and in line with business needs.

[0205] The above is a detailed introduction to the intelligent optimization method for communication tariff schemes based on reinforcement learning and related products provided by this application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0206] It should be understood that in this application, "at least one (item)" means one or more, and "more" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or plural.

[0207] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

Claims

1. A communication tariff intelligent optimization method based on reinforcement learning, characterized in that: Applied to a tariff optimization model, the tariff optimization model is obtained by reinforcement learning training based on a deep Q network (DQN) model, and the method includes: Obtaining tariff fusion features of each communication tariff plan to be processed, and classifying the plurality of communication tariff plans to be processed in combination with the elimination target to obtain N elimination plans and M non-elimination plans; the tariff fusion features are obtained by fusing price structure features, service content features, user behavior features, and operation attribute features of the communication tariff plans; N and M are both non-negative integers; Grouping the non-eliminated plans based on the similarity between the tariff fusion features of the non-eliminated plans to obtain multiple tariff fusion groups; For each tariff merger group, determine a pre-merger mode for the tariff merger in combination with the merger optimization goal, and calculate the optimized reward value under the pre-merger mode using the reward function of the tariff optimization model; If the optimization reward value is greater than or equal to the reward threshold, the pre-merger mode corresponding to the optimization reward value is determined as the target merging mode; the target merging mode includes a merging strategy of some or all of the M non-eliminated solutions; Determine the non-elimination solutions that are not included in any of the target combination methods as the retention solutions; Output each of the target merging methods, the retention plan and the elimination plan.

2. The method according to claim 1, characterized in that The non-eliminated plans are grouped based on the similarity between the tariff fusion features of the non-eliminated plans to obtain multiple tariff fusion groups, including: Calculate the similarity between the tariff integration features of each non-elimination plan; Establish a connection relationship between the non-eliminated solutions corresponding to two tariff fusion features whose similarity is greater than or equal to the similarity threshold; All non-eliminated plans that are directly or indirectly connected through a connection relationship are grouped into the same group using the connectivity principle to obtain the multiple tariff combination groups.

3. The method according to claim 1, characterized in that The training process of the tariff optimization model includes: Initializing the DQN model to obtain an initial model, and obtaining multiple historical communication tariff plans and historical tariff fusion features of each of the historical communication tariff plans; each of the historical communication tariff plans is marked with an optimization method label; Combining the optimization method labels of each historical communication tariff plan, the historical tariff fusion characteristics, the reward function, and the reward weight staged adjustment strategy, the initial model is iteratively reinforced learning trained until the training cutoff condition is reached, thereby obtaining the tariff optimization model; the reward function is used to evaluate the feedback effect of the model on the tariff plan optimization goal, so as to guide the model's decision-making process for optimizing the merger, retention, and elimination of plans; In each reinforcement learning training, P historical communication tariff plans are extracted, and the optimization method labels and historical tariff fusion features of the P historical communication tariff plans are used as model inputs, where P is a positive integer.

4. The method according to claim 3, characterized in that The reward weight adjustment strategy includes dividing the model training rounds into the first stage round, the second stage round and the third stage round, and setting a corresponding reward function weight combination for each stage.

5. The method according to claim 1 or 3, characterized in that The reward function is: R=W simp ·R simp +W user ·R user +W rev ·R rev +W comp ·R comp ; Among them, R simp To streamline the rewards for the package, W simp is the weight of the reward; R user Influence rewards for users, W user is the weight of the reward; R rev is the income change reward, W rev is the weight of the reward; R comp is the operational complexity reward, W comp is the weight of the reward; R simp =γ· , γ is the global effect reward coefficient, N before is the number of communication tariff plans before the merger, N after The number of communication tariff plans after the merger; R rev =ε· , ε is the service improvement sensitivity coefficient, V is the number of users whose service is improved, i is the degree of improvement, W i is the weight of the degree of improvement; R comp =ζ· ζ is the sensitivity coefficient of income change, Revenue before Revenue is the estimated revenue of the communication tariff plan before the merger. after Estimated revenue from the combined communications tariff plan.

6. The method according to claim 1, characterized in that Before outputting the target merging modes, the retention schemes, and the elimination schemes, the method further includes: Combining the target merging method, the retention plan, and the elimination plan, a simulation evaluation is performed on user impact, business impact, and operational impact to obtain a simulation result; If the simulation result meets the merging optimization target, the step of outputting each of the target merging methods, the retention scheme and the elimination scheme is performed.

7. The method according to claim 1, characterized in that The price structure characteristics include basic price characteristics, cost composition characteristics and implicit value characteristics; The service content features include service item coding features, service restriction condition features, and service combination features; The user behavior characteristics include subscription characteristics, usage pattern characteristics and user attribute characteristics; The operation attribute characteristics include time attribute characteristics, channel attribute characteristics and marketing strategy attribute characteristics.

8. A communication tariff intelligent optimization device based on reinforcement learning, characterized in that: The device comprises: A first acquisition unit is configured to acquire a tariff fusion feature of each communication tariff plan to be processed; the tariff fusion feature is obtained by fusing the price structure feature, service content feature, user behavior feature, and operation attribute feature of the communication tariff plan; A classification unit, configured to classify the plurality of communication tariff schemes to be processed in combination with an elimination target to obtain N eliminated schemes and M non-eliminated schemes; N and M are both non-negative integers; a first grouping unit, configured to group the non-eliminated plans based on similarities between their tariff fusion features to obtain a plurality of tariff fusion groups; a determination calculation unit, configured to determine, for each tariff merging group, a pre-merger mode for the tariff merging in combination with a merger optimization objective, and calculate an optimized reward value under the pre-merger mode using a reward function of the tariff optimization model; A first determining unit determines, if the optimization reward value is greater than or equal to a reward threshold, a pre-merger mode corresponding to the optimization reward value as a target merging mode; the target merging mode includes a merging strategy of some or all of the M non-eliminated solutions; A second determining unit is configured to determine non-eliminated solutions that are not included in any of the target merging methods as retained solutions; An output unit is used to output each of the target merging methods, the retention plan and the elimination plan.

9. A communication tariff intelligent optimization device based on reinforcement learning, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for intelligent optimization of communication tariff schemes based on reinforcement learning is implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed on a terminal device, enable the terminal device to execute the communication tariff scheme intelligent optimization method based on reinforcement learning as described in any one of claims 1 to 7.

Images