Proxy service strategy optimization management system and method based on big data

Abstract

The invention discloses a big data-based agency service strategy optimization management system and method, particularly relates to the technical field of agency service strategy optimization, and relates to a big data-based agency service strategy optimization management system and method. The main purpose of the invention is to carry out optimization management on agency services through a big data analysis technology. According to the system, store request data is collected, cleaned, filed and analyzed, so that the network quality can be accurately evaluated, the time period of a high-frequency request is identified, and a resource scheduling strategy is optimized through a multi-dimensional decision framework. Through deep analysis of the store network quality and the service request, the stability and the response speed of the proxy service can be improved, so that the overall service efficiency and the user experience are improved.

Description

Technical Field

[0001] This invention relates to the field of agent service strategy optimization technology, and more specifically, to an agent service strategy optimization management system and method based on big data. Background Technology

[0002] With the rapid development of information technology and the widespread application of big data, proxy services play a crucial role in the operations of various enterprises and stores. As an intermediary connecting users and servers, proxy services not only handle data transmission but also manage tasks such as request routing and load balancing. With the continuous advancement of network technology, the demand for proxy service optimization is also increasing, especially in large-scale distributed network environments. How to effectively manage and optimize proxy services has become a major challenge for the industry.

[0003] In traditional proxy service management, most systems employ fixed configuration strategies and use static rules to adjust service load or allocation. This management approach often lacks flexibility, especially in the face of changing network environments, making it difficult to meet optimization needs in different scenarios. For example, factors such as network latency, packet loss rate, and peak request periods can affect the performance of proxy services, leading to a degraded user experience.

[0004] In existing technologies, multiple stores often share the same carrier network, regional metropolitan area network nodes, and even, in some cases, VPN tunnels or SD-WAN border devices. This structural network overlap can cause multiple stores in a certain area to be affected by similar network fluctuations at the same time, resulting in synchronous interference and affecting the accuracy of strategies and the agility of scheduling. To address these shortcomings, a technical solution is provided. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a big data-based agent service strategy optimization management system and method to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A big data-based agent service strategy optimization management method includes the following steps: First, business request data sent to the headquarters system is collected, and a corresponding unique store identifier ID and store geographical location tag are added to each business request data; a network stability score is calculated based on the store's average response time, the degree of variation in response time, and the connection failure rate. Then, a dynamic network physical path graph is constructed, integrating store public IP ranges, DNS hop counts, and VPN node identifiers, and combining geographical location tags to identify synchronization interference paths caused by shared resources; regional synchronization interference is quantified, including calculating time offset differences, fluctuation amplitude similarity, and Pearson correlation coefficients between stores; and then the weights of regional interference tags are determined. Finally, based on the preset value of business importance, network stability score, and weight of regional interference label, the initial priority score of the request is calculated; the priority of business type and store location information are determined, and the strategy with high similarity is matched in the strategy library as the optimal strategy.

[0007] In a preferred embodiment, the network stability score is calculated by weighted summation of the store's average response time, the variability of response time, and the connection failure rate.

[0008] In a preferred embodiment, the degree of variation in response time, i.e., connection jitter, is measured by the standard deviation of delay between consecutive time points; the connection failure rate is the proportion of requests that fail per unit time.

[0009] In a preferred embodiment, the time offset difference is the time difference between network fluctuations between stores, and the formula is: ,in and It represents the peak latency time for stores i and j.

[0010] In a preferred embodiment, the similarity of fluctuation amplitude is determined by comparing the differences in indicators between stores, using the following formula: },in and This refers to the range of changes in store latency, etc. It represents the maximum fluctuation range within the region.

[0011] In a preferred embodiment, the Pearson correlation coefficient formula is: Where X and Y are time series data from different stores, let the measurement sequence of stores X and Y at continuous sampling points be . and Their mean values ​​are respectively and .

[0012] In a preferred embodiment, when the Pearson correlation coefficient is greater than a system-set threshold, an alarm is triggered and the source of interference is located; then the weight of the regional interference label is determined.

[0013] In a preferred embodiment, the initial priority score of the request is calculated using a weighted summation based on a preset value of business importance, a network stability score, and the weight of the regional interference label.

[0014] In a preferred embodiment, the priority of business type and store location information are determined, and the strategy with the highest similarity in the strategy library is matched as the optimal strategy and sent to each store for execution.

[0015] In a preferred embodiment, it includes the following modules: a data acquisition and preprocessing module, a network assessment and interference diagnosis module, and a strategy decision and execution feedback module; The data acquisition and preprocessing module collects business request data from stores in real time, adds a corresponding unique store identifier ID and store geographical location tag to each request data, and uploads it to the headquarters system through a secure connection; the data received by the headquarters will be cleaned, standardized and archived in a unified manner, and finally construct a business access time series and store usage profile; The network assessment and interference diagnosis module is used to evaluate the network connectivity performance of each store. It calculates the network stability score based on the store's average response time, the degree of variability in response time, and the connection failure rate, and constructs a dynamic network path map based on information such as public IP and VPN nodes. Combining map analysis and time series comparison, the module can monitor in real time whether there is regional synchronous interference, locate the interference source and its potential impact range, and then determine the weight of the regional interference label. The strategy decision-making and execution feedback module calculates the priority of business types based on preset values ​​of business importance, network stability scores, and regional interference labels. Then, it matches the optimal processing strategy from the strategy library and distributes it to each store for execution.

[0016] The technical effects and advantages of this invention are as follows: The present invention proposes a big data-based agent service strategy optimization management system and method, which has the following technical effects and advantages: By leveraging big data technology to collect and analyze business request data from stores, and by calculating average response time, coefficient of variation, and connection failure rate, network stability can be quantified, thereby improving the ability to perceive the real-time status of store networks.

[0017] A dynamic network physical path map is constructed, and information such as public IP, DNS hop count, and VPN nodes are combined to model and analyze the network path structure. Through algorithms such as time offset, fluctuation amplitude similarity, and Pearson correlation coefficient, regional synchronization interference caused by resource sharing is accurately identified.

[0018] The system is equipped with an interference monitoring and early warning mechanism. When the interference index exceeds the threshold, an alarm is automatically triggered. The system also uses graph theory propagation modeling to identify the interference source and the affected area, thereby improving the response efficiency to abnormal events.

[0019] By introducing multi-dimensional factors such as business importance, network stability score, and interference label weight, priority scores are calculated through a weighted model to improve the accuracy and intelligence of policy scheduling.

[0020] The system can automatically match the optimal strategy based on real-time environment and historical strategy information, and push it to the store for execution. It also supports real-time monitoring of feedback indicators such as strategy hit rate and response latency, forming a closed-loop mechanism of "evaluation-execution-feedback-optimization". Attached Figure Description

[0021] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is a flowchart illustrating a big data-based agent service strategy optimization management method according to the present invention. Figure 2 This is a schematic diagram of the structure of a big data-based agent service strategy optimization management system according to the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] This invention first collects business request data sent to the headquarters system, attaching a unique store ID and geographic location tag to each data entry. Based on the store's average response time, response time coefficient of variation, and connection failure rate, a network stability score is calculated. Subsequently, a dynamic network physical path map is constructed, integrating store public IP ranges, DNS hop counts, and VPN node identifiers, and combining geographic location tags to identify synchronization interference paths caused by shared resources. Quantifying regional synchronization interference involves calculating time offset differences between stores, similarity of fluctuation amplitudes, and Pearson correlation coefficients; when this coefficient exceeds a preset threshold, an alarm is triggered and the interference source is located, thereby determining the weight of the regional interference tag. Finally, by combining the preset value of business importance, the network stability score, and the weight of the regional interference tag, an initial request priority score is calculated; based on the business type priority and store location information, the strategy with the highest similarity is matched from the strategy library as the optimal strategy.

[0024] Example 1 This invention provides a method for optimizing and managing agent service strategies based on big data, such as... Figure 1 As shown, it includes the following steps: First, business request data sent to the headquarters system is collected, and a corresponding unique store identifier ID and store geographical location tag are added to each request data; a network stability score is calculated based on the store's average response time, the degree of variation in response time, and the connection failure rate. Then, a dynamic network physical path map is constructed, integrating the public IP range of stores, DNS hop count and VPN node identifier, and combining geographical location tags to identify synchronization interference paths caused by shared resources; regional synchronization interference is quantified, including calculating the time offset difference between stores, the similarity of fluctuation amplitude and Pearson correlation coefficient, and triggering an alarm and locating the interference source when the correlation coefficient exceeds the threshold; and then the weight of regional interference tags is determined. Finally, based on the preset value of business importance, network stability score, and weight of regional interference label, the initial priority score of the request is calculated; the priority of business type and store location information are determined, and the strategy with high similarity is matched in the strategy library as the optimal strategy.

[0025] Specific; First, a lightweight local proxy data collection module is deployed in each store. This module is integrated into the store's network ingress and egress gateway and is responsible for collecting business request data sent to the headquarters system in real time. The collected data includes: the requested business type, request timestamp, request initiating terminal device number, current network latency (measured by pinging the headquarters server), packet loss rate, and access response time.

[0026] Furthermore, the data acquisition module adds a corresponding "store unique identifier ID" and "store geographical location tag" to each requested data to support subsequent regional optimization and scheduling by the headquarters system. All data, after being formatted locally, is uploaded to the headquarters data processing center in batches via a secure connection.

[0027] At the headquarters data processing center, a unified data receiving mechanism is established to continuously receive data requests from various stores via message queues, HTTP interfaces, or scheduled uploads. All received data is first written to a buffer or temporary database, and then undergoes preliminary processing through a data cleaning process. This process includes removing duplicate records, standardizing field formats, completing missing data, standardizing time formats, and screening and processing obviously abnormal values ​​to ensure data consistency and availability.

[0028] After data cleaning, the data will be archived according to store number. The system will divide all request records for each store across different time periods into multiple time segments, typically by hour, half-hour, or day. Subsequently, the system will statistically analyze the access frequency within each time segment, forming a complete access time series. This series can show the fluctuations in request intensity for a particular store over a period of time. To analyze access behavior more deeply, the system further categorizes each request by business type, statistically analyzing the distribution ratio of each type of business request on the timeline, thereby constructing an activity profile of the store in terms of business usage. Simultaneously, based on frequency analysis and time distribution heatmaps, the system will identify typical time periods when high-frequency requests occur, which helps to find periodic patterns in access behavior, such as peak periods like noon and before the end of the workday.

[0029] In terms of network quality assessment, the system extracts key information such as response time and successful connection status for each upload record. Then, it calculates the average response time of the store on an hourly or daily basis; calculates the variability of response time (i.e., connection jitter), typically measured by the standard deviation or variance of latency between consecutive time points; and statistically analyzes the proportion of failed requests per unit time to obtain the connection failure rate. These three indicators collectively reflect the store's network performance in actual operation. For ease of comparison and evaluation, the system standardizes these raw values ​​and then calculates a network stability score according to weighted rules; the specific formula is as follows: N = -(αpj + βby + γbl); where N represents the network stability score; pj represents the store's average response time (a higher average response time results in a lower network stability score, and vice versa); by represents the variability of response time (a higher variability results in a lower network stability score, and vice versa); and bl represents the connection failure rate (a higher connection failure rate results in a lower network stability score, and vice versa). α, β, and γ are the weighting coefficients for the average response time, the variability of response time, and the connection failure rate of the stores, respectively, and are all greater than zero. The weights can be set based on experience or obtained through training with historical data. For example, response time, jitter, and failure rate can each be assigned different importance weights, and then combined into a total score to measure the reliability of the network.

[0030] Multiple stores often share the same carrier network, regional metropolitan area network nodes, and even VPN tunnels or SD-WAN border devices in some cases. This structural network overlap can cause multiple stores in a certain area to be affected by similar network fluctuations at the same time, resulting in "synchronous interference," which affects the accuracy of strategies and the agility of scheduling.

[0031] First, the implementation process begins with building a dynamic network physical path graph. The system obtains information such as public IP ranges, DNS hop counts (measured via TTL tracing technology), and VPN inbound node identifiers from each store's agent data collection module. This information, combined with the store's unique identifier and geographic location tag, is formatted and uploaded to the headquarters data processing center. This information is used to construct the "network physical path graph," abstracting the store's uplinks as nodes (representing stores or network devices) and edges (representing link connections) in a connected graph. The graph is designed with a dynamic update mechanism, refreshing every second or minute to reflect network changes in real time. During updates, the system calculates connectivity weights between nodes (e.g., based on the degree of shared VPN nodes) and automatically identifies whether multiple stores are affected by the same upstream link, ensuring the graph captures potential interference paths caused by shared resources. Key outputs include a queryable, visualized graph database for subsequent analysis.

[0032] Secondly, the system enters a near real-time monitoring and quantification phase to capture and quantitatively assess synchronization interference. It acquires input data streams (sampled per second), including peak latency times, packet loss rate, and jitter volatility, focusing on stores within the same geographical area. The system calculates time offset differences (i.e., the time difference between network fluctuations occurring between stores, using the formula: ,in and (This refers to the peak delay times of stores i and j), and the similarity of fluctuation amplitude is calculated (by comparing the differences in indicators between stores, the formula is: },in and This refers to the range of changes in store latency, etc. (This represents the maximum fluctuation range within the region). Subsequently, a correlation coefficient (such as the Pearson correlation coefficient) is applied for quantification to form the regional synchronicity factor. The formula for the Pearson correlation coefficient is: Where X and Y are time series data from different stores, let the measurement sequence of stores X and Y at continuous sampling points be . and (e.g., delay value), their mean values ​​are respectively and The value of r ranges from -1 to 1, with a value close to 1 indicating a strong positive correlation (high synchronization interference). The system sets a threshold (e.g., r > 0.7) to trigger an abnormal fluctuation alarm, ensuring rapid identification of synchronization events.

[0033] Once abnormal fluctuations are detected (such as r exceeding a threshold or a sudden increase in latency exceeding 50ms), the system identifies potential sources of interference based on graph theory propagation modeling algorithms. This is achieved by combining shortest path propagation analysis with centrality ranking (e.g., proximity centrality formula). Where N is the total number of nodes and d(u,v) is the shortest path distance from node u to v), the system locates high-frequency central nodes (such as shared VPN devices) from the graph. The system predicts the impact of interference on downstream stores: by simulating the propagation path (such as calculating the shortest path), it generates an impact report and estimates the increase in latency or failure rate.

[0034] Finally, after confirming the synchronization interference event, the system marks the affected stores with "regional interference tags"; if a store is marked with a "regional interference tag", different weights for the regional interference tags are assigned according to the degree of interference.

[0035] Based on the preset values ​​of service importance, network stability scores, and the weights of regional interference labels, service type priority is calculated; the formula for prioritizing service types is as follows: R; where, This represents the priority score for business type k. These are preset values ​​based on business importance (e.g., 1.0 for urgent requests like sales revenue uploads, and 0.5 for regular requests like inventory queries; different businesses have different preset values, set by staff as needed); N is the network stability score (range 0-1); R is the weight of the regional interference label (e.g., 0 for high interference labels, and 0 for no interference); w1, w2, and w3 are dynamic weight coefficients (adjusted in real-time based on historical data and server capacity; for example, w1 increases when server load is high to prioritize critical businesses). The engine updates once per second during execution to ensure that business type priority classification is synchronized with the real-time environment and stores the results in the headquarters database for subsequent scheduling.

[0036] The system determines the priority of business types and store location information, and then matches the most similar strategies from the strategy library as the optimal strategy. The strategy library contains information on successfully implemented historical strategies.

[0037] After receiving the policy configuration, the agent module in each store processes locally initiated business requests in real time according to the policy requirements. The policy module also monitors the execution status, recording operational metrics such as policy hit rate, actual latency data, and number of failed retries for subsequent feedback.

[0038] Example 2 This invention discloses a design for a big data-based agent service strategy optimization management system, based on the method in Embodiment 1, specifically as follows: Figure 2 The module shown includes the following modules: data acquisition and preprocessing module, network assessment and interference diagnosis module, and strategy decision-making and execution feedback module; The data acquisition and preprocessing module collects business request data from stores in real time, adds a corresponding unique store identifier ID and store geographical location tag to each request data, and uploads it to the headquarters system through a secure connection; the data received by the headquarters will be cleaned, standardized and archived in a unified manner, and finally construct a business access time series and store usage profile; The network assessment and interference diagnosis module is used to evaluate the network connectivity performance of each store. It calculates the network stability score based on the store's average response time, the degree of variability in response time, and the connection failure rate, and constructs a dynamic network path map based on information such as public IP and VPN nodes. Combining map analysis and time series comparison, the module can monitor in real time whether there is regional synchronous interference, locate the interference source and its potential impact range, and then determine the weight of the regional interference label. The strategy decision-making and execution feedback module calculates the priority of business types based on preset values ​​of business importance, network stability scores, and regional interference labels. It then matches the optimal processing strategy from the strategy library and distributes it to the agent modules of each store for execution. At the same time, this module monitors the strategy execution results in real time, including indicators such as response latency, hit rate, and retry rate.

[0039] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0040] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0041] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0042] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0043] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for optimizing and managing agent service strategies based on big data, characterized in that, Includes the following steps: First, business request data sent to the headquarters system is collected, and a corresponding unique store identifier ID and store geographical location tag are added to each business request data; a network stability score is calculated based on the store's average response time, the degree of variation in response time, and the connection failure rate. Then, a dynamic network physical path graph is constructed, integrating store public IP ranges, DNS hop counts, and VPN node identifiers, and combining geographical location tags to identify synchronization interference paths caused by shared resources; regional synchronization interference is quantified, including calculating time offset differences, fluctuation amplitude similarity, and Pearson correlation coefficients between stores; and then the weights of regional interference tags are determined. Finally, based on the preset value of business importance, network stability score, and weight of regional interference label, the initial priority score of the request is calculated; Determine the priority of business types and store location information, and match the strategy with high similarity from the strategy library as the optimal strategy.

2. The method for optimizing and managing agent service strategies based on big data according to claim 1, characterized in that: The network stability score is calculated by weighted summation of the store's average response time, the variability of response time, and the connection failure rate.

3. The method for optimizing and managing agent service strategies based on big data according to claim 2, characterized in that: The degree of variation in response time, i.e., connection jitter, is measured by the standard deviation of delay between consecutive time points; the connection failure rate is the proportion of requests that fail per unit time.

4. The method for optimizing and managing agent service strategies based on big data according to claim 1, characterized in that: The time offset difference refers to the time difference in which network fluctuations occur between stores, and the formula is: ,in and It represents the peak latency time for stores i and j.

5. The method for optimizing and managing agent service strategies based on big data according to claim 1, characterized in that: The similarity of fluctuation amplitude is determined by comparing the differences in indicators between stores, and the formula is as follows: },in and This refers to the range of changes in store latency, etc. It represents the maximum fluctuation range within the region.

6. The method for optimizing and managing agent service strategies based on big data according to claim 1, characterized in that: The formula for the Pearson correlation coefficient is: Where X and Y are time series data from different stores, let the measurement sequence of stores X and Y at continuous sampling points be . and Their mean values ​​are respectively and .

7. The method for optimizing and managing agent service strategies based on big data according to claim 6, characterized in that: When the Pearson correlation coefficient is greater than the system's set threshold, an alarm is triggered and the source of interference is located; then the weight of the regional interference label is determined.

8. The method for optimizing and managing agent service strategies based on big data according to claim 7, characterized in that: The initial priority score of a request is calculated using a weighted summation based on a preset value for business importance, a network stability score, and the weight of a regional interference label.

9. The method for optimizing and managing agent service strategies based on big data according to claim 8, characterized in that: Determine the priority of business types and store location information, match the strategy with the highest similarity in the strategy library as the optimal strategy, and distribute it to each store for execution.

10. A big data-based agent service strategy optimization management system, characterized in that, The management system, based on the method described in any one of claims 1-9, includes the following modules: a data acquisition and preprocessing module, a network assessment and interference diagnosis module, and a strategy decision-making and execution feedback module; The data acquisition and preprocessing module collects business request data from stores in real time and adds a corresponding unique store identifier ID and store geographical location tag to each request data; And upload it to the headquarters system via a secure connection; The network assessment and interference diagnosis module is used to evaluate the network connectivity performance of each store. It calculates the network stability score based on the store's average response time, the degree of variability in response time, and the connection failure rate, and constructs a dynamic network path map based on information such as public IP and VPN nodes. Combining map analysis and time series comparison, the module can monitor in real time whether there is regional synchronous interference, locate the interference source and its potential impact range, and then determine the weight of the regional interference label. The strategy decision-making and execution feedback module calculates the priority of business types based on preset values ​​of business importance, network stability scores, and regional interference labels. Then, it matches the optimal processing strategy from the strategy library and distributes it to each store for execution.

Images