An electronic mall data communication method and system

Abstract

The application provides an electronic mall data communication method and system, and relates to the technical field of electronic mall data communication. The method comprises the following steps: based on data grading rules, value level evaluation is performed on operation data generated by a local data generation point; a transmission channel and a transmission strategy are determined according to the value level, and the operation data is transmitted to a central data processing point; the central data processing point performs system state analysis on the operation data, and the analysis result is transmitted back to a transmission scheduling node, so that the transmission scheduling node forwards the analysis result to corresponding system components through the transmission channel, thereby driving the system components to respond. The method of the application aims to solve the problem of low communication efficiency, network congestion and key data transmission delay caused by the unified data communication strategy in the prior art in the complex scene of the electronic mall system, improve the overall stability and response speed of the electronic mall system, and ensure that the analysis result can timely drive the feedback and action inside the system.

Description

Technical Field

[0001] This invention relates to the field of e-commerce data communication technology, and more specifically, to an e-commerce data communication method and system. Background Technology

[0002] As a core component of modern commerce, the efficient operation of an e-commerce system relies on a complex and massive data communication network. With the continuous growth in user traffic and increasing business complexity, the data communication volume within e-commerce systems is experiencing explosive growth. Every user action, from simple page browsing to complex transaction completion, involves data exchange between multiple system components. Furthermore, the system generates massive amounts of operational data during runtime, which are key indicators for evaluating system performance and health, such as server response latency, error rate, database query time, and resource utilization.

[0003] To ensure the stable operation and timely response of the e-commerce system, it is necessary to collect and transmit operational data from multiple nodes throughout the system at a high granularity and frequency for detailed system status analysis and rapid identification of potential problems. However, such high-volume, high-granularity operational data communication and transmission undoubtedly places a huge network traffic and processing burden on the system infrastructure. This significant communication burden may not only affect the performance of the e-commerce's core functions but may also lead to delays in the transmission of critical operational data, thereby impacting the timeliness and accuracy of system status analysis.

[0004] In existing technologies, e-commerce systems typically employ a uniform data communication strategy, transmitting all operational data indiscriminately, regardless of its importance or urgency. This indiscriminate transmission method causes high-priority, low-latency critical operational data to share limited communication resources with low-priority, high-tolerance data, resulting in low communication efficiency. When the system faces high concurrency or sudden traffic surges, this uniform transmission strategy is more likely to cause network congestion, further exacerbating the latency of critical data transmission and thus affecting the system's ability to provide early warning and rapid response to potential problems.

[0005] There is currently no effective technical solution to the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide an e-commerce data communication method and system, which aims to solve the problems of low communication efficiency, network congestion, and delays in key data transmission caused by the unified data communication strategy in the existing technology under the complex scenarios of continuous growth in user visits, huge volume of operational data, and the need for high-granularity and high-frequency collection and transmission. This invention improves the overall stability and response speed of the e-commerce system and ensures that the analysis results can drive the internal feedback and actions of the system in a timely manner.

[0007] In a first aspect, the present invention provides an e-commerce data communication method, which is applied to an e-commerce data communication system, the e-commerce data communication system including a local data generation node, a transmission scheduling node and a central data processing node;

[0008] The data communication method for e-commerce platforms includes the following steps:

[0009] S1. After deploying local evaluation units at each local data generation point of the e-commerce system, control the local data generation point, and based on the preset data classification rules, evaluate the value level of the operational data generated by the local data generation point through the corresponding local evaluation unit to obtain the value level of the operational data, and package the value level together with the corresponding operational data into bundled data and send it to the transmission scheduling node.

[0010] S2. After receiving the bundled data, the transmission scheduling node controls the transmission scheduling node to determine the transmission channel and transmission strategy based on the value level of the operational data, and transmits the corresponding operational data to the central data processing point through the determined transmission channel based on the determined transmission strategy.

[0011] S3. After receiving the operational data from the transmission channel, the central data processing point controls the central data processing point to perform system status analysis on the operational data, obtain the first analysis result, and send the first analysis result back to the transmission scheduling node so that the transmission scheduling node forwards it to the corresponding system components through the transmission channel, thereby driving the system components to respond.

[0012] The e-commerce data communication method provided by this invention effectively solves the problems of low communication efficiency, network congestion, and delays in critical data transmission caused by a unified data communication strategy in complex scenarios such as continuously increasing user traffic, massive operational data volumes, and the need for high-granularity, high-frequency collection and transmission in existing e-commerce systems. This method significantly reduces network traffic and processing resource consumption for data communication transmission, while ensuring the timeliness and integrity of critical operational data to support real-time analysis and management of system status. It also accelerates the efficiency of analysis results driving internal system feedback and actions, ultimately improving the overall stability and response speed of the e-commerce system. This ensures that analysis results can promptly drive internal system feedback and actions, thus overcoming the shortcomings of low communication efficiency and slow system response caused by the lack of differentiation in transmission methods in existing technologies.

[0013] Secondly, the present invention provides an e-commerce data communication system, including a local data generation node, a transmission scheduling node and a central data processing node;

[0014] Also includes:

[0015] The first control module is used to deploy local evaluation units at each local data generation point of the e-commerce system, control the local data generation point, evaluate the value level of the operation data generated by the local data generation point based on the preset data classification rules, obtain the value level of the operation data, and package the value level together with the corresponding operation data into bundled data and send it to the transmission scheduling node.

[0016] The second control module is used to control the transmission scheduling node after receiving the bundled data, determine the transmission channel and transmission strategy according to the value level of the operational data, and transmit the corresponding operational data to the central data processing point through the determined transmission channel based on the determined transmission strategy.

[0017] The third control module is used to control the central data processing point to perform system status analysis on the operational data after receiving the operational data from the transmission channel, obtain the first analysis result, and send the first analysis result back to the transmission scheduling node so that the transmission scheduling node forwards it to the corresponding system components through the transmission channel, thereby driving the system components to respond.

[0018] As can be seen from the above, the e-commerce data communication method provided by this invention achieves intelligent classification of massive operational data by deploying local evaluation units at various local data generation points in the e-commerce system and evaluating the value level of operational data based on preset data classification rules. Subsequently, the transmission scheduling node determines the transmission channel and transmission strategy according to the value level of the data, and transmits data of different priorities to the central data processing point through the corresponding channel. After receiving the data, the central data processing point performs system status analysis and sends back the analysis results to drive the system components to respond. The core of this method is that it no longer transmits the massive operational data generated by the e-commerce system indiscriminately, but endows the data with "intelligence," allowing the data itself to carry information about its importance and urgency. By performing "value assessment" on the data at the data generation source and dynamically allocating different "transmission channels" and "transmission strategies" to data of different importance levels according to the assessment results, refined management of communication resources is achieved. This hierarchical transmission mechanism ensures that the most critical and urgent system status information can be delivered to the analysis system at the fastest speed and with the highest reliability, while secondary or non-urgent data can be transmitted in a more resource-efficient manner, and even be flexibly processed in extreme cases. This significantly reduces the network traffic and processing burden of overall data communication while ensuring stable system operation and timely response.

[0019] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0020] Figure 1 A flowchart illustrating an e-commerce data communication method provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of an electronic shopping mall data communication system provided in an embodiment of the present invention.

[0022] Label Explanation:

[0023] 100. First control module; 200. Second control module; 300. Third control module. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0025] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] Reference Appendix Figure 1 This invention provides an e-commerce data communication method, which is applied to an e-commerce data communication system. The e-commerce data communication system includes a local data generation node, a transmission scheduling node, and a central data processing node.

[0027] The data communication method for e-commerce platforms includes the following steps:

[0028] S1. After deploying local evaluation units at each local data generation point in the e-commerce system, control the local data generation point to evaluate the value level of the operational data generated by the local data generation point based on preset data classification rules, and obtain the value level of the operational data. Then, package the value level along with the corresponding operational data into bundled data and send it to the transmission scheduling node. The local data generation point is specifically a front-end server, back-end microservice, database proxy layer, or network device. The local evaluation unit is specifically an independent process, a proxy component in a service mesh, or a log collection function module in an application. The data classification rules are used to divide the operational data into high-priority data, medium-priority data, and low-priority data.

[0029] S2. After receiving the bundled data, the transmission scheduling node controls the transmission scheduling node to determine the transmission channel and transmission strategy based on the value level of the operational data. Based on the determined transmission strategy, the corresponding operational data is transmitted to the central data processing point through the determined transmission channel. The transmission channel includes a priority transmission channel for transmitting high-priority data, a standard transmission channel for transmitting medium-priority data, and an optimized transmission channel for transmitting low-priority data.

[0030] S3. After receiving operational data from different levels of transmission channels at the central data processing point, the central data processing point is controlled to perform system status analysis on the operational data, obtain the first analysis result, and send the first analysis result back to the transmission scheduling node so that the transmission scheduling node forwards it to the corresponding system components through the transmission channel, thereby driving the system components to respond.

[0031] The core technical concept of this solution lies in its departure from treating the massive amounts of operational data generated by the e-commerce system indiscriminately during transmission. Instead, it endows the data with "intelligence," allowing it to carry information about its importance and urgency. By conducting "value assessment" of the data at its source and dynamically allocating different "transmission channels" and "transmission strategies" to data of varying importance based on the assessment results, it achieves refined management of communication resources. This hierarchical transmission mechanism ensures that the most critical and urgent system status information is delivered to the analysis system with the fastest speed and highest reliability, while less important or non-urgent data can be transmitted in a more resource-efficient manner, and even flexibly handled in extreme cases. This significantly reduces the overall network traffic and processing burden of data communication while ensuring stable system operation and timely response.

[0032] This invention aims to effectively reduce network traffic and processing resource consumption in complex scenarios such as continuously increasing user traffic, massive operational data volumes, and the need for high-granularity, high-frequency data collection and transmission in e-commerce systems. Simultaneously, it ensures the timeliness and integrity of key operational data to support real-time system status analysis and management, and accelerates the efficiency of analysis results driving internal system feedback and action. Ultimately, it improves the overall stability and response speed of the e-commerce system, ensuring that analysis results can promptly drive internal system feedback and action.

[0033] The e-commerce data communication method proposed in this application aims to optimize the communication efficiency and resource utilization of e-commerce systems when processing massive amounts of operational data. The e-commerce data communication system is a distributed system, typically composed of multiple cooperating components, including local data generation nodes, transmission scheduling nodes, and a central data processing node. Local data generation nodes are the source of data generation, such as front-end servers, back-end microservices, database proxy layers, or network devices. Local evaluation units are software or hardware modules deployed on local data generation nodes, responsible for preliminary value assessment of operational data. Data classification rules are a pre-defined logical set used to divide operational data into high-priority, medium-priority, and low-priority data. The transmission scheduling node is responsible for receiving bundled data from local data generation nodes and determining the transmission path and strategy based on the data's value level. Transmission channels are the logical or physical paths for data transmission, including priority transmission channels, standard transmission channels, and optimized transmission channels. The central data processing node is the core component for receiving and analyzing operational data; it is typically a distributed stream processing system responsible for performing system state analysis on the data and generating initial analysis results.

[0034] The e-commerce data communication method of this application achieves refined management of e-commerce system operation data by introducing a data value level assessment and hierarchical transmission mechanism.

[0035] In step S1, local evaluation units are first deployed at each local data generation point of the e-commerce system. Local data generation points can be front-end servers, back-end microservices, database proxy layers, or network devices. A local evaluation unit can be an independent process, such as a standalone Python script or Java application, running on the local data generation node and responsible for monitoring and processing the operational data generated by that node. Alternatively, the local evaluation unit can be a proxy component in a service mesh, such as the Sidecar proxy in Istio or Linkerd, which intercepts and processes all network traffic entering and leaving the local data generation node and extracts operational data for evaluation. Furthermore, the local evaluation unit can also be a log collection module within the application, such as a module integrated into logging frameworks like Log4j or SLF4j, directly evaluating the value of the generated log data within the application.

[0036] After deploying the local evaluation unit, the local data generation point is controlled. Based on preset data classification rules, the corresponding local evaluation unit evaluates the value level of the operational data generated by the local data generation point to obtain the value level of the operational data. The data classification rules can be pre-configured in the local evaluation unit. For example, they can be defined as follows: when operational data indicates payment failure or order creation failure, it is evaluated as high-priority data; when operational data indicates user login anomalies or product inventory warnings, it is evaluated as medium-priority data; and when operational data indicates page views or ordinary log information, it is evaluated as low-priority data. After the evaluation is completed, the value level and the corresponding operational data are packaged into bundled data and sent to the transmission scheduling node. For example, a front-end server generates a log entry about payment failure. The local evaluation unit evaluates it as high-priority data according to the data classification rules, then encapsulates the "high-priority" label together with the log entry and sends it to the transmission scheduling node via the HTTP / 2 protocol.

[0037] In step S2, after the transmission scheduling node receives the bundled data, it determines the transmission channel and transmission strategy based on the value level of the operational data. For example, if the operational data in the received bundle is assessed as high priority, the transmission scheduling node selects a priority transmission channel and employs a low-latency, high-reliability transmission strategy, such as using TCP long connections and retransmission mechanisms. If the operational data is assessed as medium priority, a standard transmission channel is selected, and a transmission strategy balancing latency and throughput is adopted, such as using the HTTP / 1.1 protocol. If the operational data is assessed as low priority, an optimized transmission channel is selected, and a high-throughput, low-resource-consumption transmission strategy is adopted, such as using the UDP protocol or bulk transmission. Subsequently, based on the determined transmission strategy, the corresponding operational data is transmitted to the central data processing point through the determined transmission channel. For example, for high-priority data, the transmission scheduling node immediately sends it to the central data processing point through the priority transmission channel to ensure its rapid arrival.

[0038] In step S3, after the central data processing point receives operational data from different levels of transmission channels, it controls the central data processing point to perform system status analysis on the operational data and obtain a first analysis result. For example, if the central data processing point receives payment failure data from a priority transmission channel, it will immediately analyze it to determine whether there is a large-scale payment anomaly and generate corresponding alarm information as the first analysis result. Subsequently, the first analysis result is sent back to the transmission scheduling node, so that the transmission scheduling node forwards it to the corresponding system components through the transmission channel, thereby driving the system components to respond. For example, after receiving the alarm information of payment anomaly, the transmission scheduling node will forward it to the payment service component, which may automatically trigger rate limiting or circuit breaking mechanisms to prevent the problem from escalating further.

[0039] The e-commerce data communication method of this application achieves refined management of communication resources by assessing the value level of operational data at the data generation source and dynamically allocating transmission channels and strategies based on the assessment results. This hierarchical transmission mechanism ensures that the most critical and urgent system status information can be delivered to the analysis system with the fastest speed and highest reliability, while less important or non-urgent data can be transmitted in a more resource-efficient manner.

[0040] Compared to the uniform transmission strategy that treats all operational data equally in existing technologies, the core innovation of this application lies in the introduction of a "data value assessment" and "tiered transmission" mechanism. Traditional methods can easily cause network congestion when e-commerce systems face high concurrency or sudden traffic surges, leading to delays in the transmission of critical data and affecting the system's ability to provide early warnings and rapid responses to potential problems.

[0041] This application achieves source assessment of operational data by deploying local evaluation units at local data generation nodes, endowing the data with "intelligence" and enabling it to carry information about its importance and urgency. For example, a payment failure event will be immediately marked as high priority, while a regular webpage browsing event may be marked as low priority. This differentiated processing allows transmission scheduling nodes to dynamically select the most appropriate transmission channel and strategy based on the value level of the data. High-priority data can be transmitted through priority transmission channels in a low-latency, high-reliability manner, ensuring that critical system status information can be delivered to the central data processing point for analysis in a timely manner. Low-priority data, on the other hand, can be transmitted in a more resource-efficient manner by optimizing transmission channels, thereby effectively reducing the overall network traffic and processing burden of data communication.

[0042] Therefore, this application not only solves the problems of low communication efficiency and network congestion caused by the unified transmission strategy in the prior art, but also significantly improves the timeliness and completeness of key operational data, thereby supporting real-time analysis and management of system status. Furthermore, by accelerating the efficiency of analysis results driving internal system feedback and actions, this application ultimately improves the overall stability and response speed of the e-commerce system, ensuring that analysis results can promptly drive internal system feedback and actions, providing a strong guarantee for the efficient and stable operation of the e-commerce system in complex operational scenarios.

[0043] In some embodiments, step S1, which involves evaluating the value level of operational data generated by a local data generation point based on preset data grading rules using a corresponding local evaluation unit, to obtain the value level of the operational data, includes:

[0044] S11. Through the local evaluation unit, based on the preset data classification rules, the value level of the operational data generated by the local data generation point is evaluated to obtain the initial value level of the operational data;

[0045] S12. Periodically obtain service identifiers that are dependent on the local data generation point;

[0046] S13. Based on service identifiers, requests are sent to the corresponding neighboring services to obtain aggregated status reports of the neighboring services. This aims to solve the technical problem that when the service dependencies of an e-commerce system change dynamically, outdated or inaccurate aggregated status reports of neighboring services may be obtained due to changes in dependencies, leading to biases in the judgment of common performance degradation trends. By proactively and periodically discovering the current service dependencies and obtaining the latest neighboring service status reports based on this, the system ensures that the data used for judgment is real-time and accurate, thereby achieving reliable judgment of common performance degradation trends in dynamic environments.

[0047] S14. Determine whether the initial value level of the operational data is medium priority;

[0048] S15. If the initial value level of the operational data is medium priority, then based on the operational data generated by the local data generation point and the aggregated status summary of the neighboring services, determine whether there is a common performance decline trend between the local data generation point and the neighboring services. If there is a common performance decline trend, the value level of the operational data will be upgraded to high priority; otherwise, the value level of the operational data will remain unchanged.

[0049] This technology aims to address the challenge of identifying and prioritizing the transmission of early warning signals—aggregated from multiple medium-priority events—that foreshadow a system-wide crisis in e-commerce systems when faced with operational data streams generated by numerous distributed components. These data streams, individually classified as medium-priority but collectively posing a progressive risk of systemic performance degradation, lack a global perspective and cross-component correlation information. The core concept is to, after initial value assessment, determine the existence of aggregated risks for medium-priority data by acquiring local correlation information and combining it with its own data. Based on this, the value level of the data is adjusted to achieve earlier warnings.

[0050] Specifically, in step S11, after receiving the operational data generated by the local data generation point, the local evaluation unit will make a preliminary judgment on it according to preset data classification rules. For example, based on the data type, content, source, or predefined importance tags, it will classify the data into high-priority, medium-priority, or low-priority data, thereby obtaining the initial value level of the operational data. For example, user payment failure logs may be directly judged as high priority, while ordinary user browsing behavior logs may be judged as low priority, and a slight increase in the response time of certain specific services may be judged as medium priority.

[0051] Step S12 aims to ensure that the local evaluation unit can perceive the dynamic changes in the service ecosystem in which it operates. A service identifier can be understood as a name or ID that uniquely identifies a service instance or service group. Periodically acquiring service identifiers that are dependent on the local data generation point means that the local evaluation unit will periodically query the service registry, configuration management system, or through service discovery mechanisms to discover upstream or downstream services that it directly or indirectly depends on. This periodic acquisition mechanism ensures that even if the service topology changes, the local evaluation unit can update its understanding of neighboring services in a timely manner.

[0052] In practical applications, in step S13, based on the acquired service identifiers, the local evaluation unit proactively sends requests to these neighboring services to obtain their aggregated state summaries. These aggregated state summaries may include key performance indicators (KPIs) of the neighboring services, such as average response time, error rate, throughput, and resource utilization. These indicators are typically summary information after aggregation processing by the neighboring services themselves. In this way, the local evaluation unit does not need to obtain all the raw data from the neighboring services, but instead obtains a lightweight but representative state snapshot, thereby reducing communication overhead.

[0053] Furthermore, steps S14 and S15 constitute a secondary evaluation mechanism for medium-priority data. When the initial value level of operational data is determined to be medium priority, the local evaluation unit combines the operational data generated by the local data generation point with the aggregated status summary of neighboring services to determine whether there is a common performance degradation trend between the local data generation point and neighboring services. A common performance degradation trend means that although the performance degradation of a single service or a single data point may not be sufficient to trigger a high-priority alarm, when multiple interdependent services simultaneously experience slight but persistent performance degradation, this may indicate a broader systemic problem. If such a common performance degradation trend exists, the value level of the operational data will be upgraded to high priority to ensure that it can be processed in a timely manner through the priority transmission channel; otherwise, its value level remains unchanged.

[0054] This application's solution, after initially assessing the value of operational data at the local data generation point, introduces a dynamic perception and aggregation risk judgment mechanism for the status of neighboring services for data with an initial value level of medium priority. This solves the problem in traditional solutions of failing to effectively identify early systemic risks aggregated from multiple sources of medium-priority events. Specifically, by periodically acquiring service identifiers that are dependent on the local data generation point, the local assessment unit can monitor the dynamic changes in the service topology in real time, avoiding judgment bias due to outdated dependencies. Subsequently, based on these service identifiers, by sending requests to the corresponding neighboring services and obtaining their aggregation status summaries, the local assessment unit can obtain local correlation information in a lightweight manner, thus providing necessary data support for subsequent risk judgment. When it is determined that the local data generation point and neighboring services share a common performance degradation trend, even if the performance degradation of a single data point or service is not significant, its value level will be upgraded to high priority. This mechanism enables potential systemic problems to be identified and flagged earlier at the source of data generation, ensuring that these critical early warning signals are transmitted and processed in a timely manner through priority transmission channels, thus avoiding response delays caused by insufficient data priority.

[0055] Through the above technical solution, this application effectively solves the technical problem that when the service dependencies of an e-commerce system change dynamically, outdated or inaccurate aggregated status reports of neighboring services may be obtained due to changes in dependencies, leading to biased judgments of common performance decline trends. Furthermore, this application also addresses the technical problem of how to identify and prioritize the transmission of early signals—aggregated from multiple sources of medium-priority events—that foreshadow a system-wide crisis when the e-commerce system faces operational data streams generated by numerous distributed components, individually classified as medium-priority but collectively constituting a progressive systemic performance decline risk, in the absence of a global perspective and cross-component correlation information. This enables earlier warnings and responses than centralized analysis. By performing a secondary evaluation of medium-priority data at the data generation source and combining it with the aggregated status of neighboring services, this application can more accurately and timely identify potential systemic risks, avoiding missed or false alarms caused by insufficient evaluation of a single data point. This significantly improves the adaptability and early warning capabilities of the e-commerce system to complex dynamic environments, ensuring the timeliness of key operational data and accelerating the efficiency of analysis results driving internal system feedback and action.

[0056] Example description:

[0057] Assume the e-commerce system includes order services, payment services, and inventory services. Each service has a locally deployed evaluation unit.

[0058] S11. The local evaluation unit of the order service monitors the response time of order creation requests in real time. According to preset rules, if the response time is between milliseconds and 500 milliseconds, the corresponding operational data (such as "order creation response time log") is evaluated as "medium priority".

[0059] S12 and S13. The nearest-neighbor state awareness module in the local evaluation unit of the order service periodically sends requests to the user service and inventory service it depends on to obtain their aggregated state briefings. For example, it obtains a briefing from the user service that says "the user login API response latency has increased by 15% in the last 30 seconds" and from the inventory service that says "the inventory query API error rate has increased by 10% in the last minute".

[0060] S14. The local evaluation unit of the order service determines the initial value level of the current "Order Creation Response Time Log" as "Medium Priority".

[0061] S15. Since the "Order Creation Response Time Log" is of medium priority, the local evaluation unit further combines its own data (order creation response time is 350 milliseconds) and neighboring service reports (user service latency increased by 15%, inventory service error rate increased by 10%). Based on the preset aggregation judgment logic (e.g., if it is of medium priority and at least two neighboring service performance indicators have both decreased to a moderate degree), it determines that there is a common performance decline trend. At this time, the local evaluation unit of the order service upgrades the value level of the "Order Creation Response Time Log" to "high priority". Subsequently, this high-priority log will be sent to the central data processing node through the priority transmission channel.

[0062] In some embodiments, step S15, which involves determining whether there is a common performance degradation trend between the local data generation point and the neighboring services based on the operational data generated by the local data generation point and the aggregated status summary of the acquired neighboring services, includes:

[0063] S151. Assign weights to performance metrics in operational data and aggregated status briefings;

[0064] S152. Calculate the aggregation risk score based on the weights assigned to the performance indicators;

[0065] S153. Compare the aggregated risk score with a first preset threshold;

[0066] S154. When the aggregated risk score exceeds the first preset threshold, it is determined that the local data generation point and the neighboring service have a common performance decline trend.

[0067] Specifically, in step S151, assigning weights to performance metrics in operational data and aggregated status reports means assigning different weight values ​​to performance metrics (such as response time, error rate, throughput, etc.) contained in operational data generated by local data generation points and performance metrics obtained from aggregated status reports of neighboring services, based on their importance to overall system performance and user experience. For example, for payment services, payment success rate and response time may be assigned higher weights, while log recording volume may be assigned lower weights. These weights can be preset and dynamically adjusted based on business needs, historical data analysis, or expert experience.

[0068] In step S152, the aggregated risk score is calculated based on the weights assigned to the performance metrics. This can be understood as comparing the current values ​​of various performance metrics of the local data generation point and neighboring services with their respective baseline values ​​or normal ranges, quantifying their deviations, and then multiplying and summing these deviations by pre-assigned weights to obtain a comprehensive aggregated risk score. For example, if a performance metric (such as response time) exceeds its normal range, the greater the deviation, the greater the corresponding risk contribution. A higher aggregated risk score indicates a greater likelihood of a shared performance decline trend between the local data generation point and neighboring services, and a higher potential risk this trend poses to the system.

[0069] In practical applications, step S153, comparing the aggregated risk score with a first preset threshold, refers to comparing the calculated aggregated risk score with a pre-set critical value used to define a "common performance decline trend." This first preset threshold can be set based on factors such as the system's historical operating data, business sensitivity, and acceptable risk levels. For example, a threshold can be set whereby a significant common performance decline risk is considered to exist when the aggregated risk score reaches or exceeds this threshold.

[0070] Furthermore, in step S154, when the aggregated risk score exceeds a first preset threshold, it is determined that the local data generation point and its neighboring services share a common performance degradation trend. This means that through quantification, when the cumulative risk of multiple related performance indicators reaches a certain level, the system can objectively identify this common, potential performance degradation trend, thereby triggering subsequent priority enhancement operations.

[0071] This application's solution addresses the potential ambiguity and subjectivity in determining whether a local data generation point and its neighboring services share a common performance degradation trend by introducing a quantified weighting allocation and aggregated risk score calculation mechanism. Specifically, weights are assigned to performance indicators in operational data and aggregated status reports, ensuring that indicators of varying importance are appropriately reflected in risk assessment and avoiding a "one-size-fits-all" approach. Subsequently, an aggregated risk score is calculated based on these weights, aggregating multiple dispersed performance indicator changes into a unified, quantifiable risk metric. This provides a more comprehensive and objective reflection of the overall health of the local data generation point and its neighboring services. Finally, by comparing this aggregated risk score with a first preset threshold, a clear judgment standard is provided. When the risk score exceeds this threshold, it indicates a significant common performance degradation trend, enabling timely and accurate identification of potential systemic risks. It is precisely this quantified and standardized judgment process that makes the identification of common performance degradation trends more accurate and reliable.

[0072] Through the above technical solution, this application provides a more accurate, objective, and quantifiable method to determine whether there is a common performance degradation trend between local data generation points and neighboring services. This mechanism, based on weight allocation and aggregated risk score calculation, effectively avoids the ambiguity and subjectivity that may exist in traditional judgment methods, significantly improving the accuracy and reliability of identifying potential systemic risks. Therefore, it can identify, earlier and more accurately, those medium-priority data streams that, while individual events may not be urgent, collectively indicate a systemic crisis. This ensures that critical early warning information can be promptly elevated to high priority and transmitted, providing earlier warning signals to the central data processing point, thereby accelerating the system's response to and handling of potential problems, and ultimately improving the overall stability and resilience of the e-commerce system.

[0073] Example description:

[0074] Assuming the payment service (local data generation point) in the e-commerce system is running, its local evaluation unit needs to determine whether the payment service and its dependent order service and user service (nearby service) have a common performance degradation trend.

[0075] S151. The local evaluation unit configures a weight table. For example, for the payment service's own operational data: API response latency has a weight of 0.4, error rate has a weight of 0.3, and CPU utilization has a weight of 0.1. For aggregated status reports obtained from the order service: API latency increase percentage has a weight of 0.1, and error rate increase percentage has a weight of 0.05. For aggregated status reports obtained from the user service: request queue growth percentage has a weight of 0.05. These weights are distributed through the configuration management service.

[0076] S152. Assume the current payment service's API response latency is 400 milliseconds (medium priority), the error rate is 1%, and the CPU utilization is 60%. Briefings from the order service show a 20% increase in API latency and an increase in the error rate. Briefings from the user service show a 15% increase in the request queue.

[0077] The local assessment unit standardizes these metrics (e.g., mapping latency, error rate, etc., to risk values ​​of 0-100), and then calculates an aggregated risk score based on weights.

[0078] Aggregate risk score = (Payment service API latency risk value * 0.4) + (Payment service error rate risk value * 0.3) + (Payment service CPU utilization risk value * 0.1) + (Order service API latency increase risk value * 0.1) + (Order service error rate increase risk value * 0.05) + (User service request queue growth risk value * 0.05).

[0079] S153. The local assessment unit compares the calculated aggregate risk score (e.g., assuming 75) with a preset first threshold (e.g., 60).

[0080] S154. Since the calculated aggregate risk score of 75 exceeds the first threshold of 60, the local evaluation unit determines that the payment service, order service, and user service share a common performance decline trend. At this time, the operational data generated by the payment service (even if its own indicators have not yet reached the high-priority threshold) will be prioritized for transmission.

[0081] In some embodiments, the central data processing point is specifically a distributed stream processing system; the distributed stream processing system internally maintains a rule engine, which includes an average response time threshold for the payment service API and an alarm threshold for the payment failure rate;

[0082] In step S3, the specific steps for controlling the central data processing point to perform system status analysis on operational data and obtain the first analysis result include:

[0083] S31. Control the distributed stream processing system to monitor the promotional activities of the e-commerce system through subscription services, and when it is determined that there are promotional activities in the e-commerce system, obtain the adjusted rule engine by increasing the average response time threshold of the payment service API and reducing the alarm threshold of the payment failure rate.

[0084] S32. Utilize the adjusted rules engine to analyze and determine whether there are any payment anomalies in the operational data;

[0085] S33. If payment anomalies are found, output the first analysis result related to the operational scenario;

[0086] This solution aims to address the technical challenge of static system state analysis methods failing to adapt to dynamic changes in payment service scope, risk thresholds, and dependencies between system components during the operation of e-commerce systems, which can occur due to variations in promotional activities. This results in reduced accuracy and relevance of early warnings. The core concept is to dynamically adjust the logic and parameters of system state analysis by acquiring and utilizing the operational context information of the e-commerce system in real time. This allows the analysis process to adapt to the current operational scenario, ensuring the accuracy and relevance of the analysis results.

[0087] Specifically, the central data processing point can be understood as a distributed stream processing system. A distributed stream processing system is a computing system capable of processing large volumes of data streams in real time, such as Apache Flink, Apache Kafka Streams, or Apache Spark Streaming. Internally, it maintains a rules engine used to define and enforce business rules, such as average response time thresholds for payment service APIs and alarm thresholds for payment failure rates. These thresholds are key parameters for determining whether the system is experiencing anomalies.

[0088] In step S31, the distributed stream processing system monitors promotional activities of the e-commerce system through a subscription service. The subscription service can be a message queue service (such as Kafka or RabbitMQ) or an event bus mechanism, used to receive promotional event information published by other modules of the e-commerce system (such as the marketing system and event management system). When the distributed stream processing system determines that a promotional activity exists in the e-commerce system (the marketing system and event management system are modules specifically responsible for planning, configuring, and managing various promotional activities. Therefore, event information published from these specific modules can essentially be considered related to promotional activities. Furthermore, promotional event information contains information data such as promotional rules, coupons, activity time, and participating products, or specific data tags set manually. Identifying this information data and specific data tags can be used to distinguish it from other activity event information), it dynamically adjusts the thresholds in the rule engine according to the characteristics of the promotional activity. Specifically, it can increase the average response time threshold of the payment service API to accommodate the increased response time caused by normal high concurrency during promotional activities; at the same time, it can decrease the alarm threshold for payment failure rate to improve sensitivity to payment anomalies and ensure that potential payment problems can be detected in a timely manner during promotional activities. This results in a modified rules engine that adapts to the current operational scenario.

[0089] In practical applications, step S32 uses the adjusted rule engine to analyze the received operational data and determine whether there are any payment anomalies. For example, if the adjusted rule engine reduces the alarm threshold for payment failure rate from 0.5% to 0.1%, then when the actual payment failure rate reaches 0.15%, it may not be considered an anomaly under static rules, but it will be identified as an anomaly in a promotional activity scenario.

[0090] Furthermore, in step S33, if a payment anomaly exists, a first analysis result related to the operational scenario is output. This first analysis result not only indicates the existence of the payment anomaly but also includes contextual information related to the current promotional activity, such as the name of the promotional activity in which the anomaly occurred and its scope of impact, so that system components can make a more accurate response.

[0091] This application's solution introduces real-time monitoring of promotional activities within an e-commerce system and dynamically adjusts the rule engine parameters for system status analysis based on this monitoring. This allows the central data processing point to adaptively adjust its analysis logic for operational data according to the current operational context. This dynamic adjustment mechanism solves the problem of false alarms or missed alarms that may occur due to threshold mismatches in traditional static analysis methods when facing high-concurrency, highly dynamic scenarios such as promotional activities. By increasing the average response time threshold of the payment service API, unnecessary alarms can be avoided under normal high load conditions; by lowering the alarm threshold for payment failure rate, higher sensitivity to payment anomalies can be ensured during critical periods, thereby enabling earlier detection and response to potential payment risks.

[0092] Through the above technical solutions, the central data processing point can achieve more accurate and relevant analysis of the e-commerce system's status. Especially in dynamic operational scenarios such as promotional activities, this solution effectively avoids false alarms or missed alarms caused by improper static threshold settings, thereby improving the accuracy and timeliness of early warnings. This enables the system to more effectively identify and respond to payment anomalies, ensuring a smooth payment experience for users during promotional activities, reducing business losses caused by payment issues, and ultimately improving the overall stability and reliability of the e-commerce system.

[0093] Example description:

[0094] Assume that the e-commerce system uses Apache Flink as a distributed stream processing system and integrates service registration and discovery mechanisms (such as Consul) and configuration management services (such as Apollo).

[0095] S31. The Flink application subscribes to the "Promotional Activity Status" information published by the e-commerce system configuration service through Consul. When the configuration service publishes "Promotional Activity: Double Eleven, Start Time: xxx, End Time: yyy", the Flink application receives this operational context information. Based on the preset promotional activity analysis strategy, the Flink application pulls the average response time threshold for the payment service API (e.g., increased from the usual 500ms to 800ms) and the payment failure rate alarm threshold (e.g., decreased from the usual 1% to 0.5%) for the "Double Eleven" promotion from the Apollo configuration center. These adjusted thresholds are loaded into the Flink application's internal rule engine, forming an analysis rule set for the current promotional scenario.

[0096] S32. The Flink application continuously receives payment service operation data streams from Kafka topics. For each payment service API call log, the Flink application uses the adjusted rule engine loaded in S31 for real-time judgment. For example, if the response time of a payment API is 700ms, it might be considered abnormal under normal rules, but under the adjusted rules, because the threshold is increased to 800ms, it will not be considered abnormal. Similarly, if the payment failure rate reaches 0.6% within 10 seconds, it might not trigger an alarm under normal rules, but under the adjusted rules, because the threshold is reduced to 0.5%, it will be considered a payment anomaly.

[0097] S33. If the Flink application determines that there is a payment anomaly based on the adjusted rule engine (e.g., the payment failure rate reaches 0.6%), the Flink application generates an alert message containing "Payment failure rate exceeded during the Double Eleven promotion period," along with detailed analysis results such as the current payment failure rate and the identifiers of affected payment gateways. This analysis result is sent to a dedicated alert Kafka topic for subsequent automated remediation or maintenance personnel to handle.

[0098] In some embodiments, step S31, which involves controlling the distributed stream processing system to monitor the promotional activities of the e-commerce system through a subscription service, includes:

[0099] S311. For each received promotional event information, its priority information and expected resource consumption information are obtained through parsing;

[0100] S312. Determine the monitoring strategy based on the priority information and expected resource consumption information of the promotional activity; the monitoring strategy includes the data collection granularity, data processing resource quota and alarm threshold for the promotional activity;

[0101] S313. Based on a defined monitoring strategy, control the distributed stream processing system to monitor operational data related to promotional activities, specifically including: continuously evaluating the matching degree between the actual resource consumption and the expected resource consumption of the promotional activities during the operation of the promotional activities;

[0102] S314. When the matching degree is lower than the second preset threshold, update the corresponding monitoring strategy;

[0103] This solution aims to address the problem that distributed stream processing systems may suffer from insufficient monitoring of critical activities or wasted resources on non-critical activities when multiple promotional activities of varying scales and importance are running simultaneously in an e-commerce system. Its core concept is to acquire information on the priority and expected resource consumption of promotional activities, formulate differentiated monitoring strategies, and adjust these strategies based on actual resource consumption during the activity's execution. This ensures high-quality monitoring of critical promotional activities while optimizing overall resource consumption.

[0104] Specifically, promotional event information can be understood as a data package containing a detailed plan for the promotional activity, released by the marketing management platform within the e-commerce system or by an external partner platform. This information typically includes the start and end times of the activity, the target user group, expected sales volume, and, as this application focuses, priority information and expected resource consumption information. Priority information indicates the importance of the promotional activity within the system, and can be categorized as "S-level (highest priority)," "A-level (high priority)," or "B-level (medium priority)," etc., to provide a basis for subsequent resource allocation and monitoring intensity. Expected resource consumption information refers to the amount of system resources that the promotional activity is expected to consume, such as expected network bandwidth, CPU utilization, memory usage, and database connection count, to provide a benchmark for developing reasonable monitoring strategies.

[0105] The monitoring strategy can be understood as a set of monitoring rules and resource allocation schemes customized for a specific promotional activity. Data collection granularity refers to the level of detail in collecting relevant operational data when monitoring the promotional activity. For example, it can be set to collect data once per second, once per minute, or only key indicators, aiming to balance data real-time performance with collection overhead. Data processing resource quota refers to the processing capacity within the distributed stream processing system allocated to the promotional activity. For example, a specific number of CPU cores, memory size, or number of concurrent processing threads can be allocated, aiming to ensure sufficient processing capacity for critical activities while avoiding resource waste. Alarm thresholds refer to the boundaries that trigger alarms when certain performance indicators (such as response time and error rate) exceed preset ranges during monitoring, aiming to promptly detect and report potential system problems.

[0106] In practical applications, continuously evaluating the match between actual and expected resource consumption during a promotional activity refers to collecting real-time data on system resource usage related to the activity during its execution and comparing it with pre-set expected resource consumption. For example, regarding network bandwidth, the match could be the difference between the actual network bandwidth and the expected network bandwidth threshold; similarly, regarding database connection counts, the match could be the ratio of actual to expected database connection counts, but is not limited to these. When the match falls below a second preset threshold—for example, if actual resource consumption significantly exceeds or falls far short of expectations—this may indicate a deviation between the promotional activity's effectiveness and expectations, or an unreasonable allocation of system resources. In this case, the system will automatically update the corresponding monitoring strategy. For instance, if actual resource consumption significantly exceeds expectations, it may be necessary to increase data processing resource quotas or adjust alarm thresholds to handle higher loads; if actual resource consumption is significantly lower than expected, it may be necessary to reduce data collection granularity or release some resources for other services.

[0107] This application's solution introduces refined analysis of promotional event information to obtain priority and expected resource consumption information, enabling the tailoring of differentiated monitoring strategies for each promotional activity. This strategy is no longer uniform but dynamically adjusts data collection granularity, data processing resource quotas, and alarm thresholds based on the specific characteristics of the activity (such as importance and scale). Thus, the distributed stream processing system can prioritize the allocation of limited monitoring resources to high-priority, high-expectation promotional activities, ensuring they receive the most timely and comprehensive monitoring. Simultaneously, for low-priority or low-expectation activities, more resource-efficient monitoring methods can be employed. During the promotional activity's operation, the system continuously evaluates the degree of matching between actual and expected resource consumption. Once the matching degree falls below a second preset threshold, indicating a significant deviation between the actual situation and expectations, the system promptly updates the monitoring strategy. This dynamic adjustment mechanism allows the monitoring strategy to adapt to the actual operation of the promotional activity in real time, avoiding resource waste or monitoring blind spots caused by rigid strategies.

[0108] Through the above technical solution, this application effectively addresses the problem that when multiple promotional activities of varying scales and importance run simultaneously in an e-commerce system, the distributed stream processing system may suffer from insufficient monitoring of critical activities or wasted resources on non-critical activities due to the adoption of a unified monitoring strategy. Specifically, by assessing the priority and expected resource consumption of promotional activities and formulating differentiated monitoring strategies accordingly, it ensures that high-priority, high-risk promotional activities receive more granular and real-time monitoring, thereby promptly identifying and addressing potential problems. Meanwhile, for low-priority activities, more economical monitoring methods can be used, avoiding unnecessary resource investment. Furthermore, by continuously evaluating the matching degree between actual and expected resource consumption during the activity's operation and dynamically updating the monitoring strategy, the system can flexibly respond to unexpected situations and changes during promotional activities, further optimizing overall resource consumption, improving the accuracy and efficiency of monitoring, and thus better supporting the stable operation of the e-commerce system and business decisions.

[0109] Example description:

[0110] Suppose that an e-commerce system is conducting a "limited-time flash sale" promotion.

[0111] S311: A distributed stream processing system (e.g., an Apache Flink application) receives a new promotional event message by subscribing to the event bus (e.g., a Kafka topic) of an e-commerce system. The JSON content of this message might be: {"promo_id":"flash_sale_001","type":"flash sale","priority":"high","expected_cpu_usage":"80%","expected_network_io":"500Mbps"}. The Flink application parses this message, extracting "flash_sale_001" as the `promo_id`, "high" as the `priority`, "80%" as the `expected_cpu_usage`, and "500Mbps" as the `expected_network_io`.

[0112] S312: Based on the parsed "high" priority and the expected resource consumption of "80% CPU utilization" and "500Mbps network I / O", the Flink application determines the monitoring policy for "flash_sale_001". This policy may define: data collection granularity once per second, data processing resource quota of 4 CPU cores and 8GB memory, and alarm thresholds of payment service response time exceeding milliseconds or payment failure rate exceeding 1%.

[0113] S313: The Flink application applies this monitoring strategy to operational data streams related to "flash_sale_001" (e.g., payment service logs, order service metrics). During the flash sale, the Flink application continuously monitors metrics such as actual CPU utilization, network I / O, payment response time, and payment failure rate. For example, it found that actual CPU utilization stabilized at 75% and network I / O was at 480Mbps after the start of the event, which was largely in line with expectations.

[0114] S314: Midway through the activity, Flink application evaluation revealed that while CPU and network I / O matched expectations, the payment failure rate began to rise slightly, from 0.5% to 0.8%. Although this was not yet at the 1% alarm threshold, it indicated potential stress. At this point, Flink application, based on its preset strategy, adjusted the alarm threshold in the monitoring policy for "flash_sale_001" to trigger an alarm if the payment failure rate exceeded 0.7%, and simultaneously increased the data collection granularity to once every 0.5 seconds to detect problems earlier.

[0115] In some embodiments, step S313, the step of continuously evaluating the matching degree between the actual resource consumption and the expected resource consumption of the promotional activity, includes:

[0116] S3131. Obtain current affairs and industry information related to promotional activities through external platforms;

[0117] S3132. Dynamically adjust expected resource consumption based on current events and industry information;

[0118] S3133. Conduct trend analysis on the actual resource consumption data of promotional activities to obtain the trend and rate of change of actual resource consumption;

[0119] S3134. Based on the trend and rate of change of actual resource consumption, combined with the dynamically adjusted expected resource consumption, as well as current affairs and industry information, determine the degree of matching between actual resource consumption and expected resource consumption.

[0120] This solution aims to address the technical problem that during promotional activities in e-commerce systems, actual resource consumption patterns can become highly dynamic and unpredictable due to unforeseen external events or non-linear changes in user behavior. This unpredictability can lead to simple matching degree assessment methods failing to accurately capture the true trend and potential risks of resource consumption, resulting in misjudgments or omissions. Its core concept lies in introducing external event information to dynamically adjust the expected resource consumption range and combining this with trend analysis of actual resource consumption to make a comprehensive judgment, thereby achieving a robust assessment of the degree of resource consumption matching.

[0121] Specifically, in step S3131, obtaining current affairs and industry information related to the promotional activity through external platforms refers to using various public or subscribed data sources, such as news aggregation services, social media trend analysis tools, industry report databases, and data released by market research institutions, to collect external information that may affect the resource consumption of the e-commerce system in real time or near real time. This information may include macroeconomic events, social hotspots, competitor activities, market changes in specific product categories, etc., with the aim of providing more comprehensive background information for subsequent adjustments to expected resource consumption.

[0122] In step S3132, dynamically adjusting the expected resource consumption based on current events and industry information can be understood as revising the originally set expected resource consumption for the promotional activity based on newly acquired external information during the promotional process. For example, if current events indicate that a major social event is about to occur, which may cause a surge or drop in user traffic, the peak or duration of the expected resource consumption can be adjusted accordingly. The purpose is to make the expected resource consumption closer to the actual possible situation and improve the accuracy of the prediction.

[0123] In practical applications, step S3133 involves trend analysis of the actual resource consumption data of the promotional activity to obtain the trend and rate of change of actual resource consumption. Specifically, this refers to time series analysis of actual resource consumption indicators such as CPU utilization, memory usage, network bandwidth, database connection count, and request processing latency collected by the distributed stream processing system during the operation of the promotional activity. For example, statistical methods such as moving average, exponential smoothing, and regression analysis can be used to identify the upward, downward, or stable trends of resource consumption and calculate its rate of change, such as the percentage increase per minute or the decrease per hour. The purpose is to capture the dynamic changes in resource consumption and provide real-time basis for judging the degree of matching.

[0124] Further, in step S3134, judging the matching degree between actual resource consumption and expected resource consumption based on the trend and rate of change of actual resource consumption, combined with the dynamically adjusted expected resource consumption, as well as current affairs information and industry information, means comprehensively comparing the actual resource consumption after trend analysis with the expected resource consumption adjusted based on external information. Similar to the above embodiment, for example: if the rate of change of actual resource consumption per unit time is f, and the actual resource consumption after n units of time is calculated as Q_1 based on the trend and rate of change of actual resource consumption, and the actual resource consumption before trend analysis is Q_0, the expected resource consumption after adjustment based on external information is P_1, and the expected resource consumption before adjustment based on external information is P_0, then:

[0125] Q_1=Q_0±f*n (Formula 1);

[0126] P_1=k*P_0 (Formula 2);

[0127] In Formula 1, when the actual resource consumption trend is upward, Q_1 = Q_0 + f*n; when the actual resource consumption trend is downward, Q_1 = Q_0 - f*n. k is the correlation coefficient related to current events and industry information. It should be noted that the rules for determining the value of the correlation coefficient are set by user experience. For example, if the products involved in the promotional activity are electronic products, the value of k can be increased when current events and industry information such as memory price reductions or chip process upgrades appear. Another example is to obtain promotional activity information from the websites of various competitors based on a preset list of competitors. For each competitor confirmed to be conducting a promotional activity related to electronic products, the value of k is reduced by a preset proportion. For example, if each competitor conducts a promotional activity for electronic products, the value of k is reduced by 1. If multiple competitors conduct promotional activities for electronic products, the values ​​are accumulated until the preset lower limit of the value of k is reached.

[0128] It should be noted that current events and industry information related to promotional activities can be obtained through manual analysis and filtering by users, or by establishing a keyword database and using existing semantic analysis models to perform text analysis and filtering on the collected information.

[0129] Taking network bandwidth as an example, the matching degree can be the difference between the actual resource consumption after trend analysis (Q_1) and the expected resource consumption after adjustment based on external information (P_1).

[0130] Taking the number of database connections as an example, the matching degree can be the ratio of the actual resource consumption after trend analysis (Q_1) to the expected resource consumption after adjustment based on external information (P_1), but it is not limited to this.

[0131] As described in the above embodiments, in addition to network bandwidth and database connection count as resource dimensions used to analyze the matching degree between actual and expected resource consumption, CPU utilization and memory usage can also be used as more resource dimensions. In practical applications, a multi-dimensional matching degree model can be constructed based on multiple resource dimensions to analyze the matching degree between actual and expected resource consumption. The purpose is to provide a more comprehensive and robust matching degree evaluation result and avoid false alarms or false negatives caused by a single indicator or static threshold judgment.

[0132] Specifically, the calculation formula for any one dimension resource in the multi-dimensional matching degree model is the same as Formula 1 and Formula 2 above. The difference lies in the parameter values ​​(especially the correlation coefficient) for different dimension resources. For example:

[0133] Resource dimensions of network bandwidth: Q_1_a = Q_0_a ± f_a * n; P_1_a = k_a * P_0_a;

[0134] Where Q_1_a is the actual network bandwidth resource consumption after trend analysis, Q_0_a is the actual network bandwidth resource consumption before trend analysis, f_a is the rate of change of the actual network bandwidth resource consumption per unit time, P_1_a is the expected network bandwidth resource consumption after adjustment based on external information, k_a is the correlation coefficient of network bandwidth with current affairs information and industry information, and P_0_a is the expected network bandwidth resource consumption before adjustment based on external information;

[0135] Resource dimensions for database connection count: Q_1_b = Q_0_b ± f_b * n; P_1_b = k_b * P_0_b;

[0136] Where Q_1_b is the actual resource consumption of the database connection number after trend analysis, Q_0_b is the actual resource consumption of the database connection number before trend analysis, f_b is the rate of change of the actual resource consumption of the database connection number per unit time, P_1_b is the expected resource consumption of the database connection number after adjustment based on external information, k_b is the correlation coefficient of the database connection number with current affairs information and industry information, and P_0_b is the expected resource consumption of the database connection number before adjustment based on external information;

[0137] CPU utilization in resource dimensions: Q_1_c = Q_0_c ± f_c * n; P_1_c = k_c * P_0_c;

[0138] Where Q_1_c is the actual resource consumption of CPU utilization after trend analysis, Q_0_c is the actual resource consumption of CPU utilization before trend analysis, f_c is the rate of change of the actual resource consumption of CPU utilization per unit time, P_1_c is the expected resource consumption of CPU utilization after adjustment based on external information, k_c is the correlation coefficient of CPU utilization with current events and industry information, and P_0_c is the expected resource consumption of CPU utilization before adjustment based on external information.

[0139] Memory usage resource dimensions: Q_1_d = Q_0_d ± f_d * n; P_1_d = k_d * P_0_d;

[0140] Where Q_1_d is the actual memory consumption after trend analysis, Q_0_d is the actual memory consumption before trend analysis, f_d is the rate of change of the actual memory consumption per unit time, P_1_d is the expected memory consumption after adjustment based on external information, k_d is the correlation coefficient of memory consumption with current events and industry information, and P_0_d is the expected memory consumption before adjustment based on external information.

[0141] Users can determine which resource dimensions to consider in a multi-dimensional matching model based on their importance or impact. For example, if only four resource dimensions are considered: network bandwidth, database connection count, CPU utilization, and memory usage, then the multi-dimensional matching model can be designed as follows:

[0142] Z=A(Q_1_a-P_1_a)+B(Q_1_b-P_1_b)+C(Q_1_c-P_1_c)+D(Q_1_d-P_1_d);

[0143] Or, Z=A(Q_1_a / P_1_a)+B(Q_1_b / P_1_b)+C(Q_1_c / P_1_c)+D(Q_1_d / P_1_d);

[0144] Where A+B+C+D=1, Z is the matching degree output by the multi-dimensional matching degree model, A is the resource dimension influence coefficient of network bandwidth, B is the resource dimension influence coefficient of database connection number, C is the resource dimension influence coefficient of CPU utilization, and D is the resource dimension influence coefficient of memory usage.

[0145] This application's solution incorporates current affairs and industry information from external platforms, enabling real-time adjustments to the anticipated resource consumption of promotional activities, rather than remaining static, based on dynamic changes in the external environment. This solves the problem of the disconnect between anticipated and actual resource consumption in traditional methods. Furthermore, by analyzing trends in actual resource consumption data, the dynamic patterns and rates of resource consumption changes can be captured, rather than focusing solely on instantaneous values, thus allowing for earlier identification of potential performance degradation or resource bottlenecks. Finally, by comprehensively assessing these dynamically adjusted expectations, actual consumption trends, and external information, a more comprehensive and accurate evaluation of the match between actual and anticipated resource consumption can be achieved. This effectively addresses the high dynamism and unpredictability caused by unforeseen external events or non-linear changes in user behavior, avoiding misjudgments or omissions that may result from simple matching assessments.

[0146] Through the aforementioned technical solution, this application significantly improves the accuracy and robustness of resource consumption monitoring in e-commerce systems during promotional activities. By dynamically incorporating external current events and industry information, the anticipated resource consumption can more flexibly adapt to market changes and unforeseen events, enabling the system to anticipate and address potential resource pressures or waste earlier. Combined with analysis of the trends and rates of change in actual resource consumption, the system can not only identify the current state but also predict future trends, thus possessing stronger early warning capabilities in resource matching assessment. This comprehensive judgment mechanism effectively avoids misjudgments or omissions caused by simple matching assessment methods in highly dynamic and unpredictable operational scenarios, ensuring the stable operation of key promotional activities and the rational allocation of resources, thereby improving the overall operational efficiency and user experience of the e-commerce system.

[0147] Example description:

[0148] During the "Summer Mega Sale" promotion in the e-commerce system, a monitoring agent in the distributed stream processing system executes steps S3131 to S3134.

[0149] S3131. Current events and industry information related to promotional activities, such as social media trending topics and competitor activities, can be continuously monitored and obtained in real time from external data sources through a monitoring agent. For example, real-time topic popularity data related to "Summer Sale" can be obtained from social media platforms such as Weibo and Douyin through API interfaces, and the discounts and promotional volume of competitors' "Summer Carnival" during the same period can be obtained from industry intelligence services.

[0150] S3132. Assume the initial expected CPU utilization of the payment service is between 60% and 80%. When the monitoring agent discovers that a certain product topic related to "Summer Sale" suddenly tops the trending list on social media, and competitors' promotional efforts are lower than expected, the system judges that user traffic may far exceed the initial estimate. At this time, the monitoring agent dynamically adjusts the expected range of payment service CPU utilization to 75%-95%.

[0151] S3133. The monitoring agent continuously collects CPU utilization data for the payment service. For example, in the past 5 minutes, CPU utilization has increased from 70% to 85%, showing an upward trend at a rate of 3% per minute.

[0152] S3134. The monitoring agent compares the current CPU utilization of 85% (rising trend, 3% per minute) with the dynamically adjusted expected range of 75%-95%. Simultaneously, considering the traffic growth driven by social media trends, the system determines that while the current CPU utilization of 85% is high, it remains within the dynamically adjusted expected range, and the rate of increase is within a controllable range. Therefore, it determines that the actual resource consumption matches the expected resource consumption. If the CPU utilization spikes to 98% and the rate of change reaches 10% per minute, it is determined to be a mismatch, triggering an alarm.

[0153] Reference Appendix Figure 2 The present invention provides an e-commerce data communication system (the e-commerce data communication system adopts the e-commerce data communication method of the above embodiment, and the specific process is referred to the corresponding steps above), including a local data generation node, a transmission scheduling node and a central data processing node;

[0154] Also includes:

[0155] The first control module 100 is used to deploy local evaluation units at each local data generation point of the e-commerce system, control the local data generation point, evaluate the value level of the operation data generated by the local data generation point based on the preset data classification rules, obtain the value level of the operation data, and package the value level together with the corresponding operation data into bundled data and send it to the transmission scheduling node.

[0156] The second control module 200 is used to control the transmission scheduling node after receiving the bundled data, determine the transmission channel and transmission strategy according to the value level of the operational data, and transmit the corresponding operational data to the central data processing point through the determined transmission channel based on the determined transmission strategy.

[0157] The third control module 300 is used to control the central data processing point to perform system status analysis on the operational data after receiving the operational data from the transmission channel, obtain the first analysis result, and send the first analysis result back to the transmission scheduling node so that the transmission scheduling node forwards it to the corresponding system components through the transmission channel, thereby driving the system components to respond.

[0158] The first control module is one of the core components of this system, and its main function is to evaluate the value of operational data generated by local data generation points. As a preferred implementation, the first control module can be implemented as an independent microservice, deployed at each local data generation point, such as a front-end server, back-end microservice, database proxy layer, or network device. This microservice continuously monitors and collects the operational data generated by its node and evaluates the data in real time using a built-in local evaluation unit according to preset data classification rules. For example, when a local data generation point generates an event log about a payment failure, the local evaluation unit in the first control module will mark it as high-priority data according to the rules. Alternatively, the first control module can also be a library or SDK integrated into the local data generation point application, performing evaluation during data generation through hook functions or interceptors. Furthermore, the first control module can also be a lightweight proxy program, such as the Sidecar proxy, which intercepts all outbound data streams and performs value evaluation before forwarding. The specific implementation and functions of the local data generation point, local evaluation unit, and data classification rules have been described in the above embodiments and will not be repeated here. It is important to emphasize that the first control module, by conducting value assessment at the data generation source, endows the data itself with the ability to carry important information, laying the foundation for subsequent hierarchical transmission.

[0159] Furthermore, the second control module is responsible for receiving bundled data from the first control module and scheduling transmission based on the value level of the operational data. Specifically, the second control module can be implemented as an independent transmission scheduling service, deployed on the transmission scheduling node of the e-commerce data communication system. This service maintains transmission channels with different priorities, such as a priority transmission channel for high-priority data, a standard transmission channel for medium-priority data, and an optimized transmission channel for low-priority data. When the second control module receives bundled data, it parses the value level of the operational data and dynamically selects the appropriate transmission channel and transmission strategy accordingly. For example, for high-priority data, the second control module will immediately send it to the central data processing point through the priority transmission channel, possibly using a highly reliable, low-latency transmission protocol. For low-priority data, it may use an optimized transmission channel for batch transmission or employ a more resource-efficient protocol. As one implementation, the second control module can be a consumer and producer component of a message queue system (such as Kafka or RabbitMQ), routing messages to different topics or queues based on their priority, with these topics or queues corresponding to different transmission channels and strategies. The specific implementation and function of the transmission scheduling node, transmission channel, and transmission strategy have been described in the above embodiments, and will not be repeated here. It should be emphasized that the second control module, through intelligent scheduling, ensures that data of different value levels can obtain transmission resources commensurate with their importance, avoiding waste of communication resources and delays in critical data.

[0160] Furthermore, the third control module is the core of this system's analysis. Its function is to perform system status analysis on the received operational data and generate the first analysis result. The third control module can be implemented as a distributed stream processing system deployed at the central data processing point. This system can process operational data streams from different transmission channels in real time and perform in-depth analysis using preset analysis models or rule engines. For example, when the third control module receives high-priority payment failure data from a priority transmission channel, it immediately triggers an anomaly detection algorithm to determine if there are large-scale payment anomalies and generates corresponding alarm information as the first analysis result. As one implementation, the third control module can be a real-time data analysis platform based on Apache Flink or Apache Spark Streaming, processing data of different priorities by defining different data processing pipelines. In addition, the third control module can also include a rule engine to dynamically adjust analysis thresholds and alarm strategies based on business logic. The specific implementation and function of the central data processing point and system status analysis have been described in the above embodiments and will not be repeated here. It is important to emphasize that the third control module, through real-time and intelligent analysis of operational data, can promptly identify potential system problems and generate analysis results that drive system responses.

[0161] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0162] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An e-commerce data communication method, applied to an e-commerce data communication system, characterized in that, The e-commerce data communication system includes local data generation nodes, transmission scheduling nodes, and central data processing nodes; The data communication method for e-commerce platforms includes the following steps: S1. After deploying local evaluation units at each local data generation point of the e-commerce system, control the local data generation point, and based on the preset data classification rules, evaluate the value level of the operational data generated by the local data generation point through the corresponding local evaluation unit to obtain the value level of the operational data, and package the value level together with the corresponding operational data into bundled data and send it to the transmission scheduling node. S2. After receiving the bundled data, the transmission scheduling node controls the transmission scheduling node to determine the transmission channel and transmission strategy based on the value level of the operational data, and transmits the corresponding operational data to the central data processing point through the determined transmission channel based on the determined transmission strategy. S3. After receiving the operational data from the transmission channel, the central data processing point controls the central data processing point to perform system status analysis on the operational data, obtain the first analysis result, and send the first analysis result back to the transmission scheduling node so that the transmission scheduling node forwards it to the corresponding system components through the transmission channel, thereby driving the system components to respond.

2. The e-commerce data communication method according to claim 1, characterized in that, In step S1, based on preset data grading rules, the step of evaluating the value level of the operational data generated by the local data generation point through the corresponding local evaluation unit to obtain the value level of the operational data includes: S11. Through the local evaluation unit, based on the preset data classification rules, the value level of the operational data generated by the local data generation point is evaluated to obtain the initial value level of the operational data; S12. Periodically obtain service identifiers that are dependent on the local data generation point; S13. Based on the service identifier, obtain the aggregated status summary of the neighboring services by sending a request to the corresponding neighboring services; S14. Determine whether the initial value level of the operational data is medium priority; S15. If the initial value level of the operational data is medium priority, then based on the operational data generated by the local data generation point and the aggregated status summary of the nearby services, determine whether there is a common performance decline trend between the local data generation point and the nearby services. If there is a common performance decline trend, the value level of the operational data will be upgraded to high priority; otherwise, the value level of the operational data will remain unchanged.

3. The e-commerce data communication method according to claim 2, characterized in that, In step S15, the step of determining whether there is a common performance degradation trend between the local data generation point and the neighboring services, based on the operational data generated by the local data generation point and the aggregated status summary of the acquired neighboring services, includes: S151. Assign weights to performance metrics in operational data and aggregated status briefings; S152. Calculate the aggregation risk score based on the weights assigned to the performance indicators; S153. Compare the aggregated risk score with a first preset threshold; S154. When the aggregated risk score exceeds the first preset threshold, it is determined that the local data generation point and the neighboring service have a common performance decline trend.

4. The e-commerce data communication method according to claim 1, characterized in that, Local data generation points can be front-end servers, back-end microservices, database proxy layers, or network devices.

5. The e-commerce data communication method according to claim 1, characterized in that, The local evaluation unit is specifically a standalone process, a proxy component in a service mesh, or a log collection module in an application.

6. The e-commerce data communication method according to claim 1, characterized in that, Data grading rules are used to classify operational data into high-priority, medium-priority, and low-priority data; The transmission channels include a priority transmission channel for transmitting high-priority data, a standard transmission channel for transmitting medium-priority data, and an optimized transmission channel for transmitting low-priority data.

7. The e-commerce data communication method according to claim 1, characterized in that, The central data processing point is specifically a distributed stream processing system; the distributed stream processing system maintains a rule engine, which includes the average response time threshold for the payment service API and the alarm threshold for the payment failure rate; In step S3, the specific steps for controlling the central data processing point to perform system status analysis on operational data and obtain the first analysis result include: S31. Control the distributed stream processing system to monitor the promotional activities of the e-commerce system through subscription services, and when it is determined that there are promotional activities in the e-commerce system, obtain the adjusted rule engine by increasing the average response time threshold of the payment service API and reducing the alarm threshold of the payment failure rate. S32. Utilize the adjusted rules engine to analyze and determine whether there are any payment anomalies in the operational data; S33. If payment anomalies are found, output the first analysis result related to the operational scenario.

8. The e-commerce data communication method according to claim 7, characterized in that, In step S31, the steps of controlling the distributed stream processing system to monitor the promotional activities of the e-commerce system through a subscription service include: S311. For each received promotional event information, its priority information and expected resource consumption information are obtained through parsing; S312. Determine the monitoring strategy based on the priority information and expected resource consumption information of the promotional activities; S313. Based on a defined monitoring strategy, control the distributed stream processing system to monitor operational data related to promotional activities, specifically including: continuously evaluating the matching degree between the actual resource consumption and the expected resource consumption of the promotional activities during the operation of the promotional activities; S314. When the matching degree is lower than the second preset threshold, update the corresponding monitoring strategy.

9. The e-commerce data communication method according to claim 1, characterized in that, Step S313, the step of continuously evaluating the matching degree between the actual resource consumption and the expected resource consumption of the promotional activity, includes: S3131. Obtain current affairs and industry information related to promotional activities through external platforms; S3132. Dynamically adjust expected resource consumption based on current events and industry information; S3133. Conduct trend analysis on the actual resource consumption data of promotional activities to obtain the trend and rate of change of actual resource consumption; S3134. Based on the trend and rate of change of actual resource consumption, combined with the dynamically adjusted expected resource consumption, as well as current affairs and industry information, determine the degree of matching between actual resource consumption and expected resource consumption.

10. An e-commerce data communication system, characterized in that, This includes local data generation nodes, transmission scheduling nodes, and central data processing nodes; Also includes: The first control module is used to deploy local evaluation units at each local data generation point of the e-commerce system, control the local data generation point, evaluate the value level of the operation data generated by the local data generation point based on the preset data classification rules, obtain the value level of the operation data, and package the value level together with the corresponding operation data into bundled data and send it to the transmission scheduling node. The second control module is used to control the transmission scheduling node after receiving the bundled data, determine the transmission channel and transmission strategy according to the value level of the operational data, and transmit the corresponding operational data to the central data processing point through the determined transmission channel based on the determined transmission strategy. The third control module is used to control the central data processing point to perform system status analysis on the operational data after receiving the operational data from the transmission channel, obtain the first analysis result, and send the first analysis result back to the transmission scheduling node so that the transmission scheduling node forwards it to the corresponding system components through the transmission channel, thereby driving the system components to respond.

Images