E-commerce data intelligent processing method and system based on dynamic flow optimization

Abstract

The invention discloses an e-commerce data intelligent processing method and system based on dynamic flow optimization. The method comprises the following steps: acquiring and analyzing multi-platform e-commerce data, and converting the multi-platform e-commerce data into a standardized format; constructing a matching rule by adopting a decision tree algorithm, and performing classification and priority marking on the data; the data flow is dynamically scheduled based on the priority, so that the high-priority data is executed preferentially; data are processed through parallel nodes, performance indexes such as network delay and throughput are monitored in real time, and data stream processing efficiency is optimized in combination with a dynamic scheduling algorithm. The system correspondingly comprises a data standardization module, a rule matching module, a dynamic marking module, a parallel processing module and a monitoring scheduling module, efficient and self-adaptive processing of the multi-source e-commerce data is achieved, and the data processing real-time performance and the system throughput are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an intelligent e-commerce data processing method and system based on dynamic traffic optimization. Background Technology

[0002] With the widespread adoption of internet technology and the continuous evolution of e-commerce models, the amount of data that modern e-commerce platforms need to process daily is growing exponentially. This data covers multiple dimensions, including user behavior logs, product information, order transactions, payment notifications, inventory updates, and logistics tracking, forming a massive and complex real-time data stream. How to efficiently, reliably, and intelligently process these data streams, while ensuring the stability and timeliness of core business operations (such as order fulfillment and payment processing), has become a core technological challenge for e-commerce platforms.

[0003] Chinese Patent, Publication No. CN114663187B, Publication Date: May 12, 2023, discloses a business data processing method and system based on artificial intelligence and e-commerce. For the service activation status process of a preset e-commerce service, if it is determined that a candidate e-commerce subscription service has switched to a preset subscription traffic state, the method searches for the hotspot attention distribution of advanced placement hotspots for the preset e-commerce service. Further, based on the hotspot attention distribution analysis, optimized placement partitions of the advanced placement hotspots can be obtained. Placement nodes are optimized within these optimized placement partitions, and the optimized advanced placement hotspots are configured in the promotion plan list for promoting the preset e-commerce service. Through the embodiments of this disclosure, when the service is in a preset subscription traffic state, based on the hotspot attention distribution obtained by pre-optimizing the placement nodes of the advanced placement hotspots in the preset e-commerce service, the configuration optimization accuracy of the promotion plan can be intelligently improved. This improves the data processing efficiency of promotion and reduces resource waste.

[0004] The shortcomings of the above technical solution are: the comparison files only achieve resource allocation through partition weight adjustment, but do not prioritize individual e-commerce data entries. This leads to the congestion of high-value data, such as payment success events and user browsing logs competing for resources in the same partition, making it impossible to ensure that payment data is processed first. Meanwhile, low-value data such as logs may monopolize resources due to their high partition weight, while high-value data is processed with delays. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application aims to provide an intelligent e-commerce data processing method and system based on dynamic traffic optimization. This method can achieve intelligent scheduling and resource optimization of data traffic through dynamic priority marking, rule matching decision trees, and real-time monitoring feedback. It ensures that high-priority e-commerce data is executed first, while reducing the resource consumption of low-priority e-commerce data, thereby improving system throughput efficiency and response speed.

[0006] To achieve the above objectives, this application adopts the following technical solution: This application provides an intelligent e-commerce data processing method based on dynamic traffic optimization, the method comprising: S101, acquire e-commerce data from different e-commerce platforms; S102, parse e-commerce data from different e-commerce platforms, identify the source, data type and business characteristics of the e-commerce data, and convert the e-commerce data into a standardized data format with header fields that mark the source, data type and business characteristics of the e-commerce data; S103, a matching rule decision tree is constructed using a decision tree algorithm, and the rule decision tree includes several parallel processing nodes; S104, The standardized e-commerce data is classified using a rule matching algorithm to obtain the mapping classification results between e-commerce data and matching rules; S105, based on the mapping and classification results of e-commerce data and matching rules, dynamically adds high-priority or low-priority tags to e-commerce data; S106, Real-time scheduling of e-commerce data traffic is performed based on high-priority or low-priority tags, so that e-commerce data with high-priority tags will be given priority in executing the actions corresponding to the e-commerce data matching rules. S107: Based on the mapping and classification results of e-commerce data and matching rules, the classified e-commerce data is distributed to multiple parallel processing nodes of the rule decision tree to execute the actions corresponding to the matching rules of the e-commerce data. S108 continuously monitors key performance indicators including network latency, throughput, and node processing status, provides real-time feedback on monitoring results, and employs a dynamic traffic scheduling algorithm for e-commerce data traffic scheduling.

[0007] As a preferred technical solution, in step S101, acquiring e-commerce data from different e-commerce platforms includes: constructing a software-defined network architecture, which includes a separate data layer and a control layer. E-commerce data from different e-commerce platforms are uniformly accessed through the data layer. The data layer is configured with an open-source virtual switch, which forwards control actions and e-commerce data from the data layer to the control layer. The open-source virtual switch establishes a secure transmission channel with the control layer through transport layer security protocols and transport control protocols. The control layer receives flow tables issued by the open-source virtual switch based on open-source flow protocols, and the control layer executes actions on the e-commerce data according to the matching rules of the flow tables.

[0008] As a preferred technical solution, in step S102, parsing e-commerce data from different e-commerce platforms and identifying the source, data type, and business characteristics of the e-commerce data includes: using an open-source virtual switch at the data access point to acquire incoming e-commerce data; using a filtering algorithm to divert the incoming e-commerce data and filter out non-business traffic or malicious traffic; extracting the basic network characteristics of the filtered e-commerce data to obtain the source and data type of the e-commerce data; parsing the parsed application layer payload of the filtered e-commerce data, identifying the application layer protocol of the e-commerce data, and further parsing the request method and URL path of the application layer protocol; and determining the business characteristics of the e-commerce data based on the parsed request method and URL path.

[0009] As a preferred technical solution, when the open-source virtual switch and the control layer begin establishing a connection, they send greeting messages to each other. These greeting messages contain the open-source flow protocol versions supported by both the open-source virtual switch and the control layer, and a response message confirms the connection. When e-commerce data enters the open-source virtual switch, it first searches for matching rules in the rule decision tree. If no matching rules are found in the rule decision tree, the open-source virtual switch sends the e-commerce data to the control layer for assistance. Upon receiving the e-commerce data, the control layer extracts the source, data type, and business characteristics of the e-commerce data, performs analysis and decision-making based on a predefined rule database, and sends the decision results back to the open-source virtual switch to establish flow table entries. This allows the open-source virtual switch to classify the standardized e-commerce data using a rule matching algorithm.

[0010] As a preferred technical solution, step S103, which involves constructing a matching rule decision tree using a decision tree algorithm, includes: S103a, dividing the rules into a rule set based on the field length range of the rules to generate a first-level data matching rule set comprising several rule sets; S103b, when the number of rules in a rule set within the first-level data matching rule set exceeds half the total number of rules in the first-level data matching rule set, splitting that rule set into smaller sub-rule sets to create a second-level data matching rule set; when the number of rules in a rule set within the second-level data matching rule set exceeds half the total number of rules in the second-level data matching rule set, splitting that rule set into smaller sub-rule sets... The system first divides the rules into several rule sets to create the next layer of data matching rules, until the number of data matching rule layers reaches a preset threshold; S103c, calculates the index position of each rule set to facilitate rule splitting; S103d, constructs a rule decision tree, where the first root node of the rule decision tree includes all rules. Based on the first layer of data matching rules, the rules are divided into several rule sets, each rule set representing a child node. Based on the second layer of data matching rules, the rule sets are divided into several sub-rule sets, each sub-rule set representing a child node, until the number of rules in the sub-rule sets is less than a preset threshold and the number of data matching rule layers reaches a preset threshold, finally obtaining the rule decision tree.

[0011] As a preferred technical solution, in steps S103a and S103b, if the field length range of the rule set is [0, ..., ... -1], w is the bit width of the field, then the Hash algorithm is used to store the rules in the rule set; the prefix field of the rule is used as the mask of its rule set, and the mask and the field value in the rule are logically ANDed to obtain the operation result, which is used to represent the index position of the rule stored in the rule set.

[0012] As a preferred technical solution, step S104, classifying the standardized e-commerce data using a rule matching algorithm, includes: compiling the rules in the rule decision tree into a flow table containing several flow table entries, including the source, data type, business characteristics, and high-priority or low-priority tags of the e-commerce data; the open-source virtual switch matching the header fields of the standardized e-commerce data packets with several flow table entries in the flow table item by item; if the header fields of the e-commerce data packets match the flow table entries, the control layer executes the action corresponding to the matching rule for the e-commerce data.

[0013] As a preferred technical solution, in step S106, the real-time execution of e-commerce data traffic scheduling based on high-priority or low-priority tags, so that e-commerce data with high-priority tags will preferentially execute the actions corresponding to the e-commerce data matching rules, includes: establishing physically independent high-priority queues and low-priority queues; when outputting e-commerce data, sending the e-commerce data to the high-priority or low-priority queues according to the high-priority or low-priority tags of the e-commerce data; establishing high-priority consumer groups and low-priority consumer groups, where the high-priority consumer group is used to consume the high-priority queue and has more computing resources, and the low-priority consumer group is used to consume the low-priority queue and has relatively fewer resources.

[0014] As a preferred technical solution, in step S108, the continuous monitoring includes key performance indicators such as network latency, throughput, and node processing status, and real-time feedback of monitoring results. The e-commerce data traffic scheduling using a dynamic traffic scheduling algorithm includes: S108a, collecting network latency in real-time from the software-defined network architecture, calculating throughput by the total amount of e-commerce data processed and the time spent processing it, and collecting node processing status from parallel processing nodes, including CPU utilization, memory usage, processing queue length, and average processing time; S108b, unifying the obtained network latency, throughput, and node processing status into a time-series database, and cleaning, aggregating, and formatting the network latency, throughput, and node processing status in the time-series database to obtain monitoring results; S108c, if the CPU utilization of a parallel processing node continuously exceeds a preset threshold, it is determined to be a performance bottleneck; if the network latency of a link exceeds a preset threshold or the throughput is less than a preset threshold, the network path is determined to be congested; based on the identified problems, a corresponding traffic scheduling strategy is generated; S108d, the generated traffic scheduling strategy is compiled into new rules, and the rule decision tree is updated.

[0015] This application also provides an e-commerce data intelligent processing system based on dynamic traffic optimization. The system includes: a data acquisition and standardization module, used to acquire e-commerce data from multiple e-commerce platforms, parse the source, data type, and business characteristics of the e-commerce data, and convert the e-commerce data into a standardized data format with header fields that indicate the source, data type, and business characteristics; a rule matching and classification module, used to construct a matching rule decision tree containing multiple parallel processing nodes using a decision tree algorithm, and classify the standardized e-commerce data using the rule matching algorithm to generate a mapping classification result between e-commerce data and matching rules; and a dynamic priority marking module. The dynamic priority marking module dynamically adds high-priority or low-priority tags to e-commerce data based on the mapping classification results, and performs e-commerce data traffic scheduling in real time according to the priority tags, so that high-priority e-commerce data executes the corresponding rule actions first; the parallel processing and execution module distributes the classified e-commerce data to multiple parallel processing nodes of the rule decision tree, and executes the processing actions corresponding to the rules matching the e-commerce data; the monitoring and dynamic scheduling module continuously monitors key performance indicators such as network latency, throughput and node processing status, provides real-time feedback on monitoring results, and uses a dynamic traffic scheduling algorithm to optimize e-commerce data traffic in real time.

[0016] Compared with the prior art, the beneficial effects of this application are as follows: The e-commerce data intelligent processing method based on dynamic traffic optimization proposed in this application solves the problems of resource waste, response delay and concurrency bottleneck in traditional e-commerce data processing, and realizes a paradigm leap from static rule execution to dynamic intelligent scheduling.

[0017] First, at the data preprocessing level, by parsing the business characteristics of multi-source e-commerce data and converting them into a standardized format, a structured foundation is laid for rule matching, thus solving the problem of low processing efficiency caused by data heterogeneity.

[0018] Secondly, in terms of the core scheduling mechanism, it innovatively integrates decision tree rule matching, dynamic priority marking, and parallel node distribution: the decision tree algorithm maps rules to parallel processing nodes, supporting millisecond-level classification; high / low priority tags are dynamically added based on the classification results, so that each piece of e-commerce data has an independent scheduling priority identifier, breaking through the coarse-grained limitations of traditional partition-level optimization; high-priority e-commerce data skips the queue in real time and goes directly to parallel nodes to execute business actions, thereby improving the response speed of high-value business, while low-priority data is processed during idle periods, reducing resource contention.

[0019] Finally, by continuously monitoring key performance indicators such as network latency, throughput, and node processing status and feeding them back to the dynamic scheduling algorithm, a closed loop of "execution-monitoring-optimization" is formed: the system automatically expands parallel nodes during peak traffic periods, increasing throughput; resource allocation on demand reduces server costs and increases annual carbon reduction, thus possessing both commercial and environmental value. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the steps of the intelligent e-commerce data processing method based on dynamic traffic optimization proposed in this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in specific embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0022] like Figure 1 As shown, this application provides an intelligent e-commerce data processing method based on dynamic traffic optimization, the method comprising: S101, acquire e-commerce data from different e-commerce platforms; S102, parse e-commerce data from different e-commerce platforms, identify the source, data type and business characteristics of the e-commerce data, and convert the e-commerce data into a standardized data format with header fields that mark the source, data type and business characteristics of the e-commerce data; S103, a matching rule decision tree is constructed using a decision tree algorithm, and the rule decision tree includes several parallel processing nodes; S104, The standardized e-commerce data is classified using a rule matching algorithm to obtain the mapping classification results between e-commerce data and matching rules; S105, based on the mapping and classification results of e-commerce data and matching rules, dynamically adds high-priority or low-priority tags to e-commerce data; S106, Real-time scheduling of e-commerce data traffic is performed based on high-priority or low-priority tags, so that e-commerce data with high-priority tags will be given priority in executing the actions corresponding to the e-commerce data matching rules. S107: Based on the mapping and classification results of e-commerce data and matching rules, the classified e-commerce data is distributed to multiple parallel processing nodes of the rule decision tree to execute the actions corresponding to the matching rules of the e-commerce data. S108 continuously monitors key performance indicators including network latency, throughput, and node processing status, provides real-time feedback on monitoring results, and employs a dynamic traffic scheduling algorithm for e-commerce data traffic scheduling.

[0023] Furthermore, in step S101, obtaining e-commerce data from different e-commerce platforms includes: Build a Software Defined Network (SDN) architecture.

[0024] The software-defined network architecture includes a separate data layer and a control layer. E-commerce data from different e-commerce platforms are uniformly accessed through the data layer. The data layer is configured with an open-source virtual switch, which forwards control actions and e-commerce data from the data layer to the control layer.

[0025] The open-source virtual switch establishes a secure transmission channel with the control layer through transport layer security protocols and transmission control protocols. The control layer receives flow tables issued by the open-source virtual switch based on open-source flow protocols, and executes actions on e-commerce data according to the matching rules of the flow tables.

[0026] In traditional network architectures, network devices (Switch, Router) must be responsible for both the network control logic of the Control Layer and the data forwarding of the Data Layer. Because this architecture is limited by the hardware specifications of the vendor, it makes it difficult to configure and manage the network and limits the network's scalability.

[0027] SDN separates the control layer and the data layer. Network control logic and decision-making are centrally managed by a programmable "Controller," while data forwarding is handled by open-source virtual switches (OpenvSwitch, OVS). The SDN control layer can be divided into northbound and southbound interfaces. The northbound interface is the communication interface between the application layer and the API, used for network monitoring and management configuration. The southbound interface is the communication interface between the data layer and the OpenFlow protocol, used for issuing control commands, collecting network information, and controlling the underlying network operation.

[0028] Furthermore, in step S102, parsing e-commerce data from different e-commerce platforms and identifying the source, data type, and business characteristics of the e-commerce data includes: Open-source virtual switches are used at data access points to acquire incoming e-commerce data. Filtering algorithms (for example, filtering algorithms based on IP network segments or port ranges) are used to divert the incoming e-commerce data and filter out non-business traffic (such as network management protocols) or malicious traffic.

[0029] Extract the basic network characteristics of the filtered e-commerce data to determine its source and data type. These basic network characteristics include: source IP address, destination IP address, source port, destination port, and transport layer protocol (such as TCP / UDP).

[0030] The application layer payload of the filtered e-commerce data is parsed to identify the application layer protocol (such as HTTP / 1.1, HTTP / 2, gRPC, WebSocket, etc.). In this application, the application layer protocol adopted is HTTP, and the request method (GET / POST / PUT), URL path, Host field, Content-Type and other header information of the application layer protocol are further parsed.

[0031] Based on the parsed request method and URL path, the business characteristics of e-commerce data can be determined. For example, if a POST request with the path / api / v1 / order / create is identified, its business characteristic can be initially marked as "creating an order".

[0032] Furthermore, when the open-source virtual switch and the control layer begin establishing a connection, they exchange greeting messages. These messages contain the open-source flow protocol versions supported by both the open-source virtual switch and the control layer, and a response message confirms the connection. When e-commerce data enters the open-source virtual switch, it first searches for matching rules in the rule decision tree. If no matching rules are found in the rule decision tree, the open-source virtual switch sends the e-commerce data to the control layer for assistance. Upon receiving the e-commerce data, the control layer extracts its source, data type, and business characteristics. Based on a predefined rule database, it performs analysis and decision-making, and sends the decision results back to the open-source virtual switch to establish flow table entries. This allows the open-source virtual switch to classify the standardized e-commerce data using a rule-matching algorithm.

[0033] Through the above settings, efficient traffic adaptation and intelligent decision-making capabilities are achieved. By using centralized deep analysis of unknown traffic (extracting source, type, and business characteristics) and rule-based intelligent analysis at the control layer, this method overcomes the limitations of traditional switches that can only perform simple, static matching. The control layer can make optimal scheduling decisions based on a global view and business logic, and then translate these decisions into flow table entries and send them to the switches, enabling the entire system to intelligently perceive and adaptively process complex and ever-changing e-commerce business.

[0034] This significantly reduces network transmission latency in high-concurrency scenarios. The "initial query + fast offloading" mechanism effectively solves the slow response problem caused by traffic bursts. Specifically, for the first packet or unknown traffic, the control layer makes an authoritative decision; for subsequent large numbers of data packets, the switch performs high-speed forwarding locally based on established flow table entries, without needing to report again. This avoids the performance bottleneck caused by all data packets needing to be processed by the controller. Real-world testing shows that this method can significantly reduce network latency during peak periods from over 500ms to below 100ms, greatly improving the user experience.

[0035] This significantly improves the overall data processing throughput efficiency of the system. On the one hand, the localized matching and forwarding by the switches greatly reduces the processing pressure on the controller, preventing it from becoming a single point of failure. On the other hand, the policies issued by the controller (such as priority marking) can guide the switches to perform fine-grained scheduling of data flows (such as prioritizing bandwidth for critical traffic such as payments and orders), ensuring optimal utilization of network resources. Compared with the throughput of traditional rule-based classification algorithms (<1Gbps), this method can increase the throughput to over 10Gbps when processing 100,000+ product data, effectively supporting the real-time processing needs of massive e-commerce data.

[0036] This establishes a dynamic control system capable of continuous optimization within a closed loop. This application does not involve a one-time static rule issuance. By continuously monitoring network performance and providing real-time feedback to the control layer, the control layer can dynamically adjust existing flow table strategies, thus forming a closed-loop control system of "monitoring-decision-execution-re-optimization." This enables the system to continuously adapt to dynamic changes in network conditions and service load, maintaining consistently high performance and high reliability.

[0037] It offers excellent compatibility and scalability. Based on standard protocols such as OpenFlow, it decouples the control layer from the data layer, enabling seamless access for systems using different models or versions of open-source virtual switches, and ensuring interoperability through protocol negotiation. Meanwhile, the rule database and analytical decision algorithms reside in the control layer, allowing for independent updates and upgrades without interrupting the data layer's forwarding services. This facilitates the future introduction of more complex analytical models (such as machine learning), resulting in strong system scalability.

[0038] Furthermore, in step S103, constructing a matching rule decision tree using a decision tree algorithm includes: S103a, rules are divided into a set of rules based on the field length range of the rules, so as to generate a first-level data matching rule that includes several sets of rules.

[0039] S103b, when the number of rules in a rule set within the first-level data matching rules exceeds half the total number of rules in the first-level data matching rules, that rule set is split into smaller sub-rule sets to create the second-level data matching rules. When the number of rules in a rule set within the second-level data matching rules exceeds half the total number of rules in the second-level data matching rules, that rule set is split into smaller sub-rule sets to create the next level of data matching rules, until the number of data matching rule levels reaches a preset threshold.

[0040] S103c, calculate the index position of each rule set to facilitate rule splitting.

[0041] S103d, construct a rule decision tree. The first root node of the rule decision tree includes all rules. Based on the first layer of data matching rules, the rules are divided into several rule sets, each rule set representing a child node. Based on the second layer of data matching rules, the rule sets are divided into several sub-rule sets, each sub-rule set representing a child node, until the number of rules in the sub-rule sets is less than a preset threshold and the number of data matching rule layers reaches a preset threshold, finally obtaining the rule decision tree.

[0042] Steps S103a and S103b result in a smaller number of rule sets (Tuples), but also store more rules in the Tuples, forming longer rule chains. This can lead to hash collisions when storing rules in the Tuples, causing matching errors. A hash collision occurs when two or more data (rules) are located in the same position. This is called a hash collision because it can happen wherever a hash exists, especially when there are too many data (rules) to store.

[0043] Through steps S103c and S103d, based on the decision tree algorithm, the rule decision tree TabTree is improved by using the data classification method MultilayerTuple in steps S103a and S103b. On the one hand, it can retain the advantages of MultilayerTuple in multi-field classification and balance performance; on the other hand, it can also retain the advantages of TabTree in fast search and avoid the problem of Hashcollision.

[0044] Furthermore, in steps S103a and S103b, if the field length range of the rule set is [0, ... [-1], where w is the bit width of the field (for example, if w=32 (the bit width of an IPv4 address), then this range is [0, 2³¹-1]). A hash algorithm is used to store the rules in this rule set. The prefix field of a rule is used as a mask for its rule set, and a logical AND operation is performed between the mask and the field values ​​in the rule to obtain the result. The result represents the index position of the rule stored in the rule set.

[0045] In Software-Defined Networking (SDN), the classification and matching of e-commerce data is based on a series of rules (using OpenFlow flow table entries in this application). For fast matching, these rules are typically organized in specific data structures. "Rule duplication" is a common problem. When a rule cannot be precisely classified into a particular partition, it may need to be copied to multiple partitions to ensure the integrity of the match. This significantly increases memory overhead and the complexity of rule maintenance. Rule trees (TabTree) and Tuple Space Search (TSS) are data structures and algorithms used to organize and manage rule sets, aiming to optimize search speed and reduce memory consumption. The core idea of ​​TSS is to partition the space based on the value range of different fields (tuples) of the rules. This application uses a specific range ([0, ...)... The rules within [-1] are managed using TSS, which optimizes the storage and retrieval of rules, thereby avoiding the need to copy these rules to multiple partitions.

[0046] Furthermore, in step S104, the standardized e-commerce data is classified using a rule-matching algorithm, including: The rules in the rule decision tree are compiled into a flow table that includes several flow table entries, including the source of e-commerce data, data type, business characteristics, and high-priority or low-priority tags.

[0047] In this application, the flow table is shown in Table 1. The matching fields include the source, data type, and business characteristics of the e-commerce data.

[0048] Table 1 Flow Table For example, a flow table is shown below: / * OXM Flow match field types for OpenFlow basic class.* / enum oxm_ofb_match_fields { OFPXMT_OFB_IN_PORT = 0, / * Switch input port * / OFPXMT_OFB_IN_PHY_PORT = 1, / * Physical input port * / OFPXMT_OFB_METADATA = 2, / * Metadata * / OFPXMT_OFB_ETH_DST = 3, / * Ethernet destination address * / OFPXMT_OFB_ETH_SRC = 4, / * Ethernet source address * / OFPXMT_OFB_ETH_TYPE = 5, / * Ethernet type * / OFPXMT_OFB_VLAN_VID = 6, / * VLAN ID * / OFPXMT_OFB_VLAN_PCP = 7, / * VLAN PCP * / OFPXMT_OFB_IP_DSCP = 8, / * IP DSCP * / OFPXMT_OFB_IP_ECN = 9, / * IP ECN * / OFPXMT_OFB_IPPROTO = 10, / * IP protocol * / OFPXMT_OFB_IPV4_SRC = 11, / * IPv4 source address * / OFPXMT_OFB_IPV4_DST = 12, / * IPv4 destination address * / OFPXMT_OFB_TCP_SRC = 13, / * TCP source port * / OFPXMT_OFB_TCP_DST = 14, / * TCP destination port * / OFPXMT_OFB_UDP_SRC = 15, / * UDP source port * / OFPXMT_OFB_UDP_DST = 16, / * UDP destination port * / OFPXMT_OFB_SCTP_SRC = 17, / * SCTP source port * / OFPXMT_OFB_SCTP_DST = 18, / * SCTP destination port * / OFPXMT_OFB_ICMPV4_TYPE = 19, / * ICMPv4 type * / OFPXMT_OFB_ICMPV4_CODE = 20, / * ICMPv4 code * / OFPXMT_OFB_ARP_OP = 21, / * ARP operations * / OFPXMT_OFB_ARP_SPA = 22, / * ARP source IPv4 address * / OFPXMT_OFB_ARP_TPA = 23, / * ARP target IPv4 address * / OFPXMT_OFB_ARP_SHA = 24, / * ARP source hardware address * / OFPXMT_OFB_ARP_THA = 25, / * ARP target hardware address * / OFPXMT_OFB_IPV6_SRC = 26, / * IPv6 source address * / OFPXMT_OFB_Extreme6_DST = 27, / * IPv6 destination address * / OFPXMT_OFB_IPV6_FLABEL_FLABEL = 28, / * IPv6 Stream Label * / OFPXMT_OFB_ICMPV6_TYPE = 29, / * ICMPv6 type * / OFPXMT_OFB_ICMPV6_CODE = 30, / * ICMPv6 code* / OFPXMT_OFB_IPV6_NDP_TARGET = 31, / * IPv6 NDP destination address * / OFPXMT_OFB_IPV6_NDP_SLL = 32, / * IPv6 NDP source link layer address * / OFPXMT_OFB_IPV6_NDP_TLL = 33, / * IPv6 NDP target link layer address * / OFPXMT_OFB_MPLS_LABEL = 34, / * MPLS tags * / OFPXMT_OF_MPLS_TC = 35, / * MPLS TC * / OFPXMT_OFB_MPLS_BOS = 36, / * MPLS BOS * / OFPXMT_OFB_PBB_ISID = 37, / * PBB ISID * / OFPXMT_OFB_TUNNEL_ID = 38, / * Tunnel ID * / OFPXMT_OFB_IPV6_EXTHDR = 39, / * IPv6 extension header * / OFPXMT_OFB_PBB_UCA = 41, / * PBB UCA * / / * Note: The value 40 is missing; refer to the image content * / OFPXMT_OFB_TCP_FLAGS = 42, / * TCP flags * / OFPXMT_OFB_ACTSET_OUTPUT = 43, / * Action set output * / OFPXMT_OFB_PACKET_TYPE = 44, / * Package type * / }

[0049] The open-source virtual switch matches the header fields of standardized e-commerce data packets against several flow table entries. If a header field of an e-commerce data packet matches a flow table entry, the control layer executes the instruction corresponding to the matching rule.

[0050] Furthermore, in step S106, e-commerce data traffic scheduling is performed in real time based on high-priority or low-priority tags, so that e-commerce data with high-priority tags is given priority in executing the actions corresponding to the e-commerce data matching rules, including: Physically independent high-priority queues (high_priority_queue) and low-priority queues (low_priority_queue) are established. When outputting e-commerce data, the data is sent to the high-priority queue or the low-priority queue based on its high-priority or low-priority flag. Specifically, the scheduler reads the flags on the e-commerce data and makes immediate routing decisions based on the identified priority field value. High-priority consumer groups and low-priority consumer groups are also established. The high-priority consumer group consumes data from the high-priority queue and has more computing resources, while the low-priority consumer group consumes data from the low-priority queue and has relatively fewer resources.

[0051] Furthermore, in step S108, key performance indicators including network latency, throughput, and node processing status are continuously monitored, and monitoring results are fed back in real time. E-commerce data traffic scheduling is performed using a dynamic traffic scheduling algorithm, including: S108a collects network latency in real time from the software-defined network architecture, calculates throughput by the total amount of e-commerce data processed and the time spent processing this e-commerce data, and collects node processing status from parallel processing nodes, including CPU utilization, memory usage, processing queue length, and average processing time.

[0052] S108b aggregates the obtained network latency, throughput, and node processing status into a time-series database, and cleans, aggregates, and formats the network latency, throughput, and node processing status in the time-series database to obtain monitoring results.

[0053] S108c: If the CPU utilization of a parallel processing node continuously exceeds a preset threshold, it is determined to be a performance bottleneck. If the network latency of a link exceeds a preset threshold or the throughput is less than a preset threshold, the network path is determined to be congested. Based on the identified problems, a corresponding traffic scheduling strategy is generated.

[0054] S108d, the generated traffic scheduling policy is compiled into new rules, and the rule decision tree is updated.

[0055] Okay, based on the detailed information you provided regarding steps S108a-S108c, the following is a clear and well-organized explanation of the beneficial effects extracted from a patent perspective. These effects closely revolve around the technical advantages brought about by closed-loop monitoring and dynamic scheduling.

[0056] Step S108 enables precise and real-time perception and localization of system performance bottlenecks. Through the collection and fusion analysis of multi-dimensional (network latency, throughput, node resource utilization, queue status) and fine-grained (constantly exceeding threshold) monitoring indicators, the system can accurately identify the specific location of performance bottlenecks (such as which server is overloaded or which link is congested) and the root cause (whether it is a CPU bottleneck, memory bottleneck, or network bandwidth bottleneck) from multiple levels, from macro-links to micro-processing nodes. This provides unprecedented data support for subsequent precise scheduling and overcomes the shortcomings of traditional methods, such as vague fault location and slow response.

[0057] Step S108 achieves a shift in operational mode from passive response to proactive prevention. Based on real-time data aggregation and cleaning using a time-series database, the system can anticipate performance degradation trends (such as a slow, continuous increase in CPU utilization and a gradual increase in queue length). This allows the dynamic scheduling algorithm to proactively generate scheduling strategies (such as load balancing before nodes reach 100% utilization) before performance bottlenecks fully erupt and impact business operations. This transforms passive fault handling into proactive performance assurance, significantly enhancing the system's stability and robustness in high-load scenarios such as e-commerce promotions.

[0058] Step S108 achieves refined resource scheduling and assurance based on business needs. The dynamic traffic scheduling algorithm generates highly targeted strategies based on identified specific problems (such as node A overload or path B congestion). For example, it diverts "product query" traffic from node A to node C; and switches "payment" traffic from path B to path D. This scheduling is no longer a simple "one-size-fits-all" approach, but rather a precise dynamic adjustment based on the type of business data and the real-time status of the system. This ensures that critical businesses (such as transactions and payments) always receive the necessary computing and network resources, thereby guaranteeing low latency and high availability even under high overall load.

[0059] Step S108 establishes a highly automated, self-optimizing intelligent closed-loop control system. Steps S108a-S108c form a complete "collection-analysis-decision-execution" closed loop. This system can continuously monitor, analyze, adjust, and optimize its own state without human intervention. This self-optimization capability enables the entire e-commerce data processing system to continuously adapt to dynamic changes in traffic and fluctuations in the state of internal nodes, always maintaining an optimal or near-optimal working state, significantly reducing operation and maintenance costs and improving resource utilization efficiency.

[0060] Step S108 provides reliable data for system capacity planning and optimization. The high-performance indicator data continuously collected and stored in the time-series database constitutes a "digital twin" of the system's operation. In-depth mining and analysis of this historical data can clearly reveal system bottleneck patterns and resource consumption trends, thus providing a solid and quantitative basis for future capacity expansion (e.g., whether to add servers) and architecture optimization (e.g., whether to adjust microservice deployment), making system optimization work targeted and effective.

[0061] This application also provides an e-commerce data intelligent processing system based on dynamic traffic optimization. The system includes: a data acquisition and standardization module, a rule matching and classification module, a dynamic priority marking module, a parallel processing and execution module, and a monitoring and dynamic scheduling module.

[0062] The data acquisition and standardization module is used to acquire e-commerce data from multiple e-commerce platforms, parse the source, data type and business characteristics of the e-commerce data, and convert the e-commerce data into a standardized data format with header fields that mark the source, data type and business characteristics.

[0063] The rule matching and classification module is used to construct a matching rule decision tree with multiple parallel processing nodes using the decision tree algorithm, and to classify the standardized e-commerce data using the rule matching algorithm to generate a mapping classification result between e-commerce data and matching rules.

[0064] The dynamic priority tagging module is used to dynamically add high-priority or low-priority tags to e-commerce data based on the mapping classification results, and to perform e-commerce data traffic scheduling in real time according to the priority tags, so that high-priority e-commerce data can be given priority to execute the corresponding rule actions.

[0065] The parallel processing and execution module is used to distribute the categorized e-commerce data to multiple parallel processing nodes in the rule decision tree, and execute the processing actions corresponding to the rules matching the e-commerce data.

[0066] The monitoring and dynamic scheduling module is used to continuously monitor key performance indicators such as network latency, throughput, and node processing status, provide real-time feedback on monitoring results, and use dynamic traffic scheduling algorithms to optimize e-commerce data traffic in real time.

[0067] It should be noted that the terms "first," "second," and similar terms used in this application specification and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, "a" or "one," and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. "A plurality" or "several" indicates at least two. Unless otherwise stated, terms such as "front," "back," "left," "right," "lower," and / or "upper" are for illustrative purposes only and are not limited to a location or spatial orientation. Terms such as "comprising" or "including" indicate that the elements or objects preceding "comprising" encompass the elements or objects listed following "comprising" or "including" and their equivalents, and do not exclude other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0068] The singular forms “a,” “the,” and “the” used in this application specification and appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0069] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. An intelligent e-commerce data processing method based on dynamic traffic optimization, characterized in that, The method includes: S101, acquire e-commerce data from different e-commerce platforms; S102, parse e-commerce data from different e-commerce platforms, identify the source, data type and business characteristics of the e-commerce data, and convert the e-commerce data into a standardized data format with header fields that mark the source, data type and business characteristics of the e-commerce data; S103, a matching rule decision tree is constructed using a decision tree algorithm, and the rule decision tree includes several parallel processing nodes; S104, The standardized e-commerce data is classified using a rule matching algorithm to obtain the mapping classification results between e-commerce data and matching rules; S105, based on the mapping and classification results of e-commerce data and matching rules, dynamically adds high-priority or low-priority tags to e-commerce data; S106, Real-time scheduling of e-commerce data traffic is performed based on high-priority or low-priority tags, so that e-commerce data with high-priority tags will be given priority in executing the actions corresponding to the e-commerce data matching rules. S107: Based on the mapping and classification results of e-commerce data and matching rules, the classified e-commerce data is distributed to multiple parallel processing nodes of the rule decision tree to execute the actions corresponding to the matching rules of the e-commerce data. S108 continuously monitors key performance indicators including network latency, throughput, and node processing status, provides real-time feedback on monitoring results, and employs a dynamic traffic scheduling algorithm for e-commerce data traffic scheduling.

2. The e-commerce data intelligent processing method based on dynamic traffic optimization according to claim 1, characterized in that, In step S101, acquiring e-commerce data from different e-commerce platforms includes: constructing a software-defined network architecture, which includes a separate data layer and a control layer. E-commerce data from different e-commerce platforms are uniformly accessed through the data layer. The data layer is configured with an open-source virtual switch, which forwards control actions and e-commerce data from the data layer to the control layer. The open-source virtual switch establishes a secure transmission channel with the control layer through transport layer security protocols and transport control protocols. The control layer receives flow tables issued by the open-source virtual switch based on open-source flow protocols, and the control layer executes actions on the e-commerce data according to the matching rules of the flow tables.

3. The e-commerce data intelligent processing method based on dynamic traffic optimization according to claim 2, characterized in that, In step S102, parsing e-commerce data from different e-commerce platforms and identifying the source, data type, and business characteristics of the e-commerce data includes: using an open-source virtual switch at the data access point to acquire incoming e-commerce data; using a filtering algorithm to divert the incoming e-commerce data and filter out non-business traffic or malicious traffic; extracting the basic network characteristics of the filtered e-commerce data to obtain the source and data type of the e-commerce data; parsing the parsed application layer payload of the filtered e-commerce data to identify the application layer protocol of the e-commerce data, and further parsing the request method and URL path of the application layer protocol; and determining the business characteristics of the e-commerce data based on the parsed request method and URL path.

4. The e-commerce data intelligent processing method based on dynamic traffic optimization according to claim 2, characterized in that, When the open-source virtual switch and the control layer begin establishing a connection, they exchange greeting messages. These messages contain the open-source flow protocol versions supported by both the open-source virtual switch and the control layer, and a response message confirms the connection. When e-commerce data enters the open-source virtual switch, it first searches for matching rules in the rule decision tree. If no matching rules are found in the rule decision tree, the open-source virtual switch sends the e-commerce data to the control layer for assistance. Upon receiving the e-commerce data, the control layer extracts its source, data type, and business characteristics. Based on a predefined rule database, it performs analysis and decision-making, and sends the decision results back to the open-source virtual switch to establish flow table entries. This allows the open-source virtual switch to classify the standardized e-commerce data using a rule matching algorithm.

5. The e-commerce data intelligent processing method based on dynamic traffic optimization according to claim 1, characterized in that, In step S103, constructing a matching rule decision tree using a decision tree algorithm includes: S103a, rules are divided into a set of rules based on the field length range of the rules, so as to generate a first-level data matching rule that includes several sets of rules; S103b, when the number of rules in a rule set in the first-level data matching rules exceeds half of the total number of rules in the first-level data matching rules, the rule set is split into smaller sub-rule sets to create the second-level data matching rules; when the number of rules in a rule set in the second-level data matching rules exceeds half of the total number of rules in the second-level data matching rules, the rule set is split into smaller sub-rule sets to create the next-level data matching rules, until the number of data matching rule layers reaches a preset threshold; S103c, calculate the index position of each rule set to facilitate rule splitting; S103d, construct a rule decision tree. The first root node of the rule decision tree includes all rules. Based on the first layer of data matching rules, the rules are divided into several rule sets, each rule set representing a child node. Based on the second layer of data matching rules, the rule sets are divided into several sub-rule sets, each sub-rule set representing a child node, until the number of rules in the sub-rule sets is less than a preset threshold and the number of data matching rule layers reaches a preset threshold, finally obtaining the rule decision tree.

6. The e-commerce data intelligent processing method based on dynamic traffic optimization according to claim 5, characterized in that, In steps S103a and S103b, if the field length range of the rule set is [0, ..., ... -1], w is the bit width of the field, then the Hash algorithm is used to store the rules in the rule set; the prefix field of the rule is used as the mask of its rule set, and the mask and the field value in the rule are logically ANDed to obtain the operation result, which is used to represent the index position of the rule stored in the rule set.

7. The e-commerce data intelligent processing method based on dynamic traffic optimization according to claim 2, characterized in that, In step S104, classifying the standardized e-commerce data using a rule matching algorithm includes: compiling the rules in the rule decision tree into a flow table containing several flow table entries, including the source, data type, business characteristics, and high-priority or low-priority tags of the e-commerce data; the open-source virtual switch matching the header fields of the standardized e-commerce data packets with several flow table entries in the flow table item by item; if the header fields of the e-commerce data packets match the flow table entries, the control layer executes the action corresponding to the matching rule for the e-commerce data.

8. The e-commerce data intelligent processing method based on dynamic traffic optimization according to claim 1, characterized in that, In step S106, the real-time execution of e-commerce data traffic scheduling based on high-priority or low-priority tags, so that e-commerce data with high-priority tags are given priority in executing the actions corresponding to the matching rules of e-commerce data, includes: establishing physically independent high-priority queues and low-priority queues; when outputting e-commerce data, sending e-commerce data to the high-priority queue or low-priority queue based on the high-priority or low-priority tags of the e-commerce data. Establish a high-priority consumer group and a low-priority consumer group. The high-priority consumer group is used to consume high-priority queues and has more computing resources, while the low-priority consumer group is used to consume low-priority queues and has relatively fewer resources.

9. The e-commerce data intelligent processing method based on dynamic traffic optimization according to claim 2, characterized in that, In step S108, the continuous monitoring includes key performance indicators such as network latency, throughput, and node processing status, providing real-time feedback of monitoring results, and employing a dynamic traffic scheduling algorithm for e-commerce data traffic scheduling, including: S108a collects network latency in real time from the software-defined network architecture, calculates throughput by the total amount of e-commerce data processed and the time spent processing this e-commerce data, and collects node processing status from parallel processing nodes, including CPU utilization, memory usage, processing queue length, and average processing time. S108b aggregates the obtained network latency, throughput and node processing status into a time series database, and cleans, aggregates and formats the network latency, throughput and node processing status in the time series database to obtain monitoring results. S108c: If the CPU utilization of a parallel processing node is detected to continuously exceed a preset threshold, it is determined to be a performance bottleneck. If the network latency of a link is detected to exceed a preset threshold or the throughput is less than a preset threshold, it is determined that the network path is congested. Based on the identified problems, a corresponding traffic scheduling strategy is generated. S108d, the generated traffic scheduling policy is compiled into new rules, and the rule decision tree is updated.

10. An e-commerce data intelligent processing system based on dynamic traffic optimization, characterized in that: The system includes: The data acquisition and standardization module is used to acquire e-commerce data from multiple e-commerce platforms, parse the source, data type and business characteristics of the e-commerce data, and convert the e-commerce data into a standardized data format with header fields that mark the source, data type and business characteristics. The rule matching and classification module is used to construct a matching rule decision tree with multiple parallel processing nodes using a decision tree algorithm, and to classify the standardized e-commerce data using a rule matching algorithm to generate a mapping classification result between e-commerce data and matching rules. The dynamic priority marking module is used to dynamically add high-priority or low-priority markings to e-commerce data based on the mapping classification results, and to perform e-commerce data traffic scheduling in real time according to the priority markings, so that high-priority e-commerce data can be given priority to execute the corresponding rule actions. The parallel processing and execution module is used to distribute the classified e-commerce data to multiple parallel processing nodes of the rule decision tree and execute the processing actions corresponding to the e-commerce data matching rules. The monitoring and dynamic scheduling module continuously monitors key performance indicators such as network latency, throughput, and node processing status, provides real-time feedback on monitoring results, and uses a dynamic traffic scheduling algorithm to optimize e-commerce data traffic in real time.

Images