An order flow dynamic optimization method and system based on big data analysis

Abstract

The application discloses an order flow dynamic optimization method and system based on big data analysis, relates to the technical field of order flow, and is used for solving the problem that it is difficult to identify the persistent congestion state caused by order flow inertia. After a target order enters a flow thread, when it is detected that the event queue depth meets the blocking early warning condition, the order backtracking cumulative number and the material tree hash value are collected, the order stubborn characteristics are formed, the historical passing rate is inquired, the resource squeezing type is analyzed in combination with the current node memory page frequency, the order cursor rollback frequency is weighted and corrected according to the resource squeezing type, the congestion inertia value reflecting the blocking trend is generated, the pre-reading cache or the flow limiting strategy is executed according to the congestion inertia value, the order throughput and the flow trend are continuously monitored, whether the blocking state is removed is judged, the order flow dynamic optimization closed loop is formed, the order flow efficiency is improved, and the processing blocking caused by resource contention is reduced.

Description

Technical Field

[0001] This invention relates to the field of order process technology, and more specifically, to a method and system for dynamic optimization of order processes based on big data analysis. Background Technology

[0002] As business processes become increasingly digitized, order processing systems typically employ a process-oriented approach based on thread scheduling, event queues, and memory caching mechanisms to automate order creation, material matching, status updates, and subsequent workflow processes. During order processing, due to differences in the material structure, historical processing paths, and node execution status of different orders, some orders may experience repeated rollbacks, continuous waiting, or multiple reschedulings at specific process nodes, resulting in localized congestion with a continuous impact on the order flow process.

[0003] The existing technology has the following shortcomings:

[0004] Currently, existing order process optimization technologies typically rely on static scheduling control based on the current load status of nodes, the number of tasks, or processing time. This approach fails to dynamically analyze the correlation between the historical flow behavior of orders, abnormal rollback characteristics, and the resource competition status of nodes. Consequently, it becomes difficult to identify persistent congestion caused by the inertia of order flow and to implement matching optimization and adjustment strategies for different types of resource squeeze situations. This can easily lead to reduced order processing efficiency and insufficient stability of process nodes. Therefore, this paper proposes a dynamic order process optimization method and system based on big data analysis.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method and system for dynamic optimization of order processes based on big data analysis. This method addresses the problems mentioned in the background art by employing order flow congestion feature extraction, resource squeeze status analysis, congestion inertia dynamic assessment, and adaptive pre-reading and flow limiting collaborative adjustment mechanism.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for dynamic optimization of order processes based on big data analysis, comprising the following steps:

[0008] Step S1: After the target order is passed to the flow thread, monitor the event queue depth of the current node to determine whether a blocking warning is triggered. If a blocking warning is triggered, collect the cumulative number of backtracking times of the target order and the material tree hash value of the bill of materials contained therein.

[0009] Step S2: Generate persistent characteristics of the target order by combining the material tree hash value and the cumulative number of backtracking, access the memory database to retrieve the historical throughput rate of persistent characteristics, collect the memory paging frequency of the current node, and analyze the resource squeeze type by combining the historical throughput rate;

[0010] Step S3: Access the operation log to read the cursor rollback frequency of the target order at the current node, set different rollback coefficients based on the resource rollback type, use the rollback coefficients to weight and correct the cursor rollback frequency and generate a congestion inertia value.

[0011] Step S4: After implementing the pre-reading rate limiting strategy based on the congestion inertia value, monitor the order throughput of the current node and evaluate the flow trend, and determine whether to lift the congestion warning based on the flow trend.

[0012] In a preferred embodiment, in step S1, after the target order enters the flow thread of the process engine, the current event queue depth is counted according to a fixed sampling period;

[0013] Event queue depth refers to the number of order events in the current node's event queue that are waiting to be processed but have not yet been completed.

[0014] Count the number of enqueued and dequeued events in the current sampling period, and calculate the current event queue depth growth rate;

[0015] Access the historical running data cache, read the event queue depth growth rate corresponding to each sampling period within the historical sliding window, and calculate the historical average growth rate;

[0016] When the current event queue depth growth rate is higher than the historical average growth rate, it is determined that the current node's processing capacity has been continuously lower than the order entry rate, thus triggering a congestion warning.

[0017] Otherwise, the blockage warning will not be triggered.

[0018] In a preferred embodiment, in step S1, after triggering the blocking warning, the cumulative number of times the target order has been automatically rolled back to the previous node by the process engine since entering the current order process due to process verification failure, transaction rollback or rule verification failure is read to obtain the cumulative number of rollbacks.

[0019] Access the order database to read the bill of materials corresponding to the target order, and build a material tree structure based on the bill of materials. The material tree structure takes the final product as the root node, the sub-components at all levels as child nodes, and the material inclusion relationship as the parent-child connection relationship. The complete tree structure is generated by recursive traversal.

[0020] After the material tree structure is completed, the node identifier, level number and parent-child relationship information are output according to the preset traversal order, and the material tree hash value is calculated using a hash algorithm.

[0021] In a preferred embodiment, in step S2, the material tree hash value is used as the order structure identifier, and the cumulative number of backtracking is used as the order history duplicate processing feature. The persistent features are generated by splicing the data according to a fixed field order.

[0022] After generating persistent features, access the in-memory database, use the persistent features as the query key to retrieve historical running records, count the number of orders that successfully completed the current node processing and the corresponding historical statistical period duration, and calculate the historical throughput rate.

[0023] Call the operating system performance monitoring interface to collect the total number of page swaps in and out within the current sampling period, obtain the memory page swap count corresponding to the current node's working set memory, and calculate the memory page swap frequency based on two consecutive sampling periods.

[0024] In a preferred embodiment, in step S2, the memory page swapping frequency corresponding to the previous sampling period is further read, and the memory page swapping frequency change rate is calculated.

[0025] When the historical throughput rate is less than the preset throughput rate threshold and the memory page swapping frequency change rate is greater than zero, the resource squeeze type is determined to be resource stress type.

[0026] When the historical throughput rate is greater than or equal to the preset throughput rate threshold and the memory page swapping frequency change rate is less than or equal to zero, the resource squeeze type is determined to be resource-sufficient.

[0027] In a preferred embodiment, in step S3, the operation log maintained by the process engine is accessed to count the cumulative number of reverse jumps of the database cursor and the corresponding statistical period duration, and the cursor rollback frequency of the target order is calculated.

[0028] Configure the corresponding squeeze coefficient based on the type of resource squeeze.

[0029] After obtaining the cursor backoff frequency and the congestion factor, the congestion inertia value is generated by performing a weighted correction on the cursor backoff frequency using the congestion factor.

[0030] In a preferred embodiment, in step S4, the pre-read rate limiting strategy includes a pre-read caching strategy or a rate limiting control strategy, and the pre-read caching strategy or the rate limiting control strategy is selected to be executed based on the congestion inertia value.

[0031] When the congestion inertia value is greater than or equal to the preset congestion threshold, the flow limiting control strategy is selected to be executed.

[0032] When the congestion inertia value is less than the preset congestion threshold, the pre-read caching strategy is selected.

[0033] When executing the rate limiting control strategy, read the change rate of the current node's memory page swapping frequency, and calculate the dynamic adjustment step size of the order entry based on the change rate of the memory page swapping frequency;

[0034] Send a rate limiting control command to the message middleware to reduce the rate at which subsequent orders are pushed into the current node according to the dynamic adjustment step size of the order entry, so as to reduce the spread of resource competition caused by the continuous entry of new orders into the current node.

[0035] In a preferred embodiment, in step S4, when the pre-read caching strategy is executed, the access process topology configuration table is used to read the data dependency relationship corresponding to the next node of the target order, and the target data set is generated according to the data dependency relationship;

[0036] Send a pre-read request to the cache management module to preload the target data set into the memory cache area, so that the target order can directly access the cached data in memory after entering the next node.

[0037] In a preferred embodiment, in step S4, after completing the rate limiting control strategy or the pre-read caching strategy, the number of orders that have changed from the processing state to the completed state in the current node is continuously monitored to obtain the order throughput of the current node.

[0038] Read the order throughput corresponding to the previous statistical period and calculate the order throughput change rate;

[0039] Recalculate the current node's event queue depth growth rate and compare it with the historical average growth rate.

[0040] When the current event queue depth growth rate is lower than the historical average growth rate, and the order throughput change rate is greater than or equal to zero, the blocking warning corresponding to the current node is determined to be lifted.

[0041] Otherwise, maintain the current blockage warning.

[0042] A dynamic order process optimization system based on big data analytics includes an early warning collection module, a retrieval and analysis module, a cursor correction module, and a strategy execution module. The functions of each module are as follows:

[0043] After the target order is passed to the flow thread, the early warning collection module monitors the event queue depth of the current node to determine whether a blocking early warning is triggered. If a blocking early warning is triggered, the module collects the cumulative number of backtracking events of the target order and the material tree hash value of the bill of materials contained therein. The material tree hash value and the cumulative number of backtracking events are then passed to the retrieval and analysis module.

[0044] The retrieval and analysis module generates persistent characteristics of the target order by combining the material tree hash value and the cumulative number of backtracking. It accesses the memory database to retrieve the historical throughput rate of the persistent characteristics, collects the memory page swapping frequency of the current node, and analyzes the resource squeeze type by combining the historical throughput rate. The resource squeeze type is then passed to the cursor correction module.

[0045] The cursor correction module accesses the operation log to read the cursor rollback frequency of the target order at the current node, sets different rollback coefficients based on the resource rollback type, uses the rollback coefficients to weight and correct the cursor rollback frequency and generate a congestion inertia value, and then passes the congestion inertia value into the strategy execution module.

[0046] After the strategy execution module executes the pre-reading rate limiting strategy based on the congestion inertia value, it monitors the order throughput of the current node and evaluates the flow trend, and determines whether to lift the congestion warning based on the flow trend.

[0047] The technical effects and advantages of this invention are as follows:

[0048] This invention, upon a target order entering the processing thread, collects the cumulative number of backtracking attempts and the corresponding material tree hash value of the bill of materials when the event queue depth meets the congestion warning condition. This information is combined to form a persistent order characteristic. Based on this characteristic, the historical throughput rate is queried. Simultaneously, the system resource congestion type is analyzed by combining the memory paging frequency of the current node. Based on the resource congestion type, different weighting adjustments are applied to the cursor backtracking frequency generated by the target order during its operation at the current node, generating a congestion inertia value that reflects the congestion persistence trend. Finally, based on the congestion inertia value, corresponding pre-read caching or rate limiting scheduling strategies are executed, and the node's order throughput and processing trend are continuously monitored to determine whether the congestion warning has been lifted. This forms a closed-loop control mechanism for order process optimization, improving order processing efficiency, reducing processing congestion caused by software resource contention, and enhancing the stability and adaptive optimization capabilities of the order process. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the implementation of a dynamic optimization method for order processes based on big data analysis, as described in this invention.

[0050] Figure 2 This is a module framework diagram of an order process dynamic optimization system based on big data analysis according to the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] This invention, upon a target order entering the processing thread, collects the cumulative number of backtracking attempts and the corresponding material tree hash value of the bill of materials when the event queue depth meets the congestion warning condition. This information is combined to form a persistent order characteristic. Based on this characteristic, the historical throughput rate is queried. Simultaneously, the system resource strain type is analyzed by combining the memory paging frequency of the current node. Depending on the resource strain type, different weighting adjustments are applied to the cursor backtracking frequency generated by the target order during its operation at the current node, generating a congestion inertia value that reflects the congestion persistence trend. Finally, based on the congestion inertia value, corresponding pre-read caching or rate limiting control scheduling strategies are executed, and the node's order throughput and processing trend are continuously monitored to determine whether the congestion warning has been lifted, thus forming a closed-loop control mechanism for order process optimization.

[0053] Example 1, as Figure 1 As shown, a method for dynamically optimizing order processes based on big data analytics includes the following steps:

[0054] Step S1: After the target order is passed to the flow thread, monitor the event queue depth of the current node to determine whether a blocking warning is triggered. If a blocking warning is triggered, collect the cumulative number of backtracking times of the target order and the material tree hash value of the bill of materials contained therein.

[0055] Step S2: Generate persistent characteristics of the target order by combining the material tree hash value and the cumulative number of backtracking, access the memory database to retrieve the historical throughput rate of persistent characteristics, collect the memory paging frequency of the current node, and analyze the resource squeeze type by combining the historical throughput rate;

[0056] Step S3: Access the operation log to read the cursor rollback frequency of the target order at the current node, set different rollback coefficients based on the resource rollback type, use the rollback coefficients to weight and correct the cursor rollback frequency and generate a congestion inertia value.

[0057] Step S4: After implementing the pre-reading rate limiting strategy based on the congestion inertia value, monitor the order throughput of the current node and evaluate the flow trend, and determine whether to lift the congestion warning based on the flow trend.

[0058] The specific implementation is as follows:

[0059] In step S1, after the target order enters the flow thread of the process engine, the running status of the current node event queue is continuously monitored, and the depth of the event queue at the current moment is counted according to a fixed sampling period.

[0060] Event queue depth refers to the number of order events in the current node's event queue that are waiting to be processed but have not yet been completed, reflecting the cumulative scale of tasks pending processing at the current node.

[0061] It should be noted that the process engine refers to the business process scheduling component deployed in the order processing system, used for scheduling order execution nodes, status transitions, rule validation, and transaction control. Internally, the process engine maintains an event queue, transition threads, node states, and process control logic. Based on the current processing status of the order, it calls the corresponding node's business processing module and records the order's transition information between nodes. When an order meets the node completion conditions, the process engine drives the order to the next processing node; when a transaction rollback or rule validation failure occurs, the process engine performs an automatic rollback operation according to the process definition.

[0062] Simultaneously, the number of enqueues and dequeues in the event queue within the current sampling period is counted, and the current event queue depth growth rate is calculated. The event queue depth growth rate is defined as the difference between the number of enqueues and dequeues within a unit sampling period. The larger the event queue depth growth rate, the more new pending orders are added per unit time than the number of completed orders, and the higher the order backlog of the current node. The smaller the event queue depth growth rate, the closer the current node's processing capacity is to or exceeds the order entry speed, and the event queue tends to be stable.

[0063] Subsequently, the historical running data cache is accessed, the event queue depth growth rate corresponding to each sampling period within the historical sliding window is read, and the historical average growth rate is calculated.

[0064] When the current event queue depth growth rate is higher than the historical average growth rate, it is determined that the current node's processing capacity has been continuously lower than the order entry rate, thus triggering a congestion warning.

[0065] Otherwise, the blockage warning will not be triggered.

[0066] After triggering a blocking alert, the order process log is accessed first to read the cumulative number of times the target order has been automatically rolled back to the previous node by the process engine since entering the current order process due to process validation failure, transaction rollback, or rule validation failure. This cumulative rollback count only includes rollbacks performed automatically and does not include rollbacks caused by manual intervention.

[0067] The cumulative backtracking count is used to characterize the degree of repeated processing of the target order in the current business process. The larger the value, the more times the target order undergoes repeated verification, the longer it occupies node processing resources, and the higher the possibility that the order will continue to be processed repeatedly after entering the current node again. The smaller the cumulative backtracking count, the faster the target order can complete the process and the lower the degree of repeated processing.

[0068] It should be noted that the order process log refers to the collection of full-process operation records automatically generated by the process engine during the order flow. It is used to record event information such as order entry node, departure node, node processing result, transaction commit, transaction rollback, rule verification result, and automatic rollback.

[0069] Subsequently, the order database is accessed to retrieve the bill of materials corresponding to the target order, and a material tree structure is constructed based on the bill of materials. The material tree structure uses the final product as the root node, each level of sub-component as a child node, and the material inclusion relationship as the parent-child connection relationship, generating a complete tree structure through recursive traversal.

[0070] After constructing the material tree structure, the identifiers, level numbers, and parent-child relationship information of each node are output according to the preset traversal order, and the material tree hash value is calculated using a hash algorithm. Different material structures generate different material tree hash values, and the same material structure generates the same material tree hash value. Therefore, the material tree hash value is used to identify the material structure type of an order. When the material tree hierarchy is more complex and the node relationships are richer, the generated material tree hash value corresponds to a more complex material structure type, which means that the target order needs to undergo more business rule parsing and material expansion processing; when the material tree structure is relatively simple, the corresponding processing path is shorter and the number of node visits is less.

[0071] It should be noted that the order database is a relational or distributed database used to store order business data. It internally stores basic order information, bill of materials, order status, node execution results, and related business attributes. The hash algorithm is a data processing algorithm that converts input data of arbitrary length into a hash value of fixed length according to a preset mathematical mapping rule. Its characteristic is that the same input data always generates the same hash value, while different input data will generate different hash values ​​under normal circumstances.

[0072] In step S2, the material tree hash value and the cumulative backtracking count are received. The material tree hash value is used as the order structure identifier, and the cumulative backtracking count is used as the order history duplicate processing feature. These are concatenated according to a fixed field order to generate a persistent feature, the expression of which is:

[0073] ;

[0074] in, This indicates the persistent nature of the target order; Represents the material tree hash value; Indicates the cumulative number of backtracking iterations; This indicates a string concatenation operation.

[0075] Since the material tree hash value can uniquely identify the material structure type of an order, and the cumulative number of backtracking can reflect the degree of repeated processing in the order's history, orders with the same material structure but different degrees of repeated processing correspond to different stubborn features. Orders with different material structures also generate different stubborn features, thereby ensuring that stubborn features can uniquely correspond to a set of orders with the same structural complexity and the same historical processing behavior, providing a unified index for subsequent historical data retrieval.

[0076] After generating persistent features, the system accesses the in-memory database, uses the persistent features as the query key to retrieve historical execution records, and reads the processing results of all orders with the same persistent features within the historical statistical period.

[0077] It should be noted that an in-memory database refers to a high-speed data management module that uses physical memory as the primary data storage medium, and is used to cache historical order operation statistics and operation analysis data.

[0078] The in-memory database stores the entry time, exit time, and completion status of orders corresponding to each stubborn feature. Based on the query results, the number of orders successfully processed at the current node and the corresponding historical statistical period duration are counted, and the historical throughput rate is calculated using the following formula:

[0079] ;

[0080] in, Indicates the rate at which history passes; This represents the number of orders that share the same persistent characteristics and have successfully completed processing at the current node within the statistical period. This indicates the duration of the corresponding historical statistical period.

[0081] Historical throughput rate reflects the average ability of orders with the same business complexity to complete the current node processing in the historical process. The higher the value, the faster the node processing can be completed and the shorter the time occupied by node resources. The lower the historical throughput rate, the more likely the same type of order has long been characterized by low processing efficiency and long resource occupation time.

[0082] Subsequently, the operating system performance monitoring interface is invoked to collect the total number of page swaps in and out within the current sampling period, obtaining the memory page swap count corresponding to the current node's working set memory. The memory page swap frequency is then calculated based on two consecutive sampling periods, using the following formula:

[0083] ;

[0084] in, This indicates the memory page swapping frequency during the current sampling period; This indicates the total number of page ingestion and page outgestion events that occurred within the current sampling period; This indicates the duration of the current sampling period.

[0085] It should be noted that the operating system performance monitoring interface is a system performance collection interface that the operating system exposes to upper-layer applications. It is used to obtain the running status information of the current running node, such as memory, processor, disk, and threads.

[0086] Memory paging frequency reflects the level of data exchange between physical memory and disk during the operation of the current node. A higher memory paging frequency indicates that the current node frequently undergoes page replacement, the working set data cannot reside continuously in physical memory, and the resource contention is high; a lower memory paging frequency indicates that the working set can reside stably in physical memory, and resource utilization is relatively sufficient.

[0087] To reflect the development trend of memory resource competition, the memory page swapping frequency corresponding to the previous sampling period is further read, and the rate of change of memory page swapping frequency is calculated. The calculation formula is as follows:

[0088] ;

[0089] in, This represents the rate of change in memory page swapping frequency; This indicates the memory page swapping frequency during the current sampling period; This indicates the memory page swapping frequency in the previous sampling period.

[0090] When the rate of change of memory page swapping frequency is greater than zero, it indicates that the memory page swapping behavior of the current node is continuously increasing, and the degree of physical memory contention is increasing. The larger the value, the faster the memory resource contention is growing. When the rate of change of memory page swapping frequency is close to zero, it indicates that the memory resource usage status remains basically stable. When the rate of change of memory page swapping frequency is less than zero, it indicates that the page swapping frequency is decreasing, and the memory resource pressure of the current node is gradually easing.

[0091] After obtaining the historical throughput rate and the memory paging frequency change rate, the resource squeeze type is analyzed together. Specifically, when the historical throughput rate is less than the preset throughput rate threshold and the memory paging frequency change rate is greater than zero, it indicates that the historical processing efficiency of similar orders is low, and at the same time, the physical memory competition of the current node continues to intensify. It is determined that the resource competition of the current node has become the main factor affecting the order processing efficiency, and the resource squeeze type is identified as resource shortage type.

[0092] When the historical throughput rate is greater than or equal to the preset throughput rate threshold and the memory page swapping frequency change rate is less than or equal to zero, it indicates that the historical processing efficiency of similar orders is relatively high, the memory resources of the current node remain stable, and it is determined that there is no obvious resource competition in the current node. The resource squeeze type is then identified as resource-sufficient.

[0093] It should be noted that the preset throughput threshold is generated based on historical operating data. Specifically, the historical throughput rates corresponding to all persistent features within the historical statistical period are statistically analyzed to form a historical throughput rate sample set. The sample set is then sorted from smallest to largest value, and the median of the sorted values ​​is selected as the preset throughput threshold.

[0094] In step S3, the software operation behavior of the target order is further analyzed based on the resource squeeze type. Specifically, the operation log maintained by the access process engine is accessed, the cursor access records generated during the database query execution of the target order at the current node are read, and the cumulative number of reverse jumps of the database cursor and the corresponding statistical period duration are counted to calculate the cursor rollback frequency of the target order. The calculation formula is as follows:

[0095] ;

[0096] in, Indicates the cursor back-off frequency; This indicates the cumulative number of reverse jumps that occurred by the database cursor within the statistical period; This indicates the duration of the corresponding statistical period.

[0097] It should be noted that the operation log is a software operation log jointly maintained by the database management system and the process engine. It is used to record various operation events during database access, including SQL execution records, transaction commit records, transaction rollback records, cursor movement records, lock wait records, and exception retry records. A cursor is a data access control object established by the database management system after executing a query statement. It is used to read records in the query result set one by one according to a predetermined access order. Cursor rollback refers to the operation behavior of repositioning the database cursor from the current position to the previous record to continue reading during the process of reading order records due to transaction retries, record lock waits, or consistency check failures.

[0098] Cursor rollback frequency is used to characterize the degree to which the same data area is repeatedly accessed during the processing of a target order. The higher the value, the more times repeated scans or transaction rollbacks occur during the database reading process, the higher the degree of repetition in the software execution path, and the longer the database resources are occupied. The lower the cursor rollback frequency, the better the continuity of the database reading process and the fewer repeated accesses occur during the execution of transactions.

[0099] Subsequently, a corresponding congestion coefficient is configured based on the resource congestion type. This congestion coefficient originates from a resource type mapping table pre-maintained by the process engine and is used to describe the weight of the impact of cursor rollback behavior on system congestion levels under different resource operating states. When the resource congestion type is resource-scarce, it indicates that the current node's memory resource competition is significant, and a single cursor rollback is more likely to trigger lock wait propagation and transaction blocking. Therefore, a higher congestion coefficient is read from the resource type mapping table.

[0100] When the resource squeeze type is resource-sufficient, it means that the current node's resources can meet the processing needs. Cursor rollback reflects the business processing characteristics of the order itself, so the corresponding lower squeeze coefficient is read.

[0101] The larger the squeeze coefficient, the greater the impact of cursor rollback behavior on congestion diffusion in the current resource environment; the smaller the squeeze coefficient, the more the cursor rollback behavior is mainly a local business process and has a weaker impact on overall resource competition.

[0102] It should be noted that the resource type mapping table is a strategy configuration table pre-maintained by the process engine, used to establish the mapping relationship between different resource demand types and their corresponding demand coefficients.

[0103] After obtaining the cursor backoff frequency and the congestion factor, the congestion inertia value is generated by weighting the cursor backoff frequency using the congestion factor and calculating it using the following formula:

[0104] ;

[0105] in, This represents the congestion inertia value; Indicates the cursor back-off frequency; This represents the run-out coefficient.

[0106] Congestion inertia is used to comprehensively characterize the trend of a target order continuously occupying software resources under the current resource environment and causing subsequent congestion to spread. It not only reflects the repeated access behavior of the database cursor, but also reflects the amplifying effect of the resource contention state on the congestion persistence when the behavior occurs.

[0107] When the congestion inertia value is large, it indicates that the target order will continue to experience high frequency cursor rollback in a resource contention environment, the probability of repeated execution of database transactions is high, and the target order is likely to continue to occupy lock resources and computing resources in subsequent processing, which can easily lead to continuous processing blockage.

[0108] When the congestion inertia value is small, it indicates that there is less cursor backoff behavior or that the current resource environment is relatively abundant. The target order has little impact on the overall processing capacity of the system, and the possibility of subsequent resource competition escalation is low.

[0109] In step S4, a pre-read rate limiting strategy is executed based on the congestion level corresponding to the congestion inertia value. The pre-read rate limiting strategy includes a pre-read caching strategy or a rate limiting control strategy. The pre-read caching strategy or the rate limiting control strategy is selected to be executed based on the congestion inertia value.

[0110] Specifically, when the congestion inertia value is greater than or equal to the preset congestion threshold, a flow limiting control strategy is selected to be implemented.

[0111] When the congestion inertia value is less than the preset congestion threshold, the pre-read caching strategy is selected.

[0112] It should be noted that the preset congestion threshold is established based on a sample of congestion inertia values ​​corresponding to historical orders. Specifically, the congestion inertia values ​​corresponding to all orders during the historical operation are statistically analyzed to form a sample set of congestion inertia values. These values ​​are then sorted in ascending order, and the upper quartile of the sample set is selected as the preset congestion threshold.

[0113] Since the congestion inertia value reflects the trend of target orders continuously occupying software resources and forming congestion propagation under the current resource environment, the larger the congestion inertia value, the higher the possibility that the current orders will continue to intensify the competition for node resources, and the speed of new orders should be reduced first; the smaller the congestion inertia value, the less obvious the resource competition at the current node, and the more important it is to reduce the data access waiting time of subsequent nodes to further improve order processing efficiency.

[0114] When implementing a rate limiting control strategy, the current node's memory page swapping frequency change rate is read, and the dynamic adjustment step size for the order entry is calculated based on this rate. The calculation formula is as follows:

[0115] ;

[0116] in, This indicates that the step size for dynamically adjusting the order entry point is... This represents the rate of change in memory page swapping frequency; This represents the preset step size adjustment coefficient, which is used to map the degree of change in memory resources to the adjustment range of the order entry rate.

[0117] The preset step size adjustment coefficient is obtained offline based on the correspondence between the historical memory page swapping frequency change rate and the order throughput change rate. Specifically, the memory page swapping frequency change rate and the corresponding order throughput change rate are collected over multiple historical operating cycles, and a model of the relationship between the two is established. Under the condition that the order throughput change rate remains non-negative, a proportional coefficient that keeps the dynamic adjustment step size of the order entry consistent with the change in memory resource contention is obtained, and this proportional coefficient is used as the preset step size adjustment coefficient.

[0118] The higher the rate of change in memory paging frequency, the faster the physical memory contention at the current node increases. In this case, the dynamic adjustment step size for order entry should be larger, and the reduction in the rate of subsequent order entry should be greater. Conversely, the lower the rate of change in memory paging frequency, the slower the resource contention increases, and the corresponding dynamic adjustment step size for order entry should be smaller.

[0119] Subsequently, a rate limiting control command is sent to the message middleware to reduce the rate at which subsequent orders are pushed into the current node according to the dynamic adjustment step size of the order entry, in order to reduce the spread of resource competition caused by the continuous entry of new orders into the current node.

[0120] When executing the pre-read caching strategy, the process topology configuration table is first accessed to retrieve the data dependencies corresponding to the next node of the target order. Based on these dependencies, a target data set is generated, and a pre-read request is sent to the cache management module. This pre-loads the target data set into the memory cache area, allowing the target order to directly access the cached data in memory when it enters the next node, without needing to initiate a database query again. This reduces the number of database accesses and page scheduling operations. After the target data is pre-loaded, the cache index is updated so that the next node can directly access the cached data, improving the efficiency of continuous order processing.

[0121] It should be noted that the process topology configuration table is a process configuration data table maintained by the process engine, used to store the connection relationships and node dependency information between nodes in the order process; the cache management module is a data cache management component deployed inside the process engine, used to complete the loading, updating, eviction and index maintenance of hot business data.

[0122] After implementing the rate limiting or pre-read caching strategy, continuously monitor the number of orders that have transitioned from the processing state to the completed state in the current node to obtain the order throughput of the current node.

[0123] Order throughput reflects the current node's actual order processing capacity per unit of time. A higher order throughput indicates a stronger processing capacity for the current node; a lower order throughput indicates a lower processing efficiency for the current node.

[0124] Subsequently, the order throughput corresponding to the previous statistical period is read, and the order throughput change rate is calculated using the following formula:

[0125] ;

[0126] in, This indicates the rate of change in order throughput; This indicates the order throughput for the current statistical period; This represents the order throughput of the previous statistical period.

[0127] The order throughput change rate is used to reflect the development trend of the current node's processing capacity. A value greater than zero indicates that the order processing capacity continues to improve; a value equal to zero indicates that the processing capacity remains stable; and a value less than zero indicates that the processing capacity is still declining.

[0128] At the same time, the current node's event queue depth growth rate is recalculated and compared with the historical average growth rate. When the current event queue depth growth rate is lower than the historical average growth rate, and the order throughput change rate is greater than or equal to zero, it indicates that the backlog of new orders has been alleviated, and the node's processing capacity has returned to stable growth. This indicates that the current node's flow trend has returned to normal, and the corresponding congestion warning for the current node is lifted.

[0129] Otherwise, the current congestion warning will be maintained, and congestion inertia value analysis and scheduling strategy updates will continue to be performed according to the preset sampling period. The software scheduling strategy will be continuously adjusted according to the node's operating status, thereby achieving dynamic recovery and stable operation of order process processing capacity without increasing hardware resources.

[0130] Example 2, as Figure 2 As shown, an order process dynamic optimization system based on big data analysis includes an early warning collection module, a retrieval and analysis module, a cursor correction module, and a strategy execution module. The functions of each module are as follows:

[0131] After the target order is passed to the flow thread, the early warning collection module monitors the event queue depth of the current node to determine whether a blocking early warning is triggered. If a blocking early warning is triggered, the module collects the cumulative number of backtracking events of the target order and the material tree hash value of the bill of materials contained therein. The material tree hash value and the cumulative number of backtracking events are then passed to the retrieval and analysis module.

[0132] The retrieval and analysis module generates persistent characteristics of the target order by combining the material tree hash value and the cumulative number of backtracking. It accesses the memory database to retrieve the historical throughput rate of the persistent characteristics, collects the memory page swapping frequency of the current node, and analyzes the resource squeeze type by combining the historical throughput rate. The resource squeeze type is then passed to the cursor correction module.

[0133] The cursor correction module accesses the operation log to read the cursor rollback frequency of the target order at the current node, sets different rollback coefficients based on the resource rollback type, uses the rollback coefficients to weight and correct the cursor rollback frequency and generate a congestion inertia value, and then passes the congestion inertia value into the strategy execution module.

[0134] After the strategy execution module executes the pre-reading rate limiting strategy based on the congestion inertia value, it monitors the order throughput of the current node and evaluates the flow trend, and determines whether to lift the congestion warning based on the flow trend.

[0135] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0136] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0137] In this document, the singular forms “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that terms such as “comprising / including” or “having” specify the presence of the stated features, integrals, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, integrals, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0138] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0139] The above description of the disclosed embodiments will enable those skilled in the art to make or use various modifications to these embodiments. It will be readily apparent to those skilled in the art that the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for dynamic optimization of order processes based on big data analysis, characterized in that: Includes the following steps: Step S1: After the target order is passed to the flow thread, monitor the event queue depth of the current node to determine whether a blocking warning is triggered. If a blocking warning is triggered, collect the cumulative number of backtracking times of the target order and the material tree hash value of the bill of materials contained therein. Step S2: Generate persistent characteristics of the target order by combining the material tree hash value and the cumulative number of backtracking, access the memory database to retrieve the historical throughput rate of persistent characteristics, collect the memory paging frequency of the current node, and analyze the resource squeeze type by combining the historical throughput rate; Step S3: Access the operation log to read the cursor rollback frequency of the target order at the current node, set different rollback coefficients based on the resource rollback type, use the rollback coefficients to weight and correct the cursor rollback frequency and generate a congestion inertia value. Step S4: After implementing the pre-reading rate limiting strategy based on the congestion inertia value, monitor the order throughput of the current node and evaluate the flow trend, and determine whether to lift the congestion warning based on the flow trend.

2. The method for dynamic optimization of order processes based on big data analysis according to claim 1, characterized in that: In step S1, after the target order enters the workflow engine's processing thread, the current event queue depth is calculated according to a fixed sampling period. Event queue depth refers to the number of order events in the current node's event queue that are waiting to be processed but have not yet been completed. Count the number of enqueued and dequeued events in the current sampling period, and calculate the current event queue depth growth rate; Access the historical running data cache, read the event queue depth growth rate corresponding to each sampling period within the historical sliding window, and calculate the historical average growth rate; When the current event queue depth growth rate is higher than the historical average growth rate, it is determined that the current node's processing capacity has been continuously lower than the order entry rate, thus triggering a congestion warning. Otherwise, the blockage warning will not be triggered.

3. The method for dynamic optimization of order processes based on big data analysis according to claim 1, characterized in that: In step S1, after triggering the blocking warning, the cumulative number of times the target order has been automatically rolled back to the previous node by the process engine since entering the current order process due to process verification failure, transaction rollback or rule verification failure is read, and the cumulative number of rollbacks is obtained. Access the order database to read the bill of materials corresponding to the target order, and build a material tree structure based on the bill of materials. The material tree structure takes the final product as the root node, the sub-components at all levels as child nodes, and the material inclusion relationship as the parent-child connection relationship. The complete tree structure is generated by recursive traversal. After the material tree structure is constructed, the node identifier, level number and parent-child relationship information are output according to the preset traversal order, and the material tree hash value is calculated using a hash algorithm.

4. The method for dynamic optimization of order processes based on big data analysis according to claim 3, characterized in that: In step S2, the material tree hash value is used as the order structure identifier, and the cumulative number of backtracking is used as the order history duplicate processing feature. The two are then concatenated according to a fixed field order to generate persistent features. After generating persistent features, access the in-memory database, use the persistent features as the query key to retrieve historical running records, count the number of orders that successfully completed the current node processing and the corresponding historical statistical period duration, and calculate the historical throughput rate. Call the operating system performance monitoring interface to collect the total number of page swaps in and out within the current sampling period, obtain the memory page swap count corresponding to the current node's working set memory, and calculate the memory page swap frequency based on two consecutive sampling periods.

5. The method for dynamic optimization of order processes based on big data analysis according to claim 4, characterized in that: In step S2, the memory page swapping frequency corresponding to the previous sampling period is further read, and the memory page swapping frequency change rate is calculated. When the historical throughput rate is less than the preset throughput rate threshold and the memory page swapping frequency change rate is greater than zero, the resource squeeze type is determined to be resource stress type. When the historical throughput rate is greater than or equal to the preset throughput rate threshold and the memory page swapping frequency change rate is less than or equal to zero, the resource squeeze type is determined to be resource-sufficient.

6. The method for dynamic optimization of order processes based on big data analysis according to claim 5, characterized in that: In step S3, the operation log maintained by the process engine is accessed to count the cumulative number of reverse jumps of the database cursor and the corresponding statistical period duration, and the cursor rollback frequency of the target order is calculated. Configure the corresponding squeeze coefficient based on the type of resource squeeze. After obtaining the cursor backoff frequency and the congestion factor, the congestion inertia value is generated by performing a weighted correction on the cursor backoff frequency using the congestion factor.

7. The method for dynamic optimization of order processes based on big data analysis according to claim 6, characterized in that: In step S4, the pre-read rate limiting strategy includes a pre-read caching strategy or a rate limiting control strategy, and the pre-read caching strategy or the rate limiting control strategy is selected to be executed based on the congestion inertia value; When the congestion inertia value is greater than or equal to the preset congestion threshold, the flow limiting control strategy is selected to be executed. When the congestion inertia value is less than the preset congestion threshold, the pre-read caching strategy is selected. When executing the rate limiting control strategy, read the change rate of the current node's memory page swapping frequency, and calculate the dynamic adjustment step size of the order entry based on the change rate of the memory page swapping frequency; Send a rate limiting control command to the message middleware to reduce the rate at which subsequent orders are pushed into the current node according to the dynamic adjustment step size of the order entry, so as to reduce the spread of resource competition caused by the continuous entry of new orders into the current node.

8. The method for dynamic optimization of order processes based on big data analysis according to claim 7, characterized in that: In step S4, when the pre-read caching strategy is executed, the access process topology configuration table is used to read the data dependency relationship corresponding to the next node of the target order, and the target data set is generated based on the data dependency relationship; Send a pre-read request to the cache management module to preload the target data set into the memory cache area, so that the target order can directly access the cached data in memory after entering the next node.

9. The method for dynamic optimization of order processes based on big data analysis according to claim 1, characterized in that: In step S4, after completing the rate limiting control strategy or the pre-read caching strategy, the number of orders that have changed from the processing state to the completed state in the current node is continuously monitored to obtain the order throughput of the current node. Read the order throughput corresponding to the previous statistical period and calculate the order throughput change rate; Recalculate the current node's event queue depth growth rate and compare it with the historical average growth rate. When the current event queue depth growth rate is lower than the historical average growth rate, and the order throughput change rate is greater than or equal to zero, the blocking warning corresponding to the current node is lifted. Otherwise, maintain the current blockage warning.

10. A dynamic order process optimization system based on big data analysis, used to implement the dynamic order process optimization method based on big data analysis as described in any one of claims 1-9, characterized in that: It includes an early warning acquisition module, a retrieval and analysis module, a cursor correction module, and a strategy execution module. The functions of each module are as follows: After the target order is passed to the flow thread, the early warning collection module monitors the event queue depth of the current node to determine whether a blocking early warning is triggered. If a blocking early warning is triggered, the module collects the cumulative number of backtracking events of the target order and the material tree hash value of the bill of materials contained therein. The material tree hash value and the cumulative number of backtracking events are then passed to the retrieval and analysis module. The retrieval and analysis module generates persistent characteristics of the target order by combining the material tree hash value and the cumulative number of backtracking. It accesses the memory database to retrieve the historical throughput rate of the persistent characteristics, collects the memory page swapping frequency of the current node, and analyzes the resource squeeze type by combining the historical throughput rate. The resource squeeze type is then passed to the cursor correction module. The cursor correction module accesses the operation log to read the cursor rollback frequency of the target order at the current node, sets different rollback coefficients based on the resource rollback type, uses the rollback coefficients to weight and correct the cursor rollback frequency and generate a congestion inertia value, and then passes the congestion inertia value into the strategy execution module. After the strategy execution module executes the pre-reading rate limiting strategy based on the congestion inertia value, it monitors the order throughput of the current node and evaluates the flow trend, and determines whether to lift the congestion warning based on the flow trend.

Images