Batch full-link view processing method and system and related equipment

Abstract

The invention discloses a processing method and system for batch full-link views and related equipment, and relates to the technical field of view processing, the original batch full-link views are intelligently compressed and optimized to obtain a plurality of sub-views, the view scale is remarkably reduced, the core topological relation and key paths of batch processing are completely reserved in the sub-views, and the processing efficiency of the batch full-link views is improved. And the topology integrity is ensured. In addition, order edges conforming to conditions are added to batch links conforming to importance evaluation conditions, acceleration processing operation is performed on a full-link view through a distributed processing architecture, and due to the fact that a core topological relation and a key path of batch processing are completely reserved in a sub-graph, topological integrity is ensured, and the processing efficiency is improved. Even under a cross-system and multi-level complex dependency relationship, the positioning efficiency and the calculation efficiency of the batch operation abnormity can be improved.

Description

Technical Field

[0001] This application relates to the technical field of view processing, and more specifically, to a method, system, and related equipment for processing batch full-link views. Background Technology

[0002] Traditional graph summarization algorithms, as a key technology for handling large-scale graph data processing, have evolved from compressing basic graph structures to multi-modal, dynamic, and intelligent processing. Traditional graph summarization algorithms compress large-scale graph data into smaller subgraphs or statistical representations, preserving key topological structures and reducing computational complexity.

[0003] Due to the exponential growth in the complexity of system architecture and the scale of data in the financial industry, core trading systems in the financial industry need to process batch jobs involving multiple related systems every day, forming an ultra-large-scale end-to-end view containing tens of thousands of nodes and cross-system dependencies. Because core transactions in the financial industry involve complex dependencies across systems and multiple levels, the computational efficiency of traditional graph summarization algorithms is insufficient to meet the processing requirements as the node scale continues to expand and the complexity continues to increase.

[0004] Therefore, how to improve computational efficiency in the face of complex dependencies involving cross-systems and multiple levels is a problem that this application urgently needs to solve. Summary of the Invention

[0005] In view of this, this application discloses a batch full-link view processing method, system and related equipment, which can improve the efficiency of locating batch operation anomalies and computational efficiency even when complex dependencies involving cross systems and multiple levels are involved.

[0006] To achieve the above objectives, the disclosed technical solution is as follows:

[0007] The first aspect of this application discloses a method for processing batch end-to-end views, the method comprising:

[0008] Obtain the original batch full-link view, and intelligently compress and optimize the original batch full-link view to obtain multiple sub-graphs; wherein each sub-graph corresponds to a set of local links with business relationships;

[0009] Hierarchical adaptive importance assessment is performed within the multiple subgraphs to obtain a batch of links that meet the importance assessment conditions;

[0010] Add qualified edges to the batch links to obtain an initial cross-system batch full link view;

[0011] The initial cross-system batch full-link view is processed and computed using a distributed processing architecture to obtain the target batch full-link view.

[0012] The trunk and branches of the target batch full-link view are displayed through a hierarchical layout.

[0013] Preferably, the step of obtaining the original batch full-link view and intelligently compressing and optimizing the original batch full-link view to obtain multiple sub-graphs includes:

[0014] Perform batch link monitoring in layers to obtain the original batch full link view;

[0015] The original batch full-link view is intelligently compressed and optimized using a community detection algorithm to segment it into multiple subgraphs.

[0016] Preferably, the step of accelerating the processing of the initial cross-system batch full-link view through a distributed processing architecture to obtain the target batch full-link view includes:

[0017] In the computing layer of the distributed processing architecture, the preset core algorithm is distributed and modified according to the parallel graph computing framework. Then, the initial cross-system batch full-link view is traversed once by the core algorithm after the distributed modification and the random sampling method to synchronously complete the bidirectional statistics of in-degree and out-degree.

[0018] By building a multi-stream concurrent pipeline for graphics processors in the physical layer of a distributed processing architecture, batch processing is accelerated on the full-link view that has completed bidirectional in-degree statistics to obtain the target batch full-link view.

[0019] A distributed graph database is built in the storage layer of the distributed processing architecture to store the target batch full-link view.

[0020] Preferably, the hierarchical layout of the trunk and branches of the target batch full-link view includes:

[0021] The hierarchical layout engine, along with a tree-like folding strategy based on a force-directed algorithm, is used to display the trunk structure and branch details of the target batch full-link view.

[0022] Preferred options also include:

[0023] The trunk and branches of the target batch full-link view are evaluated according to preset evaluation indicators;

[0024] The evaluation of the trunk and branches of the target batch full-link view based on preset evaluation indicators includes:

[0025] Calculate the compression rate and batch link retention rate of the target batch full-link view;

[0026] Based on the compression rate, the batch link retention rate, and the metric deviation, the backbone structure and branch details of the target batch full-link view are quantitatively evaluated.

[0027] A second aspect of this application discloses a batch end-to-end view processing system, the system comprising:

[0028] The acquisition unit is used to acquire the original batch full-link view and intelligently compress and optimize the original batch full-link view to obtain multiple sub-graphs; wherein each sub-graph corresponds to a set of local links with business relationships;

[0029] The first evaluation unit is used to perform hierarchical adaptive importance evaluation within the multiple subgraphs to obtain a batch of links that meet the importance evaluation conditions.

[0030] An addition unit is used to add qualified order edges to the batch links to obtain an initial cross-system batch full link view;

[0031] An accelerated processing unit is used to perform accelerated processing on the initial cross-system batch full-link view through a distributed processing architecture to obtain the target batch full-link view.

[0032] The display unit is used to display the trunk and branches of the target batch full-link view through a hierarchical layout.

[0033] Preferably, the acquisition unit includes:

[0034] The acquisition module is used to perform batch link monitoring in a hierarchical manner to obtain the original batch full link view;

[0035] The segmentation module is used to intelligently compress and optimize the original batch full-link view using a community detection algorithm to segment it into multiple subgraphs.

[0036] Preferably, the accelerated processing unit includes:

[0037] The modified traversal module is used to perform distributed modification of the preset core algorithm in the computing layer of the distributed processing architecture according to the parallel graph computing framework, and to perform a single graph traversal on the initial cross-system batch full-link view through the core algorithm after distributed modification and the random sampling method, so as to simultaneously complete the bidirectional statistics of in-degree and out-degree.

[0038] The batch processing acceleration module is used to accelerate the batch processing of the full-link view that has completed bidirectional in-degree statistics by building a multi-stream concurrent pipeline of the graphics processor in the physical layer of the distributed processing architecture, so as to obtain the target batch full-link view.

[0039] A storage module is constructed to build a distributed graph database in the storage layer of the distributed processing architecture to store the target batch full-link view.

[0040] A third aspect of this application discloses a storage medium comprising stored instructions, wherein, when the instructions are executed, the device in which the storage medium resides executes a batch end-to-end view processing method as described in any one of the first aspects.

[0041] The fourth aspect of this application discloses an electronic device including a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors using the batch full-link view processing method as described in any one of the first aspects.

[0042] As described above, this application discloses a method, system, and related equipment for processing batch full-link views. The method involves obtaining an original batch full-link view, intelligently compressing and optimizing it to obtain multiple subgraphs. Each subgraph corresponds to a set of local links with business relationships. Hierarchical adaptive importance assessment is performed within these subgraphs to obtain batch links that meet the importance assessment criteria. Edges of the appropriate order are added to these batch links to obtain an initial cross-system batch full-link view. A distributed processing architecture is used to accelerate the processing of this initial cross-system batch full-link view to obtain a target batch full-link view. A hierarchical layout is then used to display the trunk and branches of the target batch full-link view. This approach intelligently compresses and optimizes the original batch full-link view to obtain multiple subgraphs, significantly reducing the view size while fully preserving the core topological relationships and critical paths of batch processing within each subgraph, ensuring topological integrity. Furthermore, it adds qualified order edges to batch links that meet the importance assessment criteria, and accelerates the processing of the entire link view through a distributed processing architecture. Since the core topological relationships and critical paths of batch processing are fully preserved in the subgraph, ensuring topological integrity, it can improve the efficiency of locating batch operation anomalies and computational efficiency even when complex dependencies involving cross-systems and multiple levels are involved. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0044] Figure 1This is a flowchart illustrating a batch end-to-end view processing method disclosed in an embodiment of this application;

[0045] Figure 2 This is a schematic diagram of the structure of a batch end-to-end view processing system disclosed in an embodiment of this application;

[0046] Figure 3 This is a schematic diagram of the structure of the electronic device disclosed in the embodiments of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0049] As the background technology shows, since the core transactions in the financial industry involve complex dependencies across systems and multiple levels, the computational efficiency of traditional graph summarization algorithms is difficult to meet the processing needs as the node scale continues to expand and the complexity continues to increase.

[0050] To address the aforementioned issues, this application discloses a method, system, and related equipment for processing batch full-link views. The method intelligently compresses and optimizes the original batch full-link view to obtain multiple subgraphs. While significantly reducing the view size, it completely preserves the core topological relationships and critical paths of batch processing within the subgraphs, ensuring topological integrity. Furthermore, it adds qualified order edges (edges with high order centrality) to batch links that meet the importance assessment criteria, and accelerates the processing of the full-link view through a distributed processing architecture. Because the core topological relationships and critical paths of batch processing are completely preserved within the subgraphs, ensuring topological integrity, it improves the efficiency of locating batch operation anomalies and computational efficiency even in complex dependencies involving cross-systems and multiple levels. The specific implementation is described in detail through the following embodiments.

[0051] It should be noted that the batch end-to-end view processing method, system and related equipment provided in this application can be used in the technical field of view processing, etc. The above is only an example and does not limit the application field of the batch end-to-end view processing method, system and related equipment provided in this application.

[0052] refer to Figure 1 The image shows a batch end-to-end view processing method disclosed in an embodiment of this application. This batch end-to-end view processing method mainly includes the following steps:

[0053] S101: Obtain the original batch full-link view, and intelligently compress and optimize the original batch full-link view to obtain multiple sub-graphs; each sub-graph corresponds to a set of local links with business relationships.

[0054] In S101, batch link monitoring is performed in layers to obtain the original batch full link view. The original batch full link view is intelligently compressed and optimized by the community detection algorithm to divide it into multiple sub-graphs.

[0055] A full-link view refers to the process of tracing the complete path of a certain entity (such as a request, order, or user) from its origin to its destination in a complex system (such as a distributed service, supply chain, or user behavior path), forming a global dependency graph.

[0056] Batch link discovery is performed on the hierarchical structure to obtain the original batch full-link view. The original batch full-link view is then intelligently compressed and optimized according to the community detection algorithm to divide it into multiple subgraphs.

[0057] Each subgraph corresponds to a set of local links with inherent business relationships. Hierarchical adaptive importance assessment is performed within each subgraph. In-degree analysis is used to quantify the radiating capacity and dependency strength of nodes. A composite assessment model is constructed by combining approximate betweenness centrality (estimated based on random sampling of node participation frequency in the critical path) and batch link reachability (the scale of successors reachable by a node). Simultaneously, business attribute indicators (business priority, external reporting importance, etc.) are introduced as dynamic weighting factors, and the weights are adaptively adjusted through the correlation matrix between indicators.

[0058] The original batch full-link view is intelligently compressed and optimized according to the community detection algorithm, and the formulas for dividing it into multiple subgraphs are shown in formulas (1) to (5).

[0059] The original batch full-link view is partitioned using a community partitioning method based on module maximization. The specific calculation formula for the community partitioning based on module maximization is shown in formula (1).

[0060] (1)

[0061] Where Q is the subgraph; A is the adjacency matrix; Let i be the degree of the batch node i; Let be the degree of the batch node j; c be the community tag; and m be the total number of batch links. The upper bound for the batch link search time for each partition is... The formula for calculating the upper bound of the batch link search time for each partition is shown in formula (2).

[0062] (2)

[0063] in, The upper bound of the batch link search time for each partition; p is the number of partitions; b is the computation time for partition i; b is the network bandwidth. For communication overhead.

[0064] Calculate the in-degree and out-degree of the batch nodes. Out-degree O refers to the number of batch links from one batch node to another, and in-degree I refers to the number of batch links from other batch nodes to this node.

[0065] The approximate betweenness centrality method is adopted, and the amount of computation is reduced by calculating through sampling nodes, as shown in formula (3).

[0066] (3)

[0067] in, The approximate betweenness centrality of each node; n is the total number of batch nodes; k represents the number of sampled batch nodes; H represents the total number of batch nodes; s represents the sampled batch nodes; The number of links in the batch corresponding to each batch node in the downsampling; for The corresponding contribution level.

[0068] For each sampled batch node *s*, a breadth-first search algorithm is used to generate the shortest batch link tree. The number of batch links corresponding to each sampled batch node is recorded. and corresponding contribution The summation of its contribution values ​​across all sampling sources is multiplied by a scaling factor. The final result measures the importance of each batch node in the shortest batch link.

[0069] Perform reachability analysis and evaluation on a batch of nodes, and obtain a batch full-link view. Where V is the set of batch nodes and E is the set of batch links. A breadth-first search algorithm is used to count the number of reachable batch nodes R for each batch node.

[0070] Assess the importance of business metrics for each batch node, and quantify the importance of business metrics for each batch node to obtain B. s .

[0071] The importance of out-degree (O), in-degree (I), approximate betweenness centrality (C), reachability index (R), and business metrics (B) is considered. s Standardization is performed, and the specific calculation formula for standardization is shown in formula (4).

[0072] (4)

[0073] in, The standardized value range is [0, 1]. For each standardized batch node, determine its out-degree O, in-degree I, approximate betweenness centrality C, reachability index R, and importance of business metrics B. s ; For each original batch node, determine its out-degree O, in-degree I, approximate betweenness centrality C, reachability index R, and business indicator importance B. s ; For all original batch nodes, determine their out-degree O, in-degree I, approximate betweenness centrality C, reachability index R, and importance of business metrics B. s .

[0074] The adaptive weight ratio is calculated as shown in formula (5).

[0075] (5)

[0076] in, For adaptive weighting ratios; , , , , The adaptive weighting index system satisfies ; Let be the standardized variance of the i-th importance indicator.

[0077] Based on the modularity maximization community detection algorithm, the batch full-link view is divided into business coupled subgraphs. Within the subgraphs, topological indicators (in-degree centrality, approximate betweenness centrality, reachability index) and business semantic indicators (such as business priority, external reporting importance, etc.) are dynamically integrated. The weights are adaptively adjusted through the indicator association matrix, which solves the mismatch problem between traditional static evaluation rules and dynamic business scenarios.

[0078] S102: Perform hierarchical adaptive importance assessment within multiple subgraphs to obtain a batch of links that meet the importance assessment conditions.

[0079] Based on the hierarchical evaluation results, the subgraphs are optimized using a dual-track approach. Specifically, within each subgraph, core nodes and related links with high importance are retained, while low-weight end nodes are merged to form supernodes to reduce redundancy. For cross-subgraph connections, edges of order that meet certain conditions, i.e., edges with high betweenness centrality (such as cross-level call links), are proactively added to complete critical business paths that may be broken due to community partitioning, thereby enhancing the overall connectivity of the summary graph.

[0080] It should be noted that edges with a betweenness centrality greater than 70% are considered high betweenness edges.

[0081] The original batch full-link view is divided into multiple subgraphs according to the community detection algorithm. The batch full-link view is then subjected to hierarchical adaptive importance assessment. The importance assessment indicators can be dynamically adjusted to each other, and multiple indicators are combined to assess the importance of nodes.

[0082] In the complete batch full-link view, hierarchical adaptive importance assessment is used to retain the batch links with higher importance, while adding qualified order edges to increase the connectivity of the summary graph.

[0083] A hierarchical search strategy is adopted, which searches within communities based on the community detection algorithm and makes connections across communities. Optimization is performed based on a probabilistic pruning strategy, that is, optimization is performed based on the transition probability model. The expression of the transition probability model is shown in formula (6).

[0084] (6)

[0085] in, The importance of batch nodes; The cosine similarity of the embedding vectors between a batch of nodes; ; The transition probability between a batch of nodes; Normalization factor; For each adaptive weight ratio.

[0086] The pruning conditions are set as shown in formula (7).

[0087] (7)

[0088] Where t represents the number of transitions from the starting node to the current node; The transition probability of two adjacent nodes; This is the attenuation threshold.

[0089] The results of probabilistic pruning optimization are used to output Top-K batch links, and nodes with high betweenness centrality based on edges are added as important connections, while low-importance end nodes are merged.

[0090] S103: Add qualified order edges (such as cross-community edges) to the batch links to obtain the initial cross-system batch full link view.

[0091] The initial cross-system batch full-link view is the batch full-link view without accelerated processing.

[0092] In the complete batch full-link view, hierarchical adaptive importance assessment is used to retain the batch links with higher importance, while adding qualified order edges to increase the connectivity of the summary graph.

[0093] Within the subgraph, core nodes and related links with high importance are retained, and low-weight end nodes are merged into super nodes to reduce redundancy. Furthermore, cross-community edges with high betweenness centrality are added (e.g., a critical call link from the distributed core system providing transaction data, to the accounting system calculating the general ledger and sub-ledgers, and finally to the external reporting system using the general ledger and sub-ledgers for report processing and submission), to repair business path breaks that may result from community partitioning and ensure topology integrity.

[0094] S104: For the initial cross-system batch full-link view, the full-link view is processed and calculated using a distributed processing architecture to obtain the target batch full-link view.

[0095] Distributed processing architecture is a technical model that distributes computing tasks, data storage, and business logic across multiple nodes (servers or clusters) to work together, aiming to improve system processing power, reliability, and scalability.

[0096] A node is the smallest unit of task in a batch processing system. It is typically used to handle a specific task (such as data loading, data processing, data export, etc.).

[0097] Batch processing systems are systems that process and handle business data in batches. In the financial industry, there are both pure batch systems and systems that combine both online and batch processing. Typically, financial systems separate online and batch processing, meaning the two types of operations are not performed simultaneously.

[0098] Batch processing refers to a method of centrally and automatically processing a large number of transactions or data within a specific time window. Unlike real-time processing, batch processing is usually executed at fixed intervals.

[0099] The specific process of obtaining the target batch full-link view is shown in A1-A3.

[0100] A1: In the computing layer of the distributed processing architecture, the preset core algorithm is distributed and modified according to the parallel graph computing framework. Then, the full-link view is traversed once by the core algorithm after the distributed modification and the random sampling method to synchronously complete the bidirectional statistics of in-degree and out-degree.

[0101] The preset core algorithms include, but are not limited to, community detection and betweenness centrality evaluation.

[0102] In the computation layer, core algorithms such as community detection and betweenness centrality assessment are distributed based on the parallel graph computation framework. Asynchronous message passing reduces synchronization overhead, and random sampling is introduced to reduce the complexity of betweenness calculation. In-degree bidirectional statistics are completed synchronously in a single graph traversal to avoid redundant data exchange.

[0103] A2: By using a multi-stream concurrent pipeline of Graphics Processing Units (GPUs) built in the physical layer of a distributed processing architecture, batch processing is accelerated on the full-link view that has completed bidirectional in-degree statistics to obtain the target batch full-link view.

[0104] In the physical layer, for high-density computing tasks such as adjacency matrix operations and reachability assessment, a GPU multi-stream concurrent pipeline is constructed. Through memory data block alignment, mixed-precision computing, and asynchronous transmission mechanisms, batch processing of computing tasks at the memory level is accelerated, significantly improving iteration efficiency.

[0105] A3: Build a distributed graph database in the storage layer of the distributed processing architecture to store the target batch full-link view.

[0106] A distributed graph database is built in the storage layer. A business attribute pre-partitioning strategy is adopted to preferentially allocate closely related nodes (such as deposit statistics and accrual nodes in the accounting report system) to the same physical storage node. At runtime, dynamic load monitoring and hash rebalancing mechanisms are used to ensure data distribution balance and query efficiency.

[0107] Based on clustering, business attribute pre-partitioned storage is implemented to optimize data distribution and query efficiency; core tasks such as community detection and betweenness calculation are parallelized and in-degree data is obtained synchronously in a single traversal to reduce the number of calculation rounds; a GPU multi-stream concurrent pipeline is designed, and memory data block alignment and mixed precision calculation technology are adopted to accelerate adjacency matrix operations and reachability assessment.

[0108] To handle batch processing involving tens of thousands of nodes, this application employs a distributed processing architecture: it uses distributed graph storage technology to achieve parallel storage and querying of node relationships; it utilizes a parallel graph computing framework to decompose and distribute core steps such as community detection and betweenness calculation, thereby improving the efficiency of ultra-large-scale data processing; and it accelerates adjacency matrix operations and reachability assessment through GPU multi-stream concurrency technology, further optimizing computational performance.

[0109] S105: Display the trunk and branches of the target batch full-link view through a hierarchical layout.

[0110] In S105, the trunk structure and branch details of the target batch full-link view are displayed through a hierarchical layout engine and a tree-like folding strategy based on a force-directed algorithm.

[0111] The hierarchical layout clearly displays the trunk and branches of the batch full-link view, and provides core indicators such as compression rate and batch link retention rate. The view is evaluated based on the deviation of the indicators and manual approval.

[0112] Calculate the compression rate and batch link retention rate of the batch full-link view. The compression rate measures the simplification of the batch full-link view and is defined as the normalized value of the reduction ratio of batch links, as shown in formula (8).

[0113] (8)

[0114] Where CR is the compression ratio; The number of links in the target batch full-link view based on the graph summarization algorithm; This represents the number of links in the original batch full-link view.

[0115] Graph summarization algorithms compress large-scale graph data into smaller subgraphs or statistical representations, preserving key topological structures and reducing computational complexity.

[0116] Batch link retention rate is used to measure the retention of critical batch links. It is defined as the proportion of critical specific batch links in the original graph that exist in the batch full link view based on the graph summarization algorithm, as shown in formula (9).

[0117] (9)

[0118] Where PP represents the batch link retention rate; This refers to the set of key batch links corresponding to each system in the original batch full-link view; This represents the set of batch links from fixed source batch nodes to the merge node in a batch full-link view based on a graph summarization algorithm. The closer the compression ratio CR is to 1, the greater the compression of the batch links in the batch full-link view. The closer the compression ratio CR is to 0, the more original batch links are retained in the batch full-link view. The closer the batch link retention rate PP is to 1, the more critical batch links of the original graph are retained in the batch full-link view. The closer the batch link retention rate PP is to 0, the fewer critical batch links of the original graph are retained.

[0119] The main body and branches of the target batch full-link view are evaluated based on preset evaluation indicators. Alternatively, the batch full-link view can be evaluated based on its indicators and in conjunction with manual business indicator rules.

[0120] Specifically, the compression rate and batch link retention rate of the target batch full-link view are calculated. Based on the compression rate, batch link retention rate and indicator deviation, the trunk structure and branch details of the target batch full-link view are quantitatively evaluated.

[0121] The algorithm employs a force-oriented approach combined with a tree-like folding strategy to display the main path (such as cross-system transaction flow) and branch details (such as fault-tolerant sub-paths) in a hierarchical manner. It defines two core indicators: compression rate (the ratio of the link scale of the summary graph to the original graph) and batch link retention rate (the percentage of key business paths retained). It establishes a closed-loop mechanism of "intelligent generation - indicator verification - closed-loop correction". When the indicator deviates from the preset threshold, it triggers manual intervention to ensure the accuracy of the view and its adaptability to business needs.

[0122] This application employs a hierarchical layout engine, based on a force-oriented algorithm and overlaid with a tree-like folding strategy, to clearly display the backbone structure and branch details of batch links. It also defines two core metrics: compression rate (the ratio of the summary graph link size to the original graph link size) and batch link retention rate (the percentage of critical business paths retained in the summary graph), generating a quantitative evaluation report. When the metric deviation exceeds the preset tolerance range, a manual approval process is triggered, where operations personnel calibrate the summary graph based on business rules (e.g., manually restoring mistakenly deleted external report paths), ultimately forming a closed-loop governance system of algorithm generation, metric verification, and manual correction.

[0123] This application addresses core pain points in the financial industry, such as complex cross-system batch processing links, massive view scale, and inefficient fault location. It overcomes the error of accidental deletion of critical business paths caused by static weights in traditional algorithms by dynamically integrating topological features and business semantic indicators (such as business priority and external reporting importance) through a hierarchical adaptive importance assessment mechanism. Based on distributed architecture and GPU acceleration technology, it achieves minute-level processing of hundreds of thousands of node views. Experiments show that the efficiency is improved by tens of times compared to single-machine processing. The hierarchical layout engine clearly separates the trunk link from the branch details, and combines quantitative indicators such as compression rate and link retention rate with a manual review mechanism to form a batch full-link view construction system with intelligent generation, indicator verification, and closed-loop correction.

[0124] This application addresses the challenges of high complexity and difficulty in identifying key information in cross-system batch processing scenarios in industries such as finance and banking. Its core solution utilizes graph summarization technology to intelligently compress and optimize the original view, significantly reducing its size while fully preserving the core topological relationships and critical paths of batch processing. This improves the efficiency of locating batch processing anomalies and enhancing system governance effectiveness. The specific implementation comprises four processing steps: First, the view is subgraphed based on a community detection algorithm, and the weights of multi-dimensional indicators are dynamically adjusted through a hierarchical adaptive importance assessment mechanism to filter out key batch links. Next, the topological connectivity of the summary graph is enhanced by selectively adding qualified edges. For ultra-large-scale views, a distributed computing architecture is used to achieve parallel processing. Finally, a hierarchical layout engine clearly displays the view's main and branch structures and outputs quantitative indicators such as compression rate and link retention rate. Combined with quantitative indicator deviation analysis and a manual review mechanism, a closed-loop view evaluation system is formed.

[0125] In this embodiment, the original batch full-link view is intelligently compressed and optimized to obtain multiple subgraphs. While significantly reducing the view size, the core topological relationships and critical paths of batch processing are fully preserved within the subgraphs, ensuring topological integrity. Furthermore, qualified order edges (i.e., edges with high order centrality) are added to batch links that meet the importance assessment criteria. A distributed processing architecture is used to accelerate the processing of the full-link view. Because the core topological relationships and critical paths of batch processing are fully preserved within the subgraphs, ensuring topological integrity, even with complex dependencies involving cross-systems and multiple levels, the efficiency of locating batch operation anomalies and computational efficiency can be improved.

[0126] Based on the above embodiments Figure 1 This application discloses a method for processing batch end-to-end views, and also provides a corresponding system for processing batch end-to-end views. Figure 2 As shown, the processing system for this batch end-to-end view includes:

[0127] The acquisition unit 201 is used to acquire the original batch full-link view and intelligently compress and optimize the original batch full-link view to obtain multiple sub-graphs; wherein, each sub-graph corresponds to a set of local links with business relationships;

[0128] The first evaluation unit 202 is used to perform hierarchical adaptive importance evaluation within the multiple subgraphs to obtain a batch of links that meet the importance evaluation conditions.

[0129] Add unit 203 to add qualified order edges to the batch links to obtain the initial cross-system batch full link view;

[0130] The accelerated processing unit 204 is used to perform accelerated processing on the initial cross-system batch full-link view through a distributed processing architecture to obtain the target batch full-link view.

[0131] Display unit 205 is used to display the trunk and branches of the target batch full-link view through a hierarchical layout.

[0132] Furthermore, the acquisition unit 201 includes:

[0133] The acquisition module is used to perform batch link monitoring in a hierarchical manner to obtain the original batch full link view;

[0134] The segmentation module is used to intelligently compress and optimize the original batch full-link view through community detection algorithms to segment it into multiple subgraphs.

[0135] Furthermore, the accelerated processing unit 204 includes:

[0136] The modified traversal module is used in the computing layer of the distributed processing architecture to perform distributed modification of the preset core algorithm according to the parallel graph computing framework. Through the core algorithm after distributed modification and the random sampling method, a single graph traversal is performed on the initial cross-system batch full-link view to simultaneously complete the bidirectional statistics of in-degree and out-degree.

[0137] The batch processing acceleration module is used to accelerate the batch processing of the full-link view that has completed bidirectional in-degree statistics by building a multi-stream concurrent pipeline of the graphics processor in the physical layer of the distributed processing architecture, so as to obtain the target batch full-link view.

[0138] A storage module is built to construct a distributed graph database in the storage layer of a distributed processing architecture, storing target batch full-link views.

[0139] Furthermore, display unit 205 specifically uses a hierarchical layout engine and a tree-like folding strategy based on a force-directed algorithm to display the trunk structure and branch details of the target batch full-link view.

[0140] Furthermore, it also includes a second evaluation unit;

[0141] The second evaluation unit is specifically used to evaluate the trunk and branches of the target batch full-link view according to preset evaluation indicators;

[0142] The second assessment unit includes:

[0143] The calculation module is used to calculate the compression rate and batch link retention rate of the target batch full-link view;

[0144] The quantitative evaluation module is used to quantitatively evaluate the backbone structure and branch details of the target batch full-link view based on compression rate, batch link retention rate, and indicator deviation.

[0145] In this embodiment, the original batch full-link view is intelligently compressed and optimized to obtain multiple subgraphs. While significantly reducing the view size, the core topological relationships and critical paths of batch processing are fully preserved within the subgraphs, ensuring topological integrity. Furthermore, qualified order edges are added to batch links that meet the importance assessment criteria, and the full-link view is processed more quickly using a distributed processing architecture. Because the core topological relationships and critical paths of batch processing are fully preserved within the subgraphs, ensuring topological integrity, even with complex dependencies involving cross-systems and multiple levels, the efficiency of locating batch operation anomalies and computational efficiency can be improved.

[0146] This application embodiment also provides a storage medium, the storage medium including stored instructions, wherein, when the instructions are executed, the device where the storage medium is located is controlled to perform the processing method of the batch full-link view as described above.

[0147] This application also provides an electronic device, the structural schematic diagram of which is shown below. Figure 3 As shown, it specifically includes a memory 301 and one or more instructions 302, wherein one or more instructions 302 are stored in the memory 301 and configured to be executed by one or more processors 303 to perform the above-mentioned batch full-link view processing method.

[0148] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0149] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system-type embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0150] The steps in the methods of the various embodiments of this application can be adjusted, combined, or deleted according to actual needs.

[0151] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0152] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0153] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for processing batch full link views, the method comprising: The method comprises: obtaining an original batch full link view, and intelligently compressing and optimizing the original batch full link view to obtain a plurality of subgraphs; wherein each subgraph corresponds to a local link set having a business association relationship; performing hierarchical adaptive importance evaluation in the plurality of subgraphs to obtain a batch link meeting an importance evaluation condition; adding a number of edges meeting a condition to the batch link to obtain an initial cross-system batch full link view; performing accelerated processing operation on the initial cross-system batch full link view through a distributed processing architecture to obtain a target batch full link view; displaying a trunk and branches of the target batch full link view through hierarchical layout.

2. The method of claim 1, wherein, The obtaining of the original batch full link view and the intelligent compression and optimization of the original batch full link view to obtain a plurality of subgraphs comprises: obtaining an original batch full link view through hierarchical batch link monitoring; segmenting the original batch full link view into a plurality of subgraphs through a community detection algorithm.

3. The method of claim 1, wherein, The performing of the accelerated processing operation on the initial cross-system batch full link view through the distributed processing architecture to obtain the target batch full link view comprises: in a computing layer of the distributed processing architecture, performing distributed modification on a preset core algorithm according to a parallel graph computing framework, and performing single graph traversal on the initial cross-system batch full link view through the distributed modified core algorithm and a random sampling method to synchronously complete bidirectional statistics of out-degree and in-degree; performing batch processing acceleration on the full link view on which the bidirectional statistics of out-degree and in-degree are completed through a multi-stream concurrent pipeline of a graphics processing unit constructed in a physical layer of the distributed processing architecture to obtain the target batch full link view; constructing a distributed graph database in a storage layer of the distributed processing architecture to store the target batch full link view.

4. The method of claim 1, wherein, The displaying of the trunk and branches of the target batch full link view through hierarchical layout comprises: displaying a trunk structure and branch details of the target batch full link view through a hierarchical layout engine and a tree folding strategy based on a force-directed algorithm.

5. The method of claim 1, wherein, Further comprising: evaluating the trunk and branches of the target batch full link view according to a preset evaluation index; The evaluating of the trunk and branches of the target batch full link view according to the preset evaluation index comprises: calculating a compression rate and a batch link retention rate of the target batch full link view; quantitatively evaluating a trunk structure and branch details of the target batch full link view according to the compression rate, the batch link retention rate, and an index deviation.

6. A system for processing bulk full link views, the system comprising: The system comprises: an obtaining unit, configured to obtain an original batch full link view, and intelligently compress and optimize the original batch full link view to obtain a plurality of subgraphs; wherein each subgraph corresponds to a local link set having a business association relationship; a first evaluating unit, configured to perform hierarchical adaptive importance evaluation in the plurality of subgraphs to obtain a batch link meeting an importance evaluation condition; an adding unit, configured to add a number of edges meeting a condition to the batch link to obtain an initial cross-system batch full link view; and an obtaining unit, configured to perform accelerated processing operation on the initial cross-system batch full link view through a distributed processing architecture to obtain a target batch full link view. The acceleration processing operation unit is configured to perform acceleration processing operation on the initial cross-system batch full-link view through a distributed processing architecture to obtain a target batch full-link view. The display unit is configured to display the main trunk and branches of the target batch full-link view through hierarchical layout.

7. The system of claim 6, wherein, The acquisition unit comprises: The acquisition module is configured to acquire an original batch full-link view through hierarchical batch link monitoring. The segmentation module is configured to intelligently compress and optimize the original batch full-link view through a community detection algorithm to segment a plurality of subgraphs.

8. The system of claim 6, wherein, The acceleration processing operation unit comprises: The reconstruction traversal module is configured to perform distributed reconstruction of a preset core algorithm according to a parallel graph calculation framework in a calculation layer of the distributed processing architecture, and perform single graph traversal on the initial cross-system batch full-link view through the reconstructed core algorithm and a random sampling method to synchronously complete bidirectional statistics of out-degree and in-degree. The batch processing acceleration module is configured to perform batch processing acceleration on the full-link view with completed bidirectional statistics of out-degree and in-degree through a multi-stream concurrent pipeline of a graphic processing unit constructed in a physical layer of the distributed processing architecture to obtain the target batch full-link view. The construction storage module is configured to construct a distributed graph database in a storage layer of the distributed processing architecture and store the target batch full-link view.

9. A storage medium, characterized by The storage medium comprises stored instructions, wherein the instructions, when executed, control a device where the storage medium is located to perform the batch full-link view processing method according to any one of claims 1 to 5.

10. An electronic device, comprising: The device comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to perform the batch full-link view processing method according to any one of claims 1 to 5.

Images