A system and method for tracing a blood path based on quantum encoding and GPU acceleration

Abstract

The application discloses a kind of based on quantum encoding and GPU acceleration's blood relationship path tracking system and method, it is related to data processing technical field, the method includes: parsing original execution plan log, abstracts cross-platform operation as the probability superposition state representation including mapping, filtering and connection operator;Compression is carried out to quantum state blood relationship path set, generates the compressed data block that is made of binary code corresponding to operation sequence;Memory pooling strategy is used to manage compressed data block, and the memory pool address corresponding to compressed data block is output;In response to the blood query request received in real time, target memory pool address is positioned based on target path included in blood query request;According to the size of target compressed data block corresponding to target memory pool address, GPU resource allocation weight is calculated according to the size of target compressed data block corresponding to target memory pool address and the remaining rate of display memory;Scheduling GPU parallelly executes blood relationship path feature matching operation;Based on the blood atlas of the JSON format that encoding path subset screened out is assembled.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a bloodline path tracking system and method based on quantum encoding and GPU acceleration. BACKGROUND

[0002] In the field of data governance and traceability analysis, bloodline path tracking technology records the complete flow trajectory of data from generation to final application, providing key support for data quality control, compliance audit and abnormal troubleshooting. With the proliferation of distributed systems and multi-source heterogeneous data, the complexity of bloodline paths grows exponentially, and traditional tracking solutions based on relational databases or graph databases face problems such as low processing efficiency and high storage costs. In particular, in scenarios involving cross-platform data interaction, the dynamic changes in field mapping relationships between different systems and the real-time matching requirements of massive path data pose higher challenges to the performance and scalability of bloodline tracking.

[0003] Existing bloodline path tracking methods mainly rely on parsing algorithms such as syntax tree analysis, which has significant bottlenecks when dealing with large-scale data. On the one hand, traditional rule-based parsing algorithms struggle to effectively handle frequent changes in fields, resulting in low accuracy in field mapping identification. On the other hand, when dealing with a large number or high frequency of bloodline queries, there is a problem of low efficiency, making it difficult to meet the demand for rapid response in real-time audit and immediate troubleshooting scenarios.

[0004] To address the above problems, no effective solutions have been proposed so far. SUMMARY

[0005] Embodiments of the present application provide a bloodline path tracking system and method based on quantum encoding and GPU acceleration to solve the above technical problems.

[0006] The present application provides a bloodline path tracking system based on quantum encoding and GPU acceleration, comprising:

[0007] A log parsing module for parsing original execution plan logs and abstracting cross-platform operations into a probabilistic superposition state representation including mapping, filtering and connection operators;

[0008] A bloodline path set forming module for establishing field mapping relationships between different platforms based on the probabilistic superposition state representation and forming a quantum state bloodline path set;

[0009] A compressed data block generation module for performing Huffman encoding compression on the quantum state bloodline path set to generate a compressed data block composed of binary codes corresponding to operation sequences;

[0010] The memory pool address generation module is configured to manage the compressed data block by using a memory pooling strategy and output a memory pool address corresponding to the compressed data block.

[0011] The target memory pool address positioning module is configured to, in response to a bloodline query request received in real time, locate a target memory pool address based on a target path included in the bloodline query request.

[0012] The GPU resource allocation weight calculation module is configured to calculate a GPU resource allocation weight according to a query priority, a size of a target compressed data block corresponding to the target memory pool address, and a remaining rate of a video memory.

[0013] The path feature matching module is configured to schedule a GPU to perform a bloodline path feature matching operation in parallel, including: comparing the target path with encoded paths in the target compressed data block, and screening an encoded path subset that meets a path feature.

[0014] The bloodline graph assembly module is configured to assemble the screened encoded path subset into a bloodline graph in a JSON format.

[0015] The application provides a bloodline path tracking method based on quantum encoding and GPU acceleration, including:

[0016] The original execution plan log is parsed, and a cross-platform operation is abstracted into a probability superposition state representation including mapping, filtering, and connection operators.

[0017] Based on the probability superposition state representation, a field mapping relationship between different platforms is established, and a quantum state bloodline path set is formed.

[0018] Huffman encoding compression is performed on the quantum state bloodline path set, and a compressed data block composed of binary codes corresponding to operation sequences is generated.

[0019] The memory pooling strategy is used to manage the compressed data block, and a memory pool address corresponding to the compressed data block is output.

[0020] In response to a bloodline query request received in real time, a target memory pool address is located based on a target path included in the bloodline query request.

[0021] A GPU resource allocation weight is calculated according to a query priority, a size of a target compressed data block corresponding to the target memory pool address, and a remaining rate of a video memory.

[0022] The GPU is scheduled to perform a bloodline path feature matching operation in parallel, including: comparing the target path with encoded paths in the target compressed data block, and screening an encoded path subset that meets a path feature.

[0023] Assemble the JSON format bloodline graph based on the screened encoding path subset.

[0024] Further, after generating the JSON format bloodline graph, an asynchronous checkpoint operation is performed:

[0025] Backup the quantum state encoding library of the quantum state bloodline path set to the Iceberg columnar storage system every 5 minutes;

[0026] When a failure occurs, restore the quantum state encoding library according to the amount of data loss and the solid state disk reading rate;

[0027] Convert the restored quantum state encoding library into an ApacheArrow columnar format bloodline matrix;

[0028] In response to a received full bloodline query request, output the bloodline matrix through a zero-copy interface.

[0029] Further, ;

[0030] Wherein, is the probability superposition state representation; represents a mapping operation ground state, is a filtering operation ground state, is a connection operation ground state; is a mapping operation probability weight, with a value of 0.6 to 0.8; is a filtering operation probability weight, with a value of 0.1 to 0.3; is a connection operation probability weight, with a value of 0.05 to 0.15;

[0031] The establishment of the field mapping relationship includes: based on , , , the mapping probability of the Kafka source field to the Flink target field is calculated.

[0032] Further, the compression rate of the Huffman encoding compression is calculated by the following formula:

[0033]

[0034] Wherein, is the total number of bloodline paths in the quantum state bloodline path set; is the index of the bloodline path; is the encoding length of the th bloodline path in the compressed data block, in bits; is the encoding length of the The original length of the bloodline path in the set of quantum state bloodline paths, in bytes; For the first The frequency of the bloodline path in the original execution plan log;

[0035] The GPU resource allocation weight is determined based on the following formula: weight value=(query priority x target compressed data block size) / memory remaining rate; wherein the memory remaining rate is obtained in real time through the GPU driver interface; the bloodline path feature matching operation adopts a memory back pressure mechanism, and when the memory remaining rate is lower than the set remaining rate threshold, the scheduling of new tasks is suspended.

[0036] Further, the fault recovery time The total amount of quantum state encoding data that is not persisted when a fault occurs is calculated by the following formula:

[0037]

[0038] Wherein, The total amount of quantum state encoding data that is not persisted when a fault occurs, in GB; The solid state disk read rate, in GB / s; The system recovery inherent overhead time is set to 10ms.

[0039] Further, the bloodline path feature matching operation includes:

[0040] Analyzing the target path in the bloodline query request to generate a target feature vector; the target feature vector includes the mapping operation probability weight, the filtering operation probability weight and the connection operation probability weight of the target path;

[0041] The target compressed data block is divided into multiple equal-length sub-blocks according to the number of GPU stream processors, and each GPU stream processor is bound to a sub-block;

[0042] Each GPU stream processor sequentially reads the Huffman encoding bit stream of the bound sub-block, and when the bit pattern 1111 is identified, the weight accumulation value of the candidate path is initialized;

[0043] When the bit pattern 1100 is detected, the mapping operator is recorded and the mapping weight accumulation value of the current candidate path is increased; when the bit pattern 1101 is detected, the filtering operator is recorded and the filtering weight accumulation value of the current candidate path is increased; when the bit pattern 1110 is detected, the connection operator is recorded and the connection weight accumulation value of the current candidate path is increased;

[0044] When the bit pattern 1111 is identified again, an absolute error sum of the accumulated weight vector of the candidate path and the target feature vector is calculated; if the absolute error sum is less than a set tolerance threshold, a storage offset of the candidate path in the target compressed data block is written into the global matching list;

[0045] After all GPU stream processors are processed, the corresponding encoding path is extracted according to the storage offset in the global matching list, and is aggregated as the encoding path subset.

[0046] Further, the method further includes an incremental path processing mechanism, including:

[0047] The newly added execution plan log is parsed into an incremental bloodline path;

[0048] The incremental bloodline path is merged with the set of quantum state bloodline paths to form an updated bloodline path set;

[0049] The occurrence frequency of each path is recalculated based on the updated bloodline path set;

[0050] The Huffman coding tree is reconstructed according to the recalculated occurrence frequency of each path;

[0051] The updated bloodline path set is compressed using the reconstructed Huffman coding tree to generate an updated compressed data block;

[0052] The storage space of the original compressed data block is released and stored in the updated compressed data block, and the corresponding memory pool address mapping table is updated.

[0053] Further, the memory pooling strategy is implemented through data heat level storage, including:

[0054] For each compressed data block, the total number of occurrences of the bloodline path included in the original execution plan log is sorted in descending order; the top 20% is defined as a high-heat block, the last 30% is defined as a low-heat block, and the middle 50% is defined as a medium-heat block;

[0055] A three-level storage architecture is established, including: a high-heat storage layer: residing in GPU memory, storing the high-heat block; a medium-heat storage layer: residing in host memory, storing the medium-heat block; and a low-heat storage layer: residing in a solid state disk, storing the low-heat block;

[0056] When the bloodline query request is received, if the target compressed data block corresponding to the target memory pool address is not in the high-heat storage layer, the current query thread is paused, the target compressed data block is copied from the current storage layer to the GPU memory, the memory pool address mapping table is updated, and the query thread is restarted;

[0057] Real-time monitoring of GPU memory usage; when the GPU memory usage exceeds the set usage threshold, select the N compressed data blocks with the earliest access time and the least total occurrence from the high heat storage layer as the migration object, N is dynamically determined according to the memory overrun ratio; perform hierarchical degradation operation on each migration object; update the memory pool address mapping table synchronously.

[0058] Further, the element value of the blood relationship matrix represents the dependence strength between the Kafka source field and the Flink target field , and the calculation method is:

[0059]

[0060] Wherein, is the set of all blood path connections between the connection fields and . is the total number of paths in the blood path set; is an element in the blood path set; is the operation type sequence included in the path . is the length of the sequence . is the operation type in . is the composite weight function of the operation , defined as:

[0061]

[0062] is the weight factor of the mapping operation type in the path; is the weight factor of the filtering operation type in the path; is the weight factor of the connection operation type in the path; is the mapping operation probability weight, taking a value of 0.6 to 0.8; is the filtering operation probability weight, taking a value of 0.1 to 0.3; is the connection operation probability weight, taking a value of 0.05 to 0.15.

[0063] Based on the embodiments provided in the present application, the consanguinity path tracking method combined with quantum encoding and GPU acceleration has significant advantages compared to traditional solutions. By abstracting cross-platform operations into a probabilistic superposition state representation and establishing a field mapping relationship based on quantum states, the problem of consanguinity chain breakage caused by differences in operation semantics between different systems is effectively solved, ensuring the integrity and traceability of field-level consanguinity relationships across distributed systems. At the same time, Huffman encoding compression is performed on the quantum state consanguinity path set, and the memory pooling strategy is used to manage compressed data blocks, significantly reducing the storage overhead of consanguinity path data, improving memory utilization efficiency, and supporting fast positioning and access to target path data.

[0064] In addition, GPU resources are dynamically allocated based on the remaining memory rate, and the consanguinity path feature matching operation is performed in parallel, fully utilizing the parallel computing capability of the GPU, effectively improving the comparison efficiency of massive consanguinity paths, and meeting the high-frequency and real-time consanguinity query requirements. Finally, the filtered consanguinity path subset is assembled into a JSON format consanguinity graph, realizing the standardized representation of consanguinity relationships, facilitating seamless integration with downstream analysis tools and visualization systems, and improving the overall performance of data governance. BRIEF DESCRIPTION OF DRAWINGS

[0065] The drawings described herein are used to provide further understanding of the embodiments of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0066] Figure 1 is a structure diagram of an optional consanguinity path tracking system based on quantum encoding and GPU acceleration according to an embodiment of the present application;

[0067] Figure 2 is a flowchart of an optional consanguinity path tracking method based on quantum encoding and GPU acceleration according to an embodiment of the present application;

[0068] Figure 3 is a flowchart of another optional consanguinity path tracking method based on quantum encoding and GPU acceleration according to an embodiment of the present application.

[0069] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0071] Alternatively, as Figure 1 As shown, the present application provides a lineage path tracing system based on quantum coding and GPU acceleration, including:

[0072] The log parsing module 101 is used to parse the original execution plan log and abstract the cross-platform operation into a probabilistic superposition state representation including mapping, filtering and connection operators;

[0073] The lineage path set forming module 102 is used to establish a field mapping relationship between different platforms based on the probabilistic superposition state representation to form a quantum state lineage path set;

[0074] The compressed data block generation module 103 is used to perform Huffman coding compression on the quantum state lineage path set to generate a compressed data block composed of binary codes corresponding to the operation sequence;

[0075] A memory pool address generation module 104 is configured to manage compressed data blocks using a memory pooling strategy and output a memory pool address corresponding to the compressed data block;

[0076] The target memory pool address locating module 105 is configured to respond to a lineage query request received in real time and locate the target memory pool address based on a target path included in the lineage query request;

[0077] The GPU resource allocation weight calculation module 106 is used to calculate the GPU resource allocation weight according to the query priority, the size of the target compressed data block corresponding to the target memory pool address, and the remaining rate of the video memory;

[0078] The path feature matching module 107 is used to schedule the GPU to perform lineage path feature matching operations in parallel, including: comparing the target path with the encoding path in the target compressed data block, and screening the subset of encoding paths that meet the path features;

[0079] The bloodline map assembly module 108 is used to assemble a bloodline map in JSON format based on the filtered encoding path subset.

[0080] Alternatively, as Figure 2 As shown, the present application provides a lineage path tracing method based on quantum coding and GPU acceleration, including:

[0081] S201, parsing the original execution plan log, abstracting the cross-platform operation into a probabilistic superposition state representation including mapping, filtering and connection operators;

[0082] Among them, the original execution plan log, that is, the ETL execution plan log, the Chinese of ETL execution plan log is "extraction-transformation-loading execution plan", which is one of the core concepts in the field of data processing. The original execution plan log includes Spark, Flink, Iceberg log.

[0083] Specifically, Extract (extraction): extract the required data from the original data source (such as database, file, sensor, log, etc.); Transform (transformation): process the extracted data (such as cleaning error values, format conversion, calculation, filtering, merging, etc.) to make the data meet the requirements of the target system; Load (loading): store the final data after transformation into the target system (such as data warehouse, database, big data platform, etc.).

[0084] And the ETL execution plan log is the specific arrangement and detailed planning of the three steps.

[0085] Among them, the mapping operator is used to perform field conversion (such as desensitizing user ID to hash value); the filter operator is used to filter data according to conditions (such as retaining records with state "valid"); and the join operator is used to associate multi-source data (such as connecting order table and user table by ID).

[0086] In S201, the field conversion behavior in cross-platform operation (such as Kafka→Flink) is decomposed into three types of operation base states: mapping (Map), filtering (Filter), and joining (Join), and their occurrence probabilities are quantified by probability weights (α, β, γ). Traditional rule matching cannot adapt to dynamically changing field mapping relationships (such as user ID being mapped to hash value or associated order in different platforms). The probability superposition state representation converts discrete operations into continuous probability space, solving the problem of blood relationship break caused by cross-platform semantic difference. Through the concept of "superposition state" in quantum mechanics to model operation uncertainty, the blood relationship has the ability of probabilistic description, providing a mathematical basis for subsequent path matching.

[0087] S202, based on the probability superposition state representation, the field mapping relationship between different platforms is established, and a set of quantum state blood relationship paths is formed;

[0088] In one embodiment, the process of establishing the field mapping relationship includes:

[0089] Parse operation log: read cross-platform execution plan log and identify key operation records. For example, from the conversion log from Kafka to Flink, it is found that a user ID field is processed 1000 times; 720 times are converted to user hash value (mapping operation); 210 times are used for state filtering (filtering operation); and 70 times are associated with order ID (joining operation)

[0090] Calculate the probability weight: count the distribution proportion of the three types of operations: mapping operation accounts for 72%; filtering operation accounts for 21%; connection operation accounts for 7%; form operation probability superposition state representation: mapping operation weight 0.72, filtering operation weight 0.21, connection operation weight 0.07.

[0091] Build quantum mapping: establish the probability association relationship between source field and target field: when the source system user ID field changes, the target system user hash field has a 72% probability of being affected, the status field has a 21% probability of being affected, and the order ID field has a 7% probability of being affected. This association reflects the quantum entanglement feature: single point change triggers multi-target probability response.

[0092] The formation of the blood path set includes:

[0093] Combine continuous operations into complete data links, for example: path 1: user ID→user hash→user dimension table; path 2: user ID→status filter→active user table; path 3: user ID→order association→order table

[0094] Each path is marked with a probability weight: path 1 contains two operations: user ID conversion (weight 0.72) and dimension storage (weight 0.33), with an overall reliability of about 24% of their product; form probability entanglement between paths: when path 1 fails, path 2 has a 21% probability of failing simultaneously;

[0095] When a new field conversion is added (such as user geographic location), a new mapping relationship is automatically generated (user ID→geographic location, weight 0.05); the original mapping weight is dynamically adjusted (user hash weight from 0.72 to 0.68); a new blood path is added: user ID→geographic location→region analysis table.

[0096] In S202, field-level dependency relationships are established based on probability weights, and continuous operations are combined into complete paths (such as user ID→mapping→filter→target table). Single operation cannot reflect end-to-end blood link, while probability entanglement between paths (such as chain reaction of path failure) can improve the fault tolerance of blood analysis. Introduce quantum entanglement feature, when source field changes, automatically calculate the association probability of multiple target fields, to realize dynamic probability deduction of cross-system blood.

[0097] S203, perform Huffman coding compression on the set of quantum state blood paths, to generate a compressed data block composed of binary codes corresponding to operation sequences;

[0098] Among them, the operation sequence is the atomic component unit of the blood path (such as a single Map operation); the blood path is a complete data stream composed of operation sequences (such as [source system→Map→Join→target system]).

[0099] The compression processing includes: assigning a short binary code to a high-frequency bloodline path (such as [Map→Filter] appearing 1000 times); and assigning a long code to a low-frequency path (such as [Join→Map→Join] appearing 1 time).

[0100] In S203, the operation sequence is converted into a binary bit stream according to the path frequency of occurrence (high-frequency paths are assigned short codes such as 1100, and low-frequency paths are assigned long codes). The original bloodline path set is large (such as a million paths), and direct storage occupies too much memory. The Huffman coding compresses the data volume by using the operation symbol repetition mode (such as the frequent appearance of the filtering operation). The coding table is designed in combination with the probability distribution characteristics of the quantum state path, the high-frequency operation symbol (such as mapping) is fixed as a 4-bit short code (1100), the analysis complexity is reduced, and the GPU can identify the operation type in a single cycle.

[0101] In S204, a memory pool strategy is used to manage the compressed data blocks, and a memory pool address corresponding to the compressed data blocks is output;

[0102] In S204, the memory pool is used to uniformly manage the compressed blocks, and the memory pool address is output for fast addressing. The bloodline data stored in a scattered manner is difficult to efficiently locate. The memory pool strategy realizes centralized address mapping and accelerates the access to the target data block. The physical storage location is decoupled through the address abstraction layer, laying a foundation for subsequent fast positioning of the target path (S205).

[0103] In S205, in response to a bloodline query request received in real time, a target memory pool address is located based on a target path included in the bloodline query request;

[0104] In S205, the memory pool address of the corresponding compressed block is located according to the target path in the bloodline query request. In combination with the memory pool address and the path characteristics, accurate targeted positioning is realized, and the low-efficiency problem of full-scan is avoided.

[0105] In S206, a GPU resource allocation weight is calculated according to a query priority, a size of a target compressed data block corresponding to the target memory pool address, and a remaining rate of the display memory;

[0106] In S206, the GPU resources are dynamically allocated according to the query priority, the size of the target compressed data block, and the remaining rate of the display memory. High-concurrency queries need to avoid contention for GPU resources. The display memory remaining rate controls the task scheduling in real time. The dynamic weight mechanism with display memory awareness is introduced to preferentially guarantee critical queries and ensure instant response in scenarios with high real-time requirements (such as fault troubleshooting).

[0107] In S207, the GPU is scheduled to perform bloodline path feature matching operations, including: comparing the target path with the encoded paths in the target compressed data block, and screening an encoded path subset that meets the path characteristics.

[0108] In S207, the GPU is scheduled to compare the target path with the encoded paths in the compressed block in parallel, and a subset of paths meeting the characteristics is screened (such as identifying the bit pattern 1111 as a path boundary). The CPU serially matches the massive paths, which is inefficient, and the GPU stream processor analyzes the bit stream in parallel (such as 1100 triggering the mapping weight accumulation). Single-cycle operation recognition is achieved through hard-coded logic (such as 4-bit pattern detection), and a tolerance threshold is used to balance the matching accuracy and flexibility.

[0109] In S208, the encoded path subset is assembled into a JSON format of the blood relationship graph based on the screened subset.

[0110] In S208, the matched encoded path subset is converted into a standard JSON format. The JSON format supports seamless integration of downstream tools (such as visualization systems). The standardization of blood relationship results is achieved, and the cross-platform compatibility is improved.

[0111] Based on the embodiments provided in the present application, the complex cross-platform blood relationship is abstracted and encoded into a quantum state to solve the broken chain problem in traditional methods; at the same time, the parallel computing capability of the GPU is used to accelerate the matching and query process of the blood path, breaking through the performance bottleneck of the CPU. This combination aims to significantly improve the integrity, speed and efficiency of cross-system blood tracking.

[0112] Further, as shown in Figure 3 After generating the JSON format of the blood relationship graph, an asynchronous checkpoint operation is performed:

[0113] In S301, the quantum state encoding library of the quantum state blood path set is backed up to the Iceberg columnar storage system every 5 minutes;

[0114] The Iceberg columnar is a storage strategy of the data lake table format, which focuses on managing large-scale offline data using columnar storage, and solves the consistency and manageability problems of the data lake;

[0115] In S301, based on the transactional and version control capabilities of Iceberg, the consistency and traceability of the snapshot data are ensured, and data damage during the backup process is avoided. The columnar storage is suitable for the batch reading characteristics of the blood path, and provides efficient data sources for fault recovery (S302).

[0116] In S302, when a fault occurs, the quantum state encoding library is recovered according to the amount of data loss and the solid state disk reading rate;

[0117] In S302, the encoding library is recovered according to the amount of data loss and the solid state disk reading rate. The traditional database recovery takes too long. This step uses the high-speed I / O of the solid state disk to speed up the recovery process.

[0118] S303, convert the restored quantum state encoding library into an ApacheArrow columnar format bloodline matrix;

[0119] Wherein, ApacheArrow columnar format is a memory level data exchange standard, focusing on unifying the memory data formats of different tools and reducing the loss of cross-system data transmission.

[0120] In S303, the restored encoding library is converted into an ApacheArrow columnar format matrix. ApacheArrow memory format optimizes the efficiency of large-scale matrix query. Through columnar memory layout, full bloodline analysis (such as field dependency strength calculation) is accelerated. In S304, in response to the received full bloodline query request, the bloodline matrix is output through a zero-copy interface.

[0121] In S304, traditional serialization consumes CPU resources, and in this step, cross-system shared memory data eliminates serialization overhead and directly outputs matrix data to the query end through zero-copy technology. Avoid repeated copying of data in memory, compress the full query response time to sub-second level, and meet the efficiency requirements of batch audit, historical path tracing and other scenarios. Interface standardization design supports direct connection with big data analysis framework (such as Spark), and expands the analysis dimension of bloodline data (such as path clustering, abnormal pattern mining). Further, ;

[0122] Wherein, is a probabilistic superposition state representation; represents a mapping operation ground state, is a filtering operation ground state, is a connection operation ground state; is a mapping operation probability weight, taking a value of 0.6 to 0.8; is a filtering operation probability weight, taking a value of 0.1 to 0.3; is a connection operation probability weight, taking a value of 0.05 to 0.15;

[0123] The establishment of the field mapping relationship includes: based on , , , the mapping probability of the Kafka source field to the Flink target field is calculated.

[0124] Based on the embodiments provided in the present application, cross-platform operations are abstracted into probabilistic representation through quantum superposition state formula, and the field mapping probability from Kafka to Flink is calculated based on alpha, beta and gamma. The traditional scheme adopts deterministic rule matching field relationship, which is difficult to cope with the dynamic changes of cross-platform operation semantics (such as the same field may be mapped, filtered or connected in different systems. The present scheme quantifies the operation possibility through probability weight (alpha / beta / gamma), so that the field mapping relationship is upgraded from discrete judgment to dynamic description in continuous probability space. This design can effectively capture the uncertainty of cross-system operations (such as Kafka source field may be mapped to Flink target field with 70% probability and 30% probability for filtering), avoid the blood relationship break caused by the failure of a single rule, and ensure the completeness and traceability of the field-level blood relationship in a distributed scenario. At the same time, the probability weight provides a mathematical basis for subsequent path matching, so that the matching process can be compatible with the subtle differences of operation sequence.

[0125] Further, the compression rate of Huffman coding compression is calculated by the following formula:

[0126]

[0127] wherein, is the total number of blood path in the quantum state blood path set; is the index of the blood path; is the encoding length of the first blood path in the compressed data block, in bits; is the original length of the first blood path in the quantum state blood path set, in bytes; is the frequency of the first blood path in the original execution plan log;

[0128] The GPU resource allocation weight is determined based on the following formula: weight value=(query priority x target compressed data block size) / memory remaining rate; wherein the memory remaining rate is obtained in real time through the GPU driver interface; the blood path feature matching operation adopts a memory back pressure mechanism, and when the memory remaining rate is lower than the set remaining rate threshold, the new task scheduling is suspended.

[0129] wherein, the set remaining rate threshold refers to the preset memory remaining proportion critical value to avoid the influence of insufficient GPU memory on the blood path feature matching operation. When the memory remaining rate (the proportion of the current remaining memory to the total memory) obtained in real time through the GPU driver interface is lower than the threshold, the memory back pressure mechanism is triggered, and the new task scheduling is suspended to ensure that the blood path comparison task (such as the matching of the target path and the encoded path in the compressed data block) being executed is not disturbed by insufficient memory.

[0130] Exemplarily, in the real-time audit scene of the financial system, high-frequency concurrent bloodline query requests (such as 100+ times per second) are required, and the compressed data blocks (including quantum state encoding paths) involved in each query need to be quickly loaded into the GPU memory for matching. At this time, the "set remaining rate threshold" can be set to 30%: when the GPU memory remaining rate drops below 30%, the reception of new query task scheduling is suspended. This is because under high concurrency, the continuous loading of new tasks will quickly consume the memory, and the 30% threshold can reserve enough memory for the current task to complete the encoding path comparison (such as filtering the encoding path subset that meets the characteristics), avoiding task interruption or matching errors due to memory overflow.

[0131] Exemplarily, in the night batch processing of historical bloodline paths, the query concurrency is low, but the volume of compressed data blocks involved in a single task is large (such as containing million-level quantum state path encoding). At this time, the "set remaining rate threshold" can be set to 15%: due to the low concurrency, the frequency of new task scheduling is low, and the 15% threshold can not only ensure that the current batch task (such as Huffman encoding bit stream analysis of historical paths) has enough memory to complete the matching, but also make full use of GPU memory space, reducing the idle resources caused by premature suspension of new tasks.

[0132] Based on the embodiments provided in the present application, the calculation method of Huffman encoding compression rate (considering path length, frequency of occurrence) and the determination formula of GPU resource allocation weight (combining query priority, data block size, memory remaining rate) are clarified, and a memory backpressure mechanism is introduced. The traditional Huffman encoding compression rate calculation does not consider the influence of path occurrence frequency, which may lead to insufficient compression efficiency of high-frequency paths. The present scheme weights high-frequency paths (such as repeated mapping-filtering sequences) through freq (i) to obtain a more optimal compression ratio, significantly reducing storage overhead. In terms of GPU resource allocation, the traditional static allocation method is difficult to adapt to dynamic query load, while the present scheme dynamically adjusts the weight based on the real-time memory remaining rate, ensuring that high-priority tasks (such as real-time audit) and large-volume data blocks obtain computing resources first, improving resource utilization. In addition, the memory backpressure mechanism suspends new task scheduling when the memory is insufficient, avoiding matching task crashes due to memory overflow, balancing parallel computing efficiency and system stability, and being particularly suitable for high-frequency and large-flow bloodline query scenes.

[0133] Further, the fault recovery time is calculated by the following formula:

[0134]

[0135] wherein, is the total amount of quantum state encoding data that is not persisted when the fault occurs, in GB; For the read rate of a solid state drive (SSD), the unit is GB / s; For the inherent overhead time of system recovery, 10 ms is set.

[0136] Based on the embodiments provided in the present application, the traditional fault recovery lacks a clear time quantization model, and it is difficult to guarantee the availability of the key scene. The present scheme decomposes the recovery time into data reading time consumption and system inherent overhead, combines the high-speed reading characteristics of the SSD (significantly shortens the data reading time compared with the mechanical hard disk) and the fixed delay (10 ms), so that the recovery process is predictable and controllable. In the scene of finance, government affairs and other systems with extremely high requirements for system availability, this quantitative design can help the operation and maintenance personnel to evaluate the fault impact range in advance (such as data loss of 10 GB, the recovery time can be estimated based on the SSD rate), and actively reduce the recovery time consumption by optimizing the SSD performance or reducing the amount of non-persistent data (such as shortening the snapshot interval), to ensure the rapid recovery of the bloodline tracking service and reduce the business interruption loss.

[0137] Further, the bloodline path feature matching operation comprises:

[0138] Analyzing the target path in the bloodline query request to generate a target feature vector; the target feature vector comprises a mapping operation probability weight, a filtering operation probability weight and a connection operation probability weight of the target path;

[0139] The target compressed data block is divided into a plurality of equal-length sub-blocks according to the number of GPU stream processors, and each GPU stream processor is bound to a sub-block;

[0140] Each GPU stream processor sequentially reads the Huffman encoding bit stream of the bound sub-block, and initializes the weight accumulation value of the candidate path when the bit pattern 1111 is identified;

[0141] When the bit pattern 1100 is detected, the mapping operator is recorded and the mapping weight accumulation value of the current candidate path is increased; when the bit pattern 1101 is detected, the filtering operator is recorded and the filtering weight accumulation value of the current candidate path is increased; when the bit pattern 1110 is detected, the connection operator is recorded and the connection weight accumulation value of the current candidate path is increased;

[0142] When the bit pattern 1111 is identified again, the absolute error sum of the accumulated weight vector of the candidate path and the target feature vector is calculated; if the absolute error sum is less than a set tolerance threshold, the storage offset of the candidate path in the target compressed data block is written into a global matching list;

[0143] Wherein, the accumulated weight vector = [mapping weight accumulation value, filtering weight accumulation value, connection weight accumulation value];

[0144] The tolerance threshold is set in the blood path feature matching process. The maximum absolute error and critical value allowed between the target feature vector and the candidate path cumulative weight vector are pre-set. When the absolute error sum of the two is less than the threshold, it is considered that the operation features (mapping, filtering, connection probability weight distribution) of the candidate path are consistent with the target path, and will be included in the encoding path subset for subsequent assembly of the JSON blood graph. Its core role is to balance the accuracy and flexibility of path matching, and adapt to the strictness requirements of blood path feature matching in different scenarios.

[0145] Exemplarily, in the bank data compliance audit, the blood path of a certain transaction data needs to be accurately tracked (such as the operation sequence of “Kafka source field→Flink filtering→Snowflake connection”), and the weights of the target feature vector are α=0.7 (mapping), β=0.2 (filtering), and γ=0.1 (connection). At this time, the “set tolerance threshold” can be set to 0.1:

[0146] When the cumulative weight vector of the candidate path (such as α=0.68, β=0.21, and γ=0.11) and the absolute error sum of the target vector (|0.68-0.7|+|0.21-0.2|+|0.11-0.1|=0.01+0.01+0.01=0.03) is less than 0.1, it is determined to be matched. This threshold ensures that only the candidate paths with operation weight distribution close to the target path are selected, meeting the accuracy requirements of compliance audit on the path.

[0147] Exemplarily, in the data exploration of the e-commerce platform, various blood paths related to “user behavior data→order data” need to be quickly located (may include different proportions of mapping and filtering operations), and the weights of the target feature vector are α=0.65, β=0.25, and γ=0.1. At this time, the “set tolerance threshold” can be set to 0.3:

[0148] When the cumulative weight vector of the candidate path (such as α=0.5, β=0.35, and γ=0.15) and the absolute error sum of the target vector (|0.5-0.65|+|0.35-0.25|+|0.15-0.1|=0.15+0.1+0.05=0.3) is equal to 0.3, it is determined to be matched. This threshold allows the operation weight to have greater fluctuations, and can filter out more related blood paths, meeting the needs of quickly discovering potential associations in data exploration.

[0149] In this embodiment, the Huffman coding table design rules include:

[0150] Operator code assignment rule: mapping operator is assigned with 4-bit fixed code 1100, which corresponds to the mapping operation base state probability weight defined in right 3; filter operator is assigned with 4-bit fixed code 1101, which corresponds to the filter operation base state probability weight defined in right 3; connection operator is assigned with 4-bit fixed code 1110, which corresponds to the connection operation base state probability weight defined in right 3.

[0151] Path boundary control code design: path start flag is assigned with 4-bit fixed code 1111, and the physical meaning is to initialize a new blood path record; path end flag multiplexes 4-bit fixed code 1111, and the function is distinguished by the context position: when located in front of the operator sequence, it indicates the start of the path; when located behind the operator sequence, it indicates the end of the path.

[0152] Encoding space isolation mechanism: the Hamming distance between all operator codes (1100 / 1101 / 1110) and control codes (1111) is not less than 2; the operator code is prohibited to contain a 3-bit mode starting with two consecutive 1s to prevent confusion with the control code.

[0153] Weight value embedding method: when the operator code is detected, the fixed weight value is directly associated: 1100 triggers the mapping operation weight α accumulation; 1101 triggers the filter operation weight β accumulation; 1110 triggers the connection operation weight γ accumulation. The weight value is pre-set in the stream processor register in the form of a constant and does not participate in the encoding transmission;

[0154] Encoding and decoding consistency guarantee: the encoding is generated in the compression stage according to the same rule; the bit pattern is detected in real time through the hard-coded logic gate in the matching stage;

[0155] In some embodiments of the present application, the binary code generated by the Huffman coding compression contains two types of encoding units:

[0156] Operator code unit: fixed 4-bit length, in which the mapping operator code is 1100, the filter operator code is 1101, and the connection operator code is 1110;

[0157] Path boundary unit: fixed 4-bit length, encoded as 1111, which indicates the initialization of a new path when it appears at the beginning of the path, and triggers the path matching judgment when it appears at the end of the path;

[0158] The operator code unit and the path boundary unit are arranged in the order of the operation sequence of the blood path to form a complete compressed data block.

[0159] By the above encoding rule, the false detection rate can be reduced, specifically, the minimum Hamming distance between the operator code (1100 / 1101 / 1110) and the path flag (1111) is 2, ensuring that the GPU stream processor can be recognized in a single period; improve decoding efficiency, specifically, fixed 4-bit length eliminates the bit operation overhead of traditional Huffman variable length code; maintain weight consistency, specifically, the operator code directly binds the fixed weight constant, avoiding numerical bias introduced in the encoding and decoding process.

[0160] After all GPU stream processors are processed, the corresponding encoding path is extracted according to the storage offset in the global matching list, and is aggregated into an encoding path subset.

[0161] Based on the embodiments provided in the present application, the traditional CPU serial matching is difficult to cope with the real-time comparison demand of massive paths. The present scheme breaks through the bottleneck through the following design: first, the target path is converted into a feature vector containing operation weight, so that the matching is upgraded from string comparison to vector space calculation, improving the semantic matching accuracy; second, the number of GPU stream processors is divided into fragments for processing, realizing parallel analysis and fully utilizing the parallel computing capability of GPU; third, the operator is quickly identified through the fixed bit mode (such as 1100 corresponding to the mapping operation), reducing the decoding time; fourth, through the comparison of absolute error and tolerance threshold, the slight difference (such as weight fluctuation) of the path is compatible while ensuring the matching accuracy. These designs enable GPU to efficiently process the parallel matching of millions of paths, especially suitable for scenarios with high real-time requirements (such as instant fault diagnosis), while balancing the flexibility and accuracy of matching.

[0162] Further, the method further comprises an incremental path processing mechanism, comprising:

[0163] The newly added execution plan log is parsed into an incremental bloodline path;

[0164] The incremental bloodline path is merged with the set of quantum state bloodline paths to form an updated bloodline path set;

[0165] The occurrence frequency of each path is recalculated based on the updated bloodline path set;

[0166] The Huffman encoding tree is rebuilt according to the recalculated occurrence frequency of each path;

[0167] The updated bloodline path set is compressed using the rebuilt Huffman encoding tree to generate an updated compressed data block;

[0168] The storage space of the original compressed data block is released and stored in the updated compressed data block, and the corresponding memory pool address mapping table is updated.

[0169] Based on the embodiments provided in the present application, the traditional full-amount reprocessing mechanism needs to repeatedly encode all data when adding new paths, which is inefficient. The present scheme uses an incremental update strategy, only processes the merging of newly added paths, recalculates the frequency of occurrence and partially reconstructs the Huffman tree, avoiding the waste of resources of full-amount encoding. In real-time data flow scenarios (such as e-commerce platforms generating a large number of new paths per second), this design can ensure that the system quickly adapts to path changes, maintains the compression efficiency of Huffman encoding (high-frequency new paths automatically obtain short codes), and at the same time, by releasing the storage space of the original data block and updating the address mapping, it avoids storage redundancy and improves the dynamic scalability of the system, suitable for business scenarios that continuously generate new bloodline paths.

[0170] Further, the memory pooling strategy is implemented through data heat level storage, including:

[0171] For each compressed data block, sort the total number of occurrences of the bloodline paths it includes in the original execution plan log in descending order; define the top 20% as high-heat blocks, the last 30% as low-heat blocks, and the middle 50% as medium-heat blocks.

[0172] A three-level storage architecture is established, including: a high-heat storage layer: residing in GPU memory, storing high-heat blocks; a medium-heat storage layer: residing in host memory, storing medium-heat blocks; a low-heat storage layer: residing in solid state disk, storing low-heat blocks.

[0173] When receiving a bloodline query request, if the target compressed data block corresponding to the target memory pool address is not in the high-heat storage layer, pause the current query thread, copy the target compressed data block from the current storage layer to the GPU memory, update the memory pool address mapping table and restart the query thread.

[0174] Real-time monitoring of GPU memory usage; when the GPU memory usage exceeds the set usage threshold, select N compressed data blocks with the earliest access time and the least total number of occurrences from the high-heat storage layer as migration objects, N is dynamically determined according to the memory over-limit ratio; perform hierarchical degradation operation on each migration object; synchronize the memory pool address mapping table; N is a positive integer. In some embodiments, the hierarchical degradation operation includes: for data blocks ranked in the top 5% of total occurrences, migrate to the medium-heat storage layer; for other data blocks, migrate to the low-heat storage layer.

[0175] Based on the embodiments provided in the present application, three-level storage architecture is realized based on data heat (high heat resident GPU memory, medium heat resident host memory, and low heat resident SSD), and dynamic migration is supported. Traditional memory management does not distinguish data heat, resulting in high-frequency access data and low-frequency data competing for limited GPU memory resources, reducing access efficiency. The present scheme divides the heat level according to the number of path occurrences, and directly stores the high heat block with high-frequency access in the GPU memory (with the lowest access delay), stores the medium-frequency data in the host memory, and stores the low-frequency data in the SSD, thereby realizing layered optimization of storage resources. When the query involves low-frequency data, the data is dynamically migrated to the GPU memory and the address mapping is updated to ensure fast access of high-frequency data; when the GPU memory is insufficient, the data block with the earliest access and the lowest frequency is downgraded to release resources. This dynamic grading mechanism balances the storage cost and access efficiency, and is especially suitable for scenarios where the access frequency of blood relationship paths differs greatly (such as some core paths are frequently queried, while historical path access is very low), thereby improving the resource utilization of the overall system.

[0176] Further, the element value of the blood relationship matrix is represents the dependence strength between the Kafka source field and the Flink target field, and the calculation method is:

[0177]

[0178] wherein, is the set of all blood relationship paths connecting the fields and is the total number of paths in the set of blood relationship paths; is an element in the set of blood relationship paths; is the operation type sequence included in the path is the length of the sequence , i.e., the number of operation steps; is the operation type in is the composite weight function of the operation , and is defined as:

[0179]

[0180] is the weight factor of the mapping operation type in the path; is the weight factor of the filtering operation type in the path; is the weight factor of the connection operation type in the path; is the mapping operation probability weight, and takes a value of 0.6 to 0.8; is the filtering operation probability weight, and takes a value of 0.1 to 0.3;​​​​​ is the connection operation probability weight, ranging from 0.05 to 0.15.

[0181] Based on the embodiments provided in this application, traditional blood relationships are mostly represented by linked lists or graph structures, which makes it difficult to quantify the degree of dependence between fields. The calculation quantifies the dependency between the Kafka source field and the Flink target field into a specific value, which is a composite weight of all operations in the path ( Combine the operation probability weights α / β / γ with the type weight factor) and take the average of multiple paths to ensure the statistical significance of the results. This quantization matrix can not only intuitively reflect the dependency strength between fields (such as =0.8 indicates strong dependency), and also supports rapid output to downstream analysis tools (such as Apache Arrow format-compatible data visualization platforms) through a zero-copy interface, providing a quantitative basis for data governance (such as identifying strongly dependent fields to focus on monitoring data quality), and improving the depth and operability of lineage analysis.

[0182] It should be noted that in this application, the embodiments implemented on the lineage path tracing system side based on quantum coding and GPU acceleration can be referenced with the embodiments implemented on the lineage path tracing method side based on quantum coding and GPU acceleration, and this application will not go into details one by one.

[0183] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A system for tracing a kinship path based on quantum encoding and GPU acceleration, characterized in that, Comprise: A log analysis module for parsing the original execution plan log, abstracting cross-platform operations into a probabilistic superposition state representation including mapping, filtering and connection operators; A blood relationship path set forming module for establishing field mapping relationships between different platforms based on the probabilistic superposition state representation, and forming a quantum state blood relationship path set; A compressed data block generation module for performing Huffman coding compression on the quantum state blood relationship path set to generate a compressed data block composed of binary codes corresponding to operation sequences; A memory pool address generation module for managing the compressed data block using a memory pooling strategy and outputting a memory pool address corresponding to the compressed data block; A target memory pool address positioning module for positioning a target memory pool address based on a target path included in a blood relationship query request received in real time in response to the blood relationship query request; A GPU resource allocation weight calculation module for calculating a GPU resource allocation weight according to a query priority, a size of a target compressed data block corresponding to the target memory pool address, and a video memory remaining rate; A path feature matching module for scheduling GPU to perform blood relationship path feature matching operations in parallel, including: comparing the target path with encoded paths in the target compressed data block, and screening an encoded path subset that meets path features; An ancestry atlas assembly module for assembling a JSON format ancestry atlas based on the screened encoded path subset.

2. A method for tracing a kinship path based on quantum encoding and GPU acceleration, characterized in that, Comprise: Parse the original execution plan log, and abstract cross-platform operations into a probabilistic superposition state representation including mapping, filtering and connection operators; Establish field mapping relationships between different platforms based on the probabilistic superposition state representation, and form a quantum state blood relationship path set; Perform Huffman coding compression on the quantum state blood relationship path set to generate a compressed data block composed of binary codes corresponding to operation sequences; Manage the compressed data block using a memory pooling strategy, and output a memory pool address corresponding to the compressed data block; Position a target memory pool address based on a target path included in a blood relationship query request received in real time in response to the blood relationship query request; Calculate a GPU resource allocation weight according to a query priority, a size of a target compressed data block corresponding to the target memory pool address, and a video memory remaining rate; Schedule GPU to perform blood relationship path feature matching operations in parallel, including: comparing the target path with encoded paths in the target compressed data block, and screening an encoded path subset that meets path features; Assemble a JSON format ancestry atlas based on the screened encoded path subset. 3.The method of claim 2, wherein, After generating the JSON format ancestry atlas, perform an asynchronous checkpoint operation: Snapshot the quantum state encoding library of the quantum state blood relationship path set to an Iceberg columnar storage system every 5 minutes; When a fault occurs, restore the quantum state encoding library according to the amount of data loss and the solid state disk reading rate; Convert the restored quantum state encoding library into an Apache Arrow columnar format blood relationship matrix; In response to a received full blood relationship query request, output the blood relationship matrix through a zero-copy interface.

4. The method of claim 2, wherein the GPU resource allocation weight is determined based on the following formula: weight value = (query priority x target compressed data block size) / memory remaining rate, wherein the memory remaining rate is obtained in real time through a GPU driver interface; and the bloodline path feature matching operation adopts a memory back pressure mechanism, and when the memory remaining rate is lower than a set remaining rate threshold, new task scheduling is suspended. ； wherein, is the probability superposition state representation; is the mapping operation ground state representation, is the filtering operation ground state representation, is the connection operation ground state representation; is the mapping operation probability weight, taking values from 0.6 to 0.8; is the filtering operation probability weight, taking values from 0.1 to 0.3; is the connection operation probability weight, taking values from 0.05 to 0.15; The field mapping relationship is established, including: , , , the mapping probability of the Kafka source field to the Flink target field is calculated.

5. The method of claim 2, wherein the GPU resource allocation weight is determined based on the following formula: weight value = (query priority x target compressed data block size) / memory remaining rate, wherein the memory remaining rate is obtained in real time through a GPU driver interface; and the bloodline path feature matching operation adopts a memory back pressure mechanism, and when the memory remaining rate is lower than a set remaining rate threshold, new task scheduling is suspended. Compression rate of the Huffman coding compression is calculated by the following formula: wherein, is the total number of bloodline paths in the set of bloodline paths of quantum states; is the index of the bloodline path; is the index of the bloodline path; is the encoding length of the bloodline path in the compressed data block, in bits; is the index of the bloodline path; is the original length of the bloodline path in the set of bloodline paths of quantum states, in bytes; is the index of the bloodline path; is the frequency of the bloodline path in the original execution plan log; 6. The method of claim 3, wherein the bloodline path feature matching operation comprises: parsing a target path in the bloodline query request to generate a target feature vector, wherein the target feature vector comprises a mapping operation probability weight, a filtering operation probability weight and a connection operation probability weight of the target path; Fault recovery time is calculated by the formula: Wherein, is the total amount of quantum state encoding data that is not persisted when a failure occurs, in GB; is the solid state disk read rate, in GB / s; is the system recovery inherent overhead time, set to 10 ms. 7.The method of tracing the bloodline path based on quantum encoding and GPU acceleration according to claim 2, wherein, dividing the target compressed data block into a plurality of equal-length sub-blocks according to the number of GPU stream processors, and binding each GPU stream processor to a sub-block; sequentially reading, by each GPU stream processor, a Huffman encoding bit stream of the bound sub-block, and initializing a weight accumulation value of a candidate path when a bit pattern 1111 is identified; when a bit pattern 1100 is detected, recording a mapping operator and increasing a mapping weight accumulation value of the current candidate path; when a bit pattern 1101 is detected, recording a filtering operator and increasing a filtering weight accumulation value of the current candidate path; and when a bit pattern 1110 is detected, recording a connection operator and increasing a connection weight accumulation value of the current candidate path; when the bit pattern 1111 is identified again, calculating an absolute error sum of a cumulative weight vector of the candidate path and the target feature vector; and if the absolute error sum is less than a set tolerance threshold, writing a storage offset of the candidate path in the target compressed data block into a global matching list; after all GPU stream processors are processed, extracting corresponding encoding paths according to the storage offset in the global matching list, and aggregating the encoding paths into the encoding path subset. The method further comprises an incremental path processing mechanism, comprising: parsing a new execution plan log into an incremental bloodline path; 8.The method of tracing the genealogy path based on quantum encoding and GPU acceleration according to claim 2, wherein, merging the incremental bloodline path with the quantum state bloodline path set to form an updated bloodline path set; recomputing the occurrence frequency of each path based on the updated bloodline path set; reconstructing a Huffman encoding tree according to the recomputed occurrence frequency of each path; performing a compression operation on the updated bloodline path set using the reconstructed Huffman encoding tree to generate an updated compressed data block; releasing the storage space of the original compressed data block, storing the updated compressed data block, and updating the corresponding memory pool address mapping table. The memory pooling strategy is implemented through data heat level storage, comprising: for each compressed data block, sorting the bloodline paths included in the compressed data block in descending order according to the total number of occurrences of the bloodline paths in the original execution plan log; defining the top 20% as high-heat blocks, the last 30% as low-heat blocks, and the middle 50% as medium-heat blocks. 9.The method of tracing the bloodline path based on quantum encoding and GPU acceleration according to claim 2, wherein, ​ ​ A three-level storage architecture is established, including: a high heat storage layer: residing GPU memory, storing the high heat blocks; a medium heat storage layer: residing host memory, storing the medium heat blocks; a low heat storage layer: residing solid state disk, storing the low heat blocks; When the bloodline query request is received, if the target compressed data block corresponding to the target memory pool address is not in the high heat storage layer, the current query thread is paused, the target compressed data block is copied from the current storage layer to the GPU memory, the memory pool address mapping table is updated, and the query thread is restarted; The GPU memory usage is monitored in real time; when the GPU memory usage exceeds a set usage threshold, N compressed data blocks with the earliest access time and the least total occurrence times are selected from the high heat storage layer as migration objects, N is dynamically determined according to the memory over-limit ratio; a hierarchical degradation operation is performed on each migration object; and the memory pool address mapping table is updated synchronously.

10. The bloodline path tracking method based on quantum encoding and GPU acceleration according to claim 3, characterized in that, The element value of the blood relationship matrix Indicates the Kafka source field The dependence strength between the Flink target field The calculation method is: where, is the connection field and all the bloodline path sets; is the total number of paths in the bloodline path sets; is an element in the bloodline path sets; is the operation type sequence that the path includes; is the length of the sequence ; is the operation type in ; is the composite weight function of the operation defined as: a weight factor for a map operation type in the path; a weight factor for a filter operation type in the path; a weight factor for a join operation type in the path; a map operation probability weight, taking a value of 0.6 to 0.8; a filter operation probability weight, taking a value of 0.1 to 0.3; a join operation probability weight, taking a value of 0.05 to 0.15.

Images