A method for verifying string parsing results based on zero-knowledge proof

Abstract

The application discloses a method for verifying string parsing results based on zero-knowledge proof, which comprises the following steps: a data provider provides a private data set containing string attributes, the private data set contains a plurality of sample strings, and a prefix tree automaton is constructed locally based on an automatic string attribute parsing algorithm, and the states of the prefix tree automaton are combined to obtain a deterministic finite automaton; a zero-knowledge proof circuit is introduced between the data provider and a data consumer in the method for verifying string parsing results based on zero-knowledge proof, formal correctness of sample string parsing results can be ensured without leaking the strings in the private data set, the data consumer has the ability to detect artificially relaxed or tampered parsing results, and the credibility of a cross-organization data pipeline and a data exchange system is improved.

Description

Technical Field

[0001] This invention belongs to the field of database systems and cryptography, specifically relating to a method for verifying string parsing results based on zero-knowledge proofs. Background Technology

[0002] With the increasing scale and complexity of data ecosystems, the need to automatically extract metadata from raw data to support data validation, data integration, and data governance is becoming increasingly urgent. String attribute profiling (also known as string profile generation) is a fundamental data profiling task. Its goal is to automatically learn profiling results from raw datasets containing string attributes, describing the structure and format constraints of those attributes, such as regular expressions or structural descriptions in the form of Deterministic Finite Automata (DFA). Downstream systems can configure data validation rules, construct data cleaning processes, and assess data quality based on the string profiling results. In typical database systems, string profiling results are widely used as operational metadata for schema validation, data access pipeline configuration, anomaly detection, and the execution of data governance strategies. Existing string profiling techniques can be broadly categorized into two types: the first type is profiling methods based on heuristics and Domain-Specific Languages ​​(DSLs). These methods search for candidate profiles in a predefined regular expression or schema template space and score and select profiles based on objective functions such as coverage, conciseness, or description length. These methods perform well in practice, but their output depends on specific search strategies and scoring functions. The correctness of the analysis results is often tightly coupled with the entire reasoning process, making it difficult to characterize using simple static conditions. The second category is automaton-based analysis methods, which typically represent string patterns as deterministic finite automata. A typical approach is to first construct a Prefix Tree Acceptor (PTA) that accurately identifies all positive samples, and then compress the PTA into a deterministic finite automaton that can summarize the string structure through a series of state merging operations. Unlike heuristic search methods, the correctness of automaton-based analysis methods can be understood through the static relationship between the initial and final structures, such as whether the final automaton accepts all positive samples, rejects given negative samples, or minimizes the structure under given constraints.

[0003] In traditional on-premises deployments, data engineers can directly access raw data records and sample and verify the output of profiling algorithms, thus ensuring the accuracy of the profiling results to a certain extent. However, with the emergence of new application scenarios such as cross-organizational data sharing, regulated data analytics, and data marketplaces, data providers and data consumers often belong to different entities. Constrained by privacy regulations and business compliance requirements, raw data cannot be directly disclosed. In such scenarios, string attribute profiling is typically performed locally by the data provider. Data consumers can only receive the profiling results published by the data provider and cannot view the underlying data, thus lacking the technical means to independently verify the profiling results.

[0004] For example, in a common cross-organizational data integration scenario, an e-commerce platform, as a data consumer, needs to integrate data from different logistics service providers. Each logistics service provider maintains transportation records locally, including fields such as TrackingID. Due to trade secrets and privacy protection requirements, the original TrackingID content cannot be directly shared. Logistics service providers typically only publish string profiling results describing the TrackingID value format to the e-commerce platform, such as profiles in the form of regular expressions. The e-commerce platform configures data access and verification pipelines based on these profiles to screen for malformed or anomalous data. However, to avoid triggering regulatory penalties or internal quality alerts, logistics service providers may deliberately publish overly permissive profiles, such as using patterns that allow arbitrary alphanumeric combinations, thus allowing TrackingIDs that do not conform to the expected format to bypass downstream verification processes. Due to the lack of technical means to independently verify the profile results, the e-commerce platform finds it difficult to distinguish between profiles that faithfully reflect the actual data and manipulated profiles.

[0005] To verify the correctness of remote computation results while protecting data privacy, cryptographic techniques such as zero-knowledge proofs have been extensively studied in recent years. Zero-knowledge proofs allow the prover to prove the validity of a computational relation to the verifier without revealing the specific input data. Modern zero-knowledge proof systems are typically based on arithmetic circuits or constraint systems, representing the computational logic to be proven as a set of algebraic constraints and generating a concise proof against this constraint system. However, directly mapping string attribute parsing algorithms to general zero-knowledge circuits or replaying the complete parsing process in a zero-knowledge virtual machine often involves data-dependent control flow, dynamic creation and merging of states, and complex search spaces, resulting in huge constraint sizes and excessively long proof times, making it difficult to meet the performance and latency requirements of real-world data pipelines.

[0006] Specifically, inductive string profiling, exemplified by the RPNI automata learning algorithm, involves numerous branching, state merging decisions, and iterative convergence processes based on sample data distribution. Replaying these operations step-by-step in a general zero-knowledge circuit requires conservative expansion of all possible control flow paths and state changes, resulting in orders of magnitude constraints. Simulating the profiling process in a zero-knowledge virtual machine requires verifying long instruction sequences and complex memory access patterns, making the overhead of both generation and verification unacceptable. Therefore, existing solutions either abandon cryptographically verifiable string profiling results or are simply unsuitable for practical data management systems in terms of performance.

[0007] In summary, existing technologies have at least the following shortcomings: First, in cross-organizational data sharing and data governance scenarios, there is a lack of technical means to verify the correctness and "non-overly permissiveness" of string attribute profiling results without disclosing the original data; Second, directly embedding existing profiling algorithms into a general zero-knowledge proof framework would result in unacceptable performance overhead and make it difficult to deploy in real-world systems. Summary of the Invention

[0008] The purpose of this invention is to address the problems raised in the background art by proposing a method for verifying string parsing results based on zero-knowledge proof.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] The present invention proposes a method for verifying string profiling results based on zero-knowledge proof, comprising: a data provider providing a private dataset containing string attributes, wherein the private dataset contains multiple sample strings, and constructing a prefix tree automaton locally based on the string attribute profiling algorithm of the automaton, merging the states of the prefix tree automaton to obtain a deterministic finite automaton, converting the deterministic finite automaton into a corresponding regularization expression, and using the regularization expression as the sample string profiling result;

[0011] The state set, state transition function, and accepting state set of the prefix tree automaton, as well as the mapping relationship between each state of the prefix tree automaton and the deterministic finite automaton, are non-public data, while the state set, state transition function, and accepting state set of the deterministic finite automaton are public data.

[0012] Constraints are imposed on the correctness of the prefix tree automaton, the correctness of the state merging of the prefix tree automaton, and the minimum structure of the determined finite automaton using both public and non-public data.

[0013] The string attribute parsing algorithm based on automata and the constraints constitute a zero-knowledge proof circuit. The zero-knowledge proof generation operation is performed on the zero-knowledge proof circuit through the zkSNARK arithmetic framework that supports table lookup proof. The zero-knowledge proof is the credential that the sample string parsing result satisfies the constraints.

[0014] The data provider sends sample string profiling results, public data, and zero-knowledge proofs to the data consumer. The data consumer then verifies the correctness of the sample string profiling results based on the received information using the zkSNARK arithmetic framework that supports table lookup arguments.

[0015] Preferably, the sample strings in the private dataset include positive sample strings and negative sample strings, and all positive sample strings in the private dataset constitute a positive sample set, and all negative sample strings in the private dataset constitute a negative sample set.

[0016] The alphabet is composed of all types of characters in all sample strings of the private dataset;

[0017] Store the mapping relationship between each state of the prefix tree automaton and the deterministic finite automaton in a mapping table.

[0018] Preferably, the process of obtaining the sample string analysis result includes:

[0019] For each sample string in the set of positive examples, starting from the initial state, the state transition edges are expanded sequentially according to the order of each character in the sample string. If there is no transition for the corresponding character in the current state, a new state is created and a transition edge is added.

[0020] After traversing all the sample strings in the set of positive examples, we obtain the prefix tree automaton;

[0021] Construct candidate state pairs for the states in the prefix tree automaton according to a preset order;

[0022] Candidate state pairs that meet the preset conditions will be merged in a preset order.

[0023] The preset conditions are as follows: for each character in the alphabet, after reading each character, the two states in the candidate state pair meet the transition consistency; and after merging the two states in the candidate state pair in a preset order, a new state is obtained. Each merge is performed once to obtain the first automaton. Inputting each sample string in the negative sample set into the current first automaton will not cause the current first automaton to eventually stop in the accepting state.

[0024] After merging all state pairs in combinations that meet the preset conditions, a deterministic finite automaton is obtained. The deterministic finite automaton is then converted into a corresponding regularization expression, and the regularization expression is used as the result of sample string analysis.

[0025] Preferably, all characters in the positive and negative sample sets are concatenated to form a first tiled symbol vector and a second tiled symbol vector, and the number of characters in the first and second tiled symbol vectors is filled to a first preset value using zero characters. Second preset value ;

[0026] The characters in the padded positive and negative sample sets are represented by binary indicator vectors, with 1 representing the terminating character of the sample string and 0 representing the remaining characters. The padded positive and negative sample sets are then divided into multiple continuous segments, each corresponding to a sample string. The positive and negative sample sets after processing by the binary indicator vectors are called the positive set and the negative set, respectively.

[0027] When filling in non-public data, the states in the state set of the prefix tree finite automaton and the transition data in the state transition function are all filled with zero states to the third preset value. The received states in the received state set are filled with zero states to the fourth preset value. And based on the state set of the determined finite automaton and the state set of the padded prefix tree finite automaton, a mapping table is reconstructed, which is called the first mapping table;

[0028] The state transition functions of a deterministic finite automaton and a padded prefix tree automaton are respectively converted into corresponding sparse transition tables, and the sparse transition tables contain triples that correspond one-to-one with each transition data in the state transition function;

[0029] A hash operation is performed on the first tiled symbol vector, the second tiled symbol vector, the positive set, and the negative set after padding to obtain the corresponding hash values; the public data also includes an alphabet, preset values, and hash values.

[0030] Preferably, the constraint on the correctness of the prefix tree automaton includes:

[0031] Calculate the prefix codes of all characters in the positive set and form the first multiset. And calculate the prefix code corresponding to the terminating character of all strings in the positive set, and form the second multiset. ;

[0032] Calculate the encodings of all states in the padded prefix tree automaton and construct a third multiset. And calculate the encoding of all accepted states in the padded prefix tree automaton, and form the fourth multiset. ;

[0033] For each prefix code in the first multiset, verify that it exists in the third multiset, and for each code in the third multiset, verify that it exists in the first multiset.

[0034] For each prefix code in the second multiset, verify its existence in the fourth multiset, and for each code in the fourth multiset, verify its existence in the second multiset.

[0035] Preferably, the calculation process of the prefix code includes:

[0036] The alphabet is mapped to a set of integers, and each character in the positive set is represented by its corresponding integer in the set. The prefix code of each character in the positive set is calculated using a binary indicator vector, and the calculation formula is as follows:

[0037] ;

[0038] in, Indicates the first positive set Prefix encoding of 1 character, Indicates the first positive set The integer corresponding to each character in the set of integers Indicates a public base, and The number of letters in the alphabet is greater than the number of letters in the positive set, and the initial prefix code is: And it is 0. , Represents the first binary pointer vector. The binary indicator value of each character;

[0039] This represents the second multiset consisting of the prefix codes corresponding to the terminating symbols of all strings in the positive set;

[0040] The formulas for calculating the encoding of each state or the encoding of each receiving state in the padded prefix tree automaton are as follows:

[0041] ;

[0042] in, For state The encoding, root state The encoding, From Status input character Arrival Status , .

[0043] Preferably, the constraint on the correctness of state merging of the prefix tree automaton includes structural consistency constraints, acceptance state constraints, and semantic consistency constraints;

[0044] Structural consistency constraints include:

[0045] (1);

[0046] in, , , This is the set of states of the prefix tree automaton after it has been filled. For the alphabet, The result mapped according to the first mapping table, This refers to the transition data in the state transition function of the prefix tree automaton after it has been filled, and That is That is, the first prefix tree automaton after filling each state After characters Reaching the each state , To determine the transition data in the state transition function of a finite automaton, That is To determine the first finite automaton A state, That is To determine the first finite automaton A state, That is That is to say To determine the first finite automaton The state passes through characters Reaching the each state .

[0047] Preferably, formula (1) is expressed in the form of a zero-knowledge proof circuit as follows:

[0048] For each transition data in the padded prefix tree automaton, there exists a corresponding triplet in the corresponding sparse transition table;

[0049] and The mapping relationship between them, and and The mapping relationships between them all exist in the first mapping table;

[0050] For each transition data of a deterministic finite automaton, there exists a corresponding triple in the corresponding sparse transition table;

[0051] The acceptance state constraints include: for each acceptance state of the filled prefix tree automaton, there is a mapping relationship with the acceptance states in the set of acceptance states of the determined finite automaton; for each acceptance state of the determined finite automaton, there is a mapping relationship with the acceptance states in the set of acceptance states of the prefix tree automaton.

[0052] The semantic consistency constraint includes: inputting each sample string in the negative set into a deterministic finite automaton to obtain the first state trajectory corresponding to each sample string in the negative set, and each step in the first state trajectory exists in the sparse transition table of the deterministic finite automaton, and the termination state of the first state trajectory does not belong to the set of accepting states of the deterministic finite automaton.

[0053] Preferably, the constraint on the structural minimality of the finite automaton includes: specific constraints and structural constraints;

[0054] Specific constraints: A predetermined number of first state pairs are selected from all state pairs of a given finite automaton;

[0055] Based on the deterministic finite automaton, for each first state pair, the two states in the first state pair are merged to obtain the second automaton. One of the sample strings from the negative sample set is input into the current second automaton to obtain the corresponding second state trajectory. If each step in the second state trajectory exists in the sparse transition table of the deterministic finite automaton and the termination state of the second state trajectory belongs to the set of accepting states of the deterministic finite automaton, it indicates that the current first state pair cannot be merged, and the current first state pair is stored in the flag table until all first state pairs have been traversed.

[0056] Structural constraints: For all state pairs in a given finite automaton, after removing all first state pairs, the remaining state pairs in the given finite automaton are called second state pairs;

[0057] For each second state pair, select a character from the alphabet. After the selected character, the two states in the current second state pair reach two successor states respectively. If the two successor states belong to the state pair in the flag table, it means that the two states in the current second state pair cannot be merged. Then, the second state pair is stored in the flag table. When all second state pairs cannot be merged, the structural constraint is satisfied.

[0058] When both specific constraints and structural constraints are satisfied, it means that the structure of a given finite automaton is minimized.

[0059] Preferably, the step of pre-selecting a first state pair from all state pairs of a determined finite automaton includes:

[0060] To determine all state pairs of a finite automaton, a directed graph is constructed based on the predecessor state pairs and directed edges pointing to the successor state pairs. That is, each node in the directed graph corresponds to a state pair in the finite automaton.

[0061] Calculate all strongly connected components of a directed graph, and all strongly connected components and the directed edges between them form a cohesive graph.

[0062] For each strongly connected component with no outgoing edges in the agglomerative graph, a state pair is randomly selected from the strongly connected components with no outgoing edges as a first state pair.

[0063] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0064] 1. This method for verifying string profiling results based on zero-knowledge proof introduces a zero-knowledge proof circuit between the data provider and the data consumer. It can provide a formal correctness guarantee for the sample string profiling results without disclosing the strings in the private dataset. This enables the data consumer to detect profiling results that have been artificially relaxed or tampered with, and improves the credibility of cross-organizational data pipelines and data exchange systems.

[0065] 2. This method determines the finite automaton by converting it into the corresponding regularization expression, and uses the regularization expression as the parsing result of the sample string to construct a zero-knowledge proof circuit. This eliminates the need to replay the parsing process involving complex control flow and dynamic state updates in the zero-knowledge proof circuit, thus avoiding the constraint size expansion problem faced by the zero-knowledge proof circuit. This significantly reduces the computational overhead of proof generation and verification, and improves performance. Attached Figure Description

[0066] Figure 1 This is a flowchart illustrating the method for verifying string parsing results based on zero-knowledge proof according to the present invention.

[0067] Figure 2 A schematic diagram illustrating the relationship between the time required to generate zero-knowledge proofs for this invention and the filling capacity of the positive example sample set;

[0068] Figure 3 This diagram illustrates the relationship between the time required to generate a zero-knowledge proof for this invention and the filling capacity of the negative example sample set. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0071] In one embodiment, such as Figures 1-3 As shown, a method for verifying string profiling results based on zero-knowledge proofs is provided. This method is applied between data providers and data consumers, including:

[0072] Step 1: The data provider provides a private dataset containing string attributes, and the private dataset contains multiple sample strings. Based on the string attribute parsing algorithm of the automaton, a prefix tree automaton is constructed locally, and the states of the prefix tree automaton are merged to obtain a deterministic finite automaton. The deterministic finite automaton is converted into the corresponding regularization expression, and the regularization expression is used as the parsing result of the sample strings.

[0073] Private datasets can also contain other types of attributes, such as integers, floating-point numbers, enumerations, and boolean data.

[0074] The sample strings in the private dataset include positive and negative sample strings (where positive and negative refer to the string labels; positive examples are strings that meet the requirements, and negative examples are strings that do not). All positive sample strings in the private dataset constitute the positive sample set, and all negative sample strings in the private dataset constitute the negative sample set. All characters of all types in all sample strings of the private dataset constitute the alphabet. The mapping relationship between the states of the prefix tree automaton and the deterministic finite automaton is stored in a mapping table (i.e., the mapping table records the correspondence between the states in the prefix tree automaton and the deterministic finite automaton).

[0075] Step 1.1: The string attribute parsing algorithm based on automata constructs a prefix tree automaton locally, including:

[0076] For each sample string in the set of positive examples, starting from the initial state, the state transition edges are expanded sequentially according to the order of each character in the sample string. If there is no transition for the corresponding character in the current state, a new state is created and a transition edge is added.

[0077] After traversing all the sample strings in the set of positive samples, we obtain the prefix tree automaton (which can accurately identify all the sample strings in the set of positive samples).

[0078] Step 1.2: Merge the states of the prefix tree automaton to obtain a deterministic finite automaton, including:

[0079] Construct candidate state pairs for the states in the prefix tree automaton according to a preset order;

[0080] Candidate state pairs that meet the preset conditions will be merged in a preset order.

[0081] The preset conditions are: for each character in the alphabet, after reading each character, the two states in the candidate state pair meet the transition consistency; and after merging the two states in the candidate state pair in a preset order, a new state is obtained. Each merge is performed once to obtain the first automaton. Inputting each sample string in the negative sample set into the current first automaton will not cause the current first automaton to eventually stop in the accepting state. (The first automaton is obtained after performing one state merge during the conversion of the prefix tree automaton to the finite state automaton).

[0082] After merging all state pairs in combinations that satisfy the preset conditions, a deterministic finite automaton is obtained.

[0083] Step 1.3, the process of obtaining the sample string analysis results, includes:

[0084] The finite automata is converted into the corresponding regularization expression (the specific process is existing technology and will not be described in detail), and the regularization expression is used as the result of sample string analysis.

[0085] Step 2: The state set, state transition function, and accepting state set of the prefix tree automaton, as well as the mapping relationship between each state of the prefix tree automaton and the deterministic finite automaton, are kept as non-public data, while the state set, state transition function, and accepting state set of the deterministic finite automaton are kept as public data. The public data also includes the alphabet, the first preset value, the second preset value, the third preset value, and the fourth preset value, as well as each hash value.

[0086] Public data can be directly shared with data consumers, while non-public data cannot be shared with data consumers.

[0087] Step 3: Constrain the correctness of the prefix tree automaton, the correctness of the state merging of the prefix tree automaton, and the minimum structure of the deterministic finite automaton using public data and filled non-public data;

[0088] It should be noted that in zero-knowledge proof scenarios, directly representing each record with the actual length of each string would leak sensitive structural metadata. Therefore, this embodiment populates the positive and negative example sample sets and non-public data:

[0089] Step 3.1: When filling the positive and negative sample sets, concatenate all characters in the positive sample set to form a first tiled symbol vector, and use zero characters to fill the number of characters in the first tiled symbol vector to a first preset value. Then, all characters in the negative sample set are concatenated to form a second tiled symbol vector, and the number of characters in the second tiled symbol vector is filled to a second preset value using zero characters. ;

[0090] The characters in the padded positive and negative sample sets are represented by binary indicator vectors. In the binary indicator vector, 1 represents the terminating symbol of the sample string, and the rest are represented by 0. The padded positive and negative sample sets are then divided into multiple continuous segments, each corresponding to a sample string. The positive and negative sample sets after processing by the binary indicator vectors are called the positive set and the negative set, respectively.

[0091] The first and second tiled symbol vectors after padding, as well as the positive and negative sets, are hashed to obtain the corresponding hash values. Each hash value is also used as public data (therefore, the public data specifically includes the state set of the finite automaton, the state transition function, the accepting state set, the alphabet, the first preset value, the second preset value, the third preset value, and the fourth preset value, as well as each hash value).

[0092] Step 3.2: When filling in the non-public data, fill the states in the state set of the prefix tree finite automaton and the transition data in the state transition function with zero states to the third preset value. The received states in the received state set are filled with zero states to the fourth preset value. Based on the state set of the determined finite automaton and the state set of the padded prefix tree finite automaton, the mapping table is reconstructed and called the first mapping table;

[0093] Step 3.3: Convert the state transition functions of the determined finite automaton and the padded prefix tree automaton into corresponding sparse transition tables, and the sparse transition tables contain triplets that correspond one-to-one with each transition data in the state transition function, such as ( , , ).

[0094] Step 3.4: Constrain the correctness of the prefix tree automaton, including:

[0095] Calculate the prefix codes of all characters in the positive set and form the first multiset. And calculate the prefix code corresponding to the terminating character of all strings in the positive set, and form the second multiset. ;

[0096] Calculate the encodings of all states in the padded prefix tree automaton and construct a third multiset. And calculate the encoding of all accepted states in the padded prefix tree automaton, and form the fourth multiset. ;

[0097] For each prefix code in the first multiset, verify that it exists in the third multiset, and for each code in the third multiset, verify that it exists in the first multiset.

[0098] For each prefix code in the second multiset, verify its existence in the fourth multiset, and for each code in the fourth multiset, verify its existence in the second multiset.

[0099] If both of the above verifications are satisfied, it means that the correctness constraints of the prefix tree automaton are met.

[0100] The calculation process for the prefix code includes:

[0101] The alphabet is mapped to a set of integers, and each character in the positive set is represented by its corresponding integer in the set. The prefix code of each character in the positive set is calculated using a binary indicator vector, and the calculation formula is as follows:

[0102] ;

[0103] in, Indicates the first positive set Prefix encoding of 1 character, Indicates the first positive set The integer corresponding to each character in the set of integers Indicates a public base, and The number of letters in the alphabet is greater than the number of letters in the positive set, and the initial prefix code is: And it is 0. , Represents the first binary pointer vector. A binary indicator value for each character.

[0104] This represents the second multiset consisting of the prefix codes corresponding to the terminating symbols of all strings in the positive set;

[0105] The formulas for calculating the encoding of each state or the encoding of each receiving state in the padded prefix tree automaton are as follows:

[0106] ;

[0107] in, For state The encoding, root state The encoding, From Status input character Arrival Status , .

[0108] Step 3.5: Constrain the correctness of state merging in the prefix tree automaton, including structural consistency constraints, acceptance state constraints, and semantic consistency constraints;

[0109] Step 3.5.1, Structural consistency constraints include:

[0110] (1);

[0111] in, , , This is the set of states of the prefix tree automaton after it has been filled. For the alphabet, The result mapped according to the first mapping table (that is, the corresponding state in the state set of the prefix tree finite automaton in the state set of the determined finite automaton). This refers to the transition data in the state transition function of the prefix tree automaton after it has been filled, and That is That is, the first prefix tree automaton after filling each state After characters Reaching the each state , To determine the transition data in the state transition function of a finite automaton, That is To determine the first finite automaton A state, That is To determine the first finite automaton A state, That is That is to say To determine the first finite automaton The state passes through characters Reaching the each state ;

[0112] Formula (1) is expressed in the form of zero-knowledge proof circuit as follows:

[0113] For each transition data in the padded prefix tree automaton, there exists a corresponding triple in the corresponding sparse transition table (i.e., the sparse transition table of the padded prefix tree automaton);

[0114] and The mapping relationship between them, and and The mapping relationships between them all exist in the first mapping table;

[0115] For each transition data in a deterministic finite automaton, there exists a corresponding triple in the corresponding sparse transition table (i.e., the sparse transition table of the deterministic finite automaton).

[0116] Step 3.5.2, Acceptance State Constraints include: For each acceptance state of the padded prefix tree automaton, there exists a mapping relationship with the acceptance states in the set of acceptance states of the determined finite automaton (i.e., for all...). ∈ ,have ∈ ,in This is the set of accepting states for the padded prefix tree automaton. To determine the set of accepting states of a finite automaton, For each accepting state in the set of accepting states of the padded prefix tree automaton, there exists a mapping relationship between each accepting state of the deterministic finite automaton and the accepting states in the set of accepting states of the prefix tree automaton (i.e., for each q ∈ ,exist ∈ Make = q, where q is each accepting state in the set of accepting states of a finite automaton.

[0117] Step 3.5.3, Semantic consistency constraints include:

[0118] For positive sets, consistency is automatically satisfied during the construction process. The padded prefix tree automaton is defined to accept all strings in the positive sets; the merge operation only identifies the state without deleting the accepting path; under the above accepting state constraint, the acceptability of the padded prefix tree automaton's accepting state is preserved under the mapping. Therefore, the final determined finite automaton necessarily accepts all positive sets without additional constraints.

[0119] For the negative set, each sample string in the negative set is input into a deterministic finite automaton to obtain the first state trajectory corresponding to each sample string in the negative set (that is, the trajectory sequence formed by the states traversed after each sample string in the negative set is used as input to the deterministic automaton, i.e., the state sequence). , This is the initial state. The first state trajectory is the termination state, and each step in the first state trajectory exists in the sparse transition table of the deterministic finite automaton (i.e., the transition data corresponding to each state in the first state trajectory exists in the sparse transition table of the deterministic finite automaton), and the termination state of the first state trajectory does not belong to the set of accepting states of the deterministic finite automaton.

[0120] Step 3.6: Constraining the structural minimality of the finite automaton includes specific constraints and structural constraints;

[0121] Step 3.6.1 Specific constraints: Select a preset number of first state pairs from all state pairs of the finite automaton in advance (that is: construct a directed graph by pointing all state pairs of the finite automaton to the successor state pairs through directed edges based on the predecessor state pairs, that is, each node in the directed graph corresponds to each state pair in the finite automaton).

[0122] Calculate all strongly connected components of a directed graph, and all strongly connected components and the directed edges between them form a cohesive graph.

[0123] For each strongly connected component with no outgoing edges in the agglomerative graph, randomly select a state pair from the strongly connected components with no outgoing edges as a first state pair.

[0124] Based on the deterministic finite automaton, for each first state pair, the two states in the first state pair are merged to obtain the second automaton. One of the sample strings from the negative sample set is input into the current second automaton to obtain the corresponding second state trajectory. If each step in the second state trajectory exists in the sparse transition table of the deterministic finite automaton and the termination state of the second state trajectory belongs to the set of accepting states of the deterministic finite automaton, it indicates that the current first state pair cannot be merged, and the current first state pair is stored in the flag table until all first state pairs have been traversed.

[0125] Step 3.6.2, Structural Constraints: For all state pairs in a given finite automaton, remove all first state pairs, and the remaining state pairs in the given finite automaton are called second state pairs;

[0126] For each second state pair, select a character from the alphabet (arbitrarily select a character). After the two states in the current second state pair pass through the selected character, they reach two successor states respectively. If the two successor states belong to the state pair in the flag table, it means that the two states in the current second state pair cannot be merged. Then, the second state pair is stored in the flag table. When all second state pairs cannot be merged, the structural constraint is satisfied.

[0127] When both specific constraints and structural constraints are satisfied, it means that the structure of a given finite automaton is minimized.

[0128] Step 4: The string attribute parsing algorithm based on automata and the constraints constitute the zero-knowledge proof circuit. The zero-knowledge proof generation operation is performed on the zero-knowledge proof circuit through the zkSNARK arithmetic framework that supports table lookup argument (such as the Plonkish arithmetic framework, (zkSNARK, Zero-KnowledgeSuccinct Non-Interactive Argument of Knowledge)). The zero-knowledge proof is the credential that the sample string parsing result satisfies the constraints (i.e., correctness).

[0129] It should be noted that the zero-knowledge proof circuit of this method is implemented using the Plonkish arithmetic framework, which supports lookup arguments. Plonkish arithmetic is a type of constraint system based on the Plonk protocol. In addition to supporting standard multiplication gate constraints, it also supports extended constraint forms such as custom gates and lookup arguments. Among them, lookup arguments allow the circuit to verify whether a value exists in a pre-committed table with constant constraint overhead. This capability is the foundation for operations such as sparse transition table lookup, encoding set inclusion checking, and state mapping verification in the various sub-circuits of this invention.

[0130] The zero-knowledge proof circuit is written in Noir, a domain-specific language for zero-knowledge proofs that supports compiling high-level constraint logic into a Plonkish arithmetic form of the constraint system. Proof generation utilizes the UltraHonk backend, a concise, non-interactive zero-knowledge proof scheme based on the Plonk protocol with built-in table lookup support, efficiently handling the numerous table lookup operations relied upon by this invention. Cryptographic commitments employ the Poseidon hash function, optimized for arithmetic circuits and exhibiting low constraint overhead in finite field operations. To ensure data insensitivity, all arrays related to the data size are padded to the nearest power of 2.

[0131] It is understood that the zero-knowledge proof circuit design of this method does not depend on the specific implementation framework mentioned above. Theoretically, any Plonk-based zero-knowledge concise non-interactive proof (zkSNARK) framework that supports lookup proofs can be used as the proof backend of this method. For example, in addition to the Noir / UltraHonk combination mentioned above, the Halo2 framework (based on Plonkish arithmetic, supporting lookup proofs and custom gates), other zkSNARK systems based on Plonk variants (e.g., systems supporting Plookup lookup proofs), etc., can all be used as alternatives without changing the circuit constraint logic of this invention. The specific zero-knowledge proof framework and backend implementation can be selected according to the actual deployment environment, performance requirements, and security parameters, and this application does not impose specific limitations on them.

[0132] Step 5: The data provider sends sample string profiling results, public data, and zero-knowledge proofs to the data consumer. The data consumer verifies the correctness of the sample string profiling results based on the received information using a zkSNARK arithmetic framework that supports lookup arguments (such as the Plonkish arithmetic framework). This allows the data consumer to verify whether the string profiling results published by the data provider truly reflect the underlying data, rather than being artificially relaxed or tampered with, without accessing the data provider's private dataset, thereby improving the credibility of cross-organizational data pipelines.

[0133] In another embodiment, the present invention also provides a system for verifying string parsing results based on zero-knowledge proof, including a processor and a memory storing a plurality of computer instructions, which, when executed by the processor, implement the steps of the automatic response method that integrates dual word segmentation and iterative feedback.

[0134] For specific limitations on systems for verifying string parsing results based on zero-knowledge proofs, please refer to the limitations on methods for verifying string parsing results based on zero-knowledge proofs mentioned above, which will not be repeated here.

[0135] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, and the processor implements the method of the present invention by running the computer program stored in the memory.

[0136] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory is used to store programs, which the processor executes upon receiving execution instructions.

[0137] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0138] This method will be illustrated with examples:

[0139] The data consumers are e-commerce platforms, and the data providers are multiple third-party logistics service providers. Each logistics service provider stores transportation records containing the TrackingID field locally. Due to trade secret and privacy protection requirements, the original TrackingID content cannot be directly disclosed. Based on the method of this invention, each logistics service provider performs string attribute parsing on the TrackingID field locally to obtain parsing results describing the TrackingID format and generates corresponding zero-knowledge proofs.

[0140] For example, a legitimate TrackingID might satisfy the pattern "TRK-[AZ]{2}-[0-9]{4}", while a manipulated profiling might use an overly lenient pattern "[A-Za-z0-9]{1,32}", allowing numerous malformed or non-compliant identifiers to bypass downstream validation. In this approach, the logistics service provider locally constructs a prefix tree automaton and performs state merging to generate a deterministic finite automaton. This automaton structure is used as the profiling of the TrackingID, and constraints on structural validity, data consistency, and structural minimality are imposed in the zero-knowledge proof circuit to generate a zero-knowledge proof.

[0141] This method for verifying string profiling results based on zero-knowledge proofs introduces a zero-knowledge proof circuit between the data provider and the data consumer. It provides a formalized correctness guarantee for the sample string profiling results without disclosing strings within the private dataset, enabling the data consumer to detect profiling results that have been artificially relaxed or tampered with, thus improving the reliability of cross-organizational data pipelines and data exchange systems. This method converts a finite automaton into a corresponding regularization expression, uses the regularization expression as the sample string profiling result, and constructs a zero-knowledge proof circuit. This avoids replaying the profiling process, which involves complex control flow and dynamic state updates, within the zero-knowledge proof circuit, thus preventing the leakage of zero-knowledge proofs. This method addresses the constraint scaling problem faced by proof circuits, thereby significantly reducing the computational overhead of proof generation and verification and improving performance. While maintaining parsing accuracy comparable to plaintext parsing algorithms, it achieves an order-of-magnitude reduction in proof generation time compared to general solutions based on zero-knowledge virtual machines, making it feasible to deploy verifiable string attribute parsing in scenarios such as real-world data management, regulated data sharing, and data markets. The technical solution of this invention has good scalability and versatility, and can be implemented by combining different types of automata models, different string parsing algorithms, and different zero-knowledge proof backends, facilitating flexible configuration according to specific business needs and system constraints.

[0142] To verify the necessity of the three constraints in this method, the following comparative experiment was conducted:

[0143] The configurations under the three constraints in this method are compared with the configurations under the following three performance metrics: removing the correctness constraint on the prefix tree automaton, removing the correctness constraint on the state merging of the prefix tree automaton, and removing the structural minimization constraint on the deterministic finite automaton. The three performance metrics are the number of constraints (which represents the complexity of generating zero-knowledge proofs), the time to generate zero-knowledge proofs, and the speed ratio (the speed ratio is the base time / the time to generate zero-knowledge proofs under the current configuration, where the base time is the time to generate zero-knowledge proofs corresponding to the configurations under the three constraints in this method). Table 1 shows the comparison.

[0144] Table 1

[0145] Three constraints of this method 258K 41.1s 1.0× Remove the correctness constraints on the prefix tree automaton. 6120K 603.5s 14.7× Remove the correctness constraint on state merging of the prefix tree automaton. - - - Remove the structural minimum constraint on a deterministic finite automaton 3554K 336.4s 8.2×

[0146] In Table 1, K represents thousand. As can be seen from Table 1, the configuration in this method where all three constraints exist simultaneously results in the smallest ratio of constraint number, time to generate zero-knowledge proof, and speed. Therefore, each constraint in this method plays a unique role, and the absence of any one of them will lead to a multiple-fold decrease in performance or make proof generation completely infeasible.

[0147] The total complexity of the three constraints in this method is: ,in To determine the number of states in the state set of a finite automaton.

[0148] Figures 2-3 This represents the relationship between the time required to generate a zero-knowledge proof and the filling capacity of the positive and negative example sample sets, and Figures 2-3 This indicates that the time required to generate zero-knowledge proofs increases linearly with the filling capacity.

[0149] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0150] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0151] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for verifying string parsing results based on zero-knowledge proof, characterized in that: The method is applied between data providers and data consumers, including: The data provider provides a private dataset containing string attributes, and the private dataset contains multiple sample strings. Based on the string attribute parsing algorithm of the automaton, a prefix tree automaton is constructed locally, and the states of the prefix tree automaton are merged to obtain a deterministic finite automaton. The deterministic finite automaton is converted into the corresponding regularization expression, and the regularization expression is used as the parsing result of the sample strings. The state set, state transition function, and accepting state set of the prefix tree automaton, as well as the mapping relationship between each state of the prefix tree automaton and the deterministic finite automaton, are non-public data, while the state set, state transition function, and accepting state set of the deterministic finite automaton are public data. Constraints are imposed on the correctness of the prefix tree automaton, the correctness of the state merging of the prefix tree automaton, and the minimum structure of the determined finite automaton using both public and non-public data. The string attribute parsing algorithm based on automata and the constraints constitute a zero-knowledge proof circuit. The zero-knowledge proof generation operation is performed on the zero-knowledge proof circuit through the zkSNARK arithmetic framework that supports table lookup proof. The zero-knowledge proof is the credential that the sample string parsing result satisfies the constraints. The data provider sends sample string profiling results, public data, and zero-knowledge proofs to the data consumer. The data consumer verifies the correctness of the sample string profiling results based on the received information using the zkSNARK arithmetic framework that supports table lookup arguments. The sample strings in the private dataset include positive sample strings and negative sample strings, and all positive sample strings in the private dataset constitute a positive sample set, and all negative sample strings in the private dataset constitute a negative sample set. The alphabet is composed of all types of characters in all sample strings of the private dataset; Store the mapping relationship between each state of the prefix tree automaton and the deterministic finite automaton in a mapping table; All characters in the positive and negative sample sets are concatenated to form a first tiled symbol vector and a second tiled symbol vector. Zero characters are then used to fill the number of characters in the first and second tiled symbol vectors to a first preset value. Second preset value ; The characters in the padded positive and negative sample sets are represented by binary indicator vectors, with 1 representing the terminating character of the sample string and 0 representing the remaining characters. The padded positive and negative sample sets are then divided into multiple continuous segments, each corresponding to a sample string. The positive and negative sample sets after processing by the binary indicator vectors are called the positive set and the negative set, respectively. When filling in non-public data, the states in the state set of the prefix tree finite automaton and the transition data in the state transition function are all filled with zero states to the third preset value. The received states in the received state set are filled with zero states to the fourth preset value. And based on the state set of the determined finite automaton and the state set of the padded prefix tree finite automaton, a mapping table is reconstructed, which is called the first mapping table; The state transition functions of a deterministic finite automaton and a padded prefix tree automaton are respectively converted into corresponding sparse transition tables, and the sparse transition tables contain triples that correspond one-to-one with each transition data in the state transition function; A hash operation is performed on the first tiled symbol vector, the second tiled symbol vector, the positive set, and the negative set after padding to obtain the corresponding hash values; the public data also includes an alphabet, preset values, and hash values.

2. The method for verifying string parsing results based on zero-knowledge proof as described in claim 1, characterized in that: The process of obtaining the sample string parsing results includes: For each sample string in the set of positive examples, starting from the initial state, the state transition edges are expanded sequentially according to the order of each character in the sample string. If there is no transition for the corresponding character in the current state, a new state is created and a transition edge is added. After traversing all the sample strings in the set of positive examples, we obtain the prefix tree automaton; Construct candidate state pairs for the states in the prefix tree automaton according to a preset order; Candidate state pairs that meet the preset conditions will be merged in a preset order. The preset conditions are as follows: for each character in the alphabet, after reading each character, the two states in the candidate state pair meet the transition consistency; and after merging the two states in the candidate state pair in a preset order, a new state is obtained. Each merge is performed once to obtain the first automaton. Inputting each sample string in the negative sample set into the current first automaton will not cause the current first automaton to eventually stop in the accepting state. After merging all state pairs in combinations that meet the preset conditions, a deterministic finite automaton is obtained. The deterministic finite automaton is then converted into a corresponding regularization expression, and the regularization expression is used as the result of sample string analysis.

3. The method for verifying string parsing results based on zero-knowledge proof as described in claim 1, characterized in that: The constraint on the correctness of the prefix tree automaton includes: Calculate the prefix codes of all characters in the positive set and form the first multiset. And calculate the prefix code corresponding to the terminating character of all strings in the positive set, and form the second multiset. ; Calculate the encodings of all states in the padded prefix tree automaton and construct a third multiset. And calculate the encoding of all accepted states in the padded prefix tree automaton, and form the fourth multiset. ; For each prefix code in the first multiset, verify that it exists in the third multiset, and for each code in the third multiset, verify that it exists in the first multiset. For each prefix code in the second multiset, verify its existence in the fourth multiset, and for each code in the fourth multiset, verify its existence in the second multiset.

4. The method for verifying string parsing results based on zero-knowledge proof as described in claim 3, characterized in that: The calculation process for prefix encoding includes: The alphabet is mapped to a set of integers, and each character in the positive set is represented by its corresponding integer in the set. The prefix code of each character in the positive set is calculated using a binary indicator vector, and the calculation formula is as follows: ； in, Indicates the first positive set Prefix encoding of 1 character, Indicates the first positive set The integer corresponding to each character in the set of integers Indicates a public base, and The number of letters in the alphabet is greater than the number of letters in the positive set, and the initial prefix code is: And it is 0. , Represents the first binary pointer vector. The binary indicator value of each character; This represents the second multiset consisting of the prefix codes corresponding to the terminating symbols of all strings in the positive set; The formulas for calculating the encoding of each state or the encoding of each receiving state in the padded prefix tree automaton are as follows: ； in, For state The encoding, root state The encoding, From Status input character Arrival Status , .

5. The method for verifying string parsing results based on zero-knowledge proof as described in claim 1, characterized in that: The constraints on the correctness of state merging of the prefix tree automaton include structural consistency constraints, acceptance state constraints, and semantic consistency constraints. Structural consistency constraints include: （1）； in, , , This is the set of states of the prefix tree automaton after it has been filled. For the alphabet, The result mapped according to the first mapping table, This refers to the transition data in the state transition function of the prefix tree automaton after it has been filled, and That is That is, the first prefix tree automaton after filling each state After characters Reaching the each state , To determine the transition data in the state transition function of a finite automaton, That is To determine the first finite automaton A state, That is To determine the first finite automaton A state, That is That is to say To determine the first finite automaton The state passes through characters Reaching the each state .

6. The method for verifying string parsing results based on zero-knowledge proof as described in claim 5, characterized in that: Formula (1) in a zero-knowledge proof circuit is expressed as follows: For each transition data in the padded prefix tree automaton, there exists a corresponding triplet in the corresponding sparse transition table; and The mapping relationship between them, and and The mapping relationships between them all exist in the first mapping table; For each transition data of a deterministic finite automaton, there exists a corresponding triple in the corresponding sparse transition table; The acceptance state constraints include: for each acceptance state of the filled prefix tree automaton, there is a mapping relationship with the acceptance states in the set of acceptance states of the determined finite automaton; for each acceptance state of the determined finite automaton, there is a mapping relationship with the acceptance states in the set of acceptance states of the prefix tree automaton. The semantic consistency constraint includes: inputting each sample string in the negative set into a deterministic finite automaton to obtain the first state trajectory corresponding to each sample string in the negative set, and each step in the first state trajectory exists in the sparse transition table of the deterministic finite automaton, and the termination state of the first state trajectory does not belong to the set of accepting states of the deterministic finite automaton.

7. The method for verifying string parsing results based on zero-knowledge proof as described in claim 1, characterized in that: The constraints on determining the structural minimumity of a finite automaton include: specific constraints and structural constraints; Specific constraints: A predetermined number of first state pairs are selected from all state pairs of a given finite automaton; Based on the deterministic finite automaton, for each first state pair, the two states in the first state pair are merged to obtain the second automaton. One of the sample strings from the negative sample set is input into the current second automaton to obtain the corresponding second state trajectory. If each step in the second state trajectory exists in the sparse transition table of the deterministic finite automaton and the termination state of the second state trajectory belongs to the set of accepting states of the deterministic finite automaton, it indicates that the current first state pair cannot be merged, and the current first state pair is stored in the flag table until all first state pairs have been traversed. Structural constraints: For all state pairs in a given finite automaton, after removing all first state pairs, the remaining state pairs in the given finite automaton are called second state pairs; For each second state pair, select a character from the alphabet. After the selected character, the two states in the current second state pair reach two successor states respectively. If the two successor states belong to the state pair in the flag table, it means that the two states in the current second state pair cannot be merged. Then, the second state pair is stored in the flag table. When all second state pairs cannot be merged, the structural constraint is satisfied. When both specific constraints and structural constraints are satisfied, it means that the structure of a given finite automaton is minimized.

8. The method for verifying string parsing results based on zero-knowledge proof as described in claim 7, characterized in that: The step of pre-selecting a first state pair from all state pairs of a determined finite automaton includes: To determine all state pairs of a finite automaton, a directed graph is constructed based on the predecessor state pairs and directed edges pointing to the successor state pairs. That is, each node in the directed graph corresponds to a state pair in the finite automaton. Calculate all strongly connected components of a directed graph, and all strongly connected components and the directed edges between them form a cohesive graph. For each strongly connected component with no outgoing edges in the agglomerative graph, a state pair is randomly selected from the strongly connected components with no outgoing edges as a first state pair.

Images