Semi-structured data dynamic sampling and secure storage method

Abstract

The invention provides a dynamic sampling and secure storage method for semi-structured data. The dynamic sampling and secure storage method comprises the following steps: reading to-be-sampled semi-structured data from a data source; determining the data type of the to-be-sampled semi-structured data and analyzing the data structure of the to-be-sampled semi-structured data; determining a first semi-structured sampling algorithm according to the data type, the data structure and a preset sampling requirement; determining the total sampling amount according to the to-be-sampled semi-structured data; inputting the read to-be-sampled semi-structured data and the total sampling amount into a first semi-structured sampling algorithm, executing the first semi-structured sampling algorithm, and completing sampling to obtain sampled sample data; determining a first encryption algorithm, and encrypting the sampled sample data; obtaining a sampling data object according to a preset sampling basic data structure; and the sampling data object is integrated, stored or transmitted, so that the technical effects of improving the flexibility and accuracy of semi-structured data sampling and ensuring the safety and traceability of the sampling data are achieved.

Description

Technical Field

[0001] The present invention relates to the field of digital storage technology, and in particular to a method for dynamic sampling and secure storage of semi-structured data. Background Art

[0002] With the rapid development of information technology, semi-structured data such as JSON, CSV, and web data has become an essential component of big data processing. However, current semi-structured data sampling and secure storage technologies still face numerous challenges. Existing technologies typically use a single sampling algorithm, which struggles to accommodate diverse data types and specific sampling requirements, limiting the flexibility and accuracy of sampling results. Furthermore, the security of the sampling process is often overlooked, making sampling results susceptible to tampering or leakage, posing a serious threat to data security. To address these issues, the industry has begun exploring more flexible and accurate sampling algorithms and more secure data storage methods. Designing diverse sampling algorithms based on diverse data characteristics is key to improving sampling flexibility and accuracy. Furthermore, the introduction of encryption technology has provided new solutions for the secure storage of sampled data. Encryption effectively prevents data leakage and tampering during the sampling and storage processes, ensuring data integrity and security. However, improvements in algorithms and encryption alone are not sufficient to fully resolve existing challenges. Designing a reasonable storage and management structure for sampled data to ensure orderly and efficient data storage and management remains a pressing technical challenge.

[0003] Existing unstructured data sampling methods have the following shortcomings: First, sampling algorithms lack flexibility and adaptability: In existing technologies, sampling algorithms for semi-structured data are often relatively simple, lacking flexibility and adaptability to different data types and structures. For example, some algorithms may only be suitable for semi-structured data in specific formats (such as JSON or CSV) and cannot effectively process data in other formats. This limits the application scope of the sampling algorithms and may affect the accuracy and integrity of the sampling results in certain scenarios. Second, data security vulnerabilities exist: In existing technologies, insufficient attention is paid to the data security of sampling results. Sampled data may contain sensitive information. Without proper encryption, it is easily accessed or tampered with by unauthorized personnel. Moreover, even if encryption is implemented, improper encryption algorithms or vulnerabilities in the encryption process can lead to data leakage or corruption. Third, sampled data lacks traceability and integrity: In existing technologies, the traceability and integrity of sampled data are often not effectively guaranteed. Once the sampling process is completed, it is difficult to trace and verify the sampling results to ensure their accuracy and integrity. This deficiency not only limits the reliability and effectiveness of the sampled data in subsequent analysis and application, but may also lead to data disputes and legal risks; Fourth, there are deficiencies in the storage and management of sampled data: such as unreasonable storage structure, increased risk of data loss or damage, etc., which further weaken the practicality and value of the sampled data. Summary of the Invention

[0004] The present invention provides a method for dynamic sampling and secure storage of semi-structured data, which solves the technical problems in the prior art of single sampling algorithm and insufficient flexibility when processing semi-structured data, often ignoring data security during the data sampling process, resulting in sampling results being easily tampered with or leaked, and unreasonable storage structure of sampled data and inconvenient management. It achieves the technical effect of improving the flexibility and accuracy of semi-structured data sampling, ensuring the security and traceability of sampled data, orderly and efficiently storing and managing sampled data, and improving the comprehensive control and management capabilities of data sampling, and has important practical application value.

[0005] In view of the above problems, the present invention provides a method for dynamic sampling and secure storage of semi-structured data, the method comprising: step 1: reading the semi-structured data to be sampled from a data source; step 2: determining the data type of the semi-structured data to be sampled and parsing the data structure of the semi-structured data to be sampled; step 3: determining a first semi-structured sampling algorithm according to the data type, data structure and preset sampling requirements; step 4: determining a total sampling amount according to the semi-structured data to be sampled, wherein the total sampling amount is an integer; step 5: reading the data in step 1 into The semi-structured data to be sampled and the total sampling amount determined in step 4 are input into the first semi-structured sampling algorithm, and the first semi-structured sampling algorithm is executed, and sampling is completed to obtain sampling sample data; Step 6: Determine a first encryption algorithm to encrypt the sampling sample data; Step 7: Obtain a sampling data object according to a preset sampling basic data structure, wherein the sampling data object includes a description block and a data block, and the data block is used to store the encrypted sampling sample data; Step 8: Integrate and store or transmit the sampling data object.

[0006] Preferably, in step 1, the storage format of the semi-structured data to be sampled is JSON, XML, or CSV.

[0007] Preferably, in the step 3, it specifically includes: when the storage format of the semi-structured data to be sampled is JSON, determining the first semi-structured sampling algorithm includes: path-based sampling: analyzing the data structure of the JSON data, determining the data path to be extracted, and determining the data that meets the path as a sample based on the path expression; key-value pair-based sampling: treating the JSON data as a set of key-value pairs, and determining the corresponding value as a sample based on the specified key or combination of keys; statistics-based sampling: if the JSON data contains a large number of similar objects or array elements, a statistical method can be used to select samples.

[0008] Preferably, in step 3, specifically including: when the storage format of the semi-structured data to be sampled is HTML, determining the first semi-structured sampling algorithm includes: sampling based on web page structure: using an HTML parser to parse the web page, building a DOM tree, and selecting specific nodes or node sets as samples based on the structure of the DOM tree; sampling based on CSS selectors: using the CSS selector to locate specific elements in the web page, and determining qualified elements as samples based on the selector expression; sampling based on web page content: performing text analysis on the web page content, extracting keywords or topics, and selecting relevant paragraphs or sentences as samples based on the distribution of keywords or topics.

[0009] Preferably, in the step 3, it specifically includes: when the storage format of the semi-structured data to be sampled is CSV, determining the first semi-structured sampling algorithm includes: row-based sampling: selecting one or more rows in the CSV file as samples; column-based sampling: selecting one or more columns in the CSV file as samples; condition-based sampling: according to preset selection conditions, selecting rows or columns that meet the preset selection conditions as samples, wherein the preset selection conditions include value range and string matching.

[0010] Preferably, in step 4, the method specifically includes: assuming that the total capacity is N and the sampling ratio is p, where p is a real number between 0 and 1; the calculation formula of the total sampling amount can be expressed as: in, To perform a rounding operation, ensure that the total number of samples is an integer.

[0011] Preferably, in step 6, the first encryption algorithm may be a symmetric encryption algorithm or an asymmetric encryption algorithm; when the first encryption algorithm is a symmetric encryption algorithm, it is necessary to ensure that both communicating parties obtain and share the same secret key.

[0012] Preferably, in step 6, including: when the first encryption algorithm is the AES algorithm, specifically including: plaintext grouping: the AES algorithm is a block encryption algorithm, and the length of each block is 128 bits. If the length of the plaintext to be encrypted is not an integer multiple of 16 bytes, padding is required to ensure that the plaintext length meets the requirement; key selection: the AES algorithm supports three key lengths of 128 bits, 192 bits and 256 bits, which correspond to the three encryption modes of AES-128, AES-192 and AES-256 respectively; key expansion: according to the selected key length, multiple rounds of keys are generated by the key expansion algorithm ; Initial round key addition: perform an XOR operation on the initial key and the obtained plaintext group as the input of the initial encryption round; round function byte replacement: replace each byte of the input through a fixed replacement table; round function row shift: perform a row shift operation on the input state matrix; round function column confusion: perform a matrix multiplication operation on the state matrix and a fixed confusion matrix; round function round key addition: perform an XOR operation on the round key of the current round and the state matrix; the last round of the round function: omit the column confusion step, and add an XOR operation of the round key; after multiple round function processing, the encrypted ciphertext is obtained and output.

[0013] Preferably, in step 7, the description block is used to store information of the sampled data, including: sampling data type, sampling data file name, sampling data size, sampling data pattern URL or file path, sampling data content summary, special description block, sampling range or path, sampling rules, sampling algorithm, sample encryption algorithm, description block encryption algorithm.

[0014] Preferably, in step 8, after the encryption of the sample data is completed, the description block is integrated with the encrypted data block to obtain the sample data object, and the sample data object is stored on a storage medium.

[0015] The above one or more technical solutions in the embodiments of the present application have at least one or more of the following technical effects:

[0016] An embodiment of the present invention provides a method for dynamic sampling and secure storage of semi-structured data, comprising the steps of: reading semi-structured data to be sampled from a data source; determining a data type of the semi-structured data to be sampled and parsing a data structure of the semi-structured data to be sampled; determining a first semi-structured sampling algorithm based on the data type, data structure, and preset sampling requirements; determining a total sampling amount based on the semi-structured data to be sampled, wherein the total sampling amount is an integer; inputting the semi-structured data to be sampled read in step 1 and the total sampling amount determined in step 4 into the first semi-structured sampling algorithm, executing the first semi-structured sampling algorithm, completing sampling, and obtaining sampled sample data; determining a first encryption algorithm to encrypt the sampled sample data; obtaining a sampled data object based on a preset sampling basic data structure, wherein the sampled data object includes a description block and a data block, and the data block is used to store the encrypted sampled sample data; and integrating, storing, or transmitting the sampled data object. The method carefully designs the structure of the sampled data, including the basic data structure and the sampled data storage structure, to ensure the accuracy of the sampling process and the integrity of the data. In the overall control algorithm design, the method first reads the semi-structured data to be sampled, then parses its structure and selects a suitable sampling algorithm based on the data type, overall capacity and specific sampling requirements. For semi-structured data in different formats such as JSON, web pages and CSV, the method provides a variety of sampling algorithms based on paths, key-value pairs, statistics, web page structure, CSS selectors and web page content, as well as rows, columns and conditions to meet the sampling needs in different scenarios. After determining the total sampling amount, the method executes the selected sampling algorithm and encrypts the sampling results to ensure data security. Finally, based on the designed sampling basic data structure, this method generates a sampling data object containing a description block and a data block, and finally encrypts the data description block and the data block to achieve secure storage, thereby solving the technical problems in the existing technology that the sampling algorithm is single and lacks flexibility when processing semi-structured data, and the security of the data is often ignored during the data sampling process, which makes the sampling results easy to be tampered with or leaked. It achieves the technical effect of improving the flexibility and accuracy of semi-structured data sampling, ensuring the security and traceability of the sampled data, and enhancing the comprehensive control and management capabilities of data sampling, which has important practical application value.

[0017] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of a process for dynamic sampling and secure storage of semi-structured data in an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a sample data storage structure of a method for dynamic sampling and secure storage of semi-structured data in an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of part of the data to be sampled in the second embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of path and sample sampling results in the second embodiment of the present invention;

[0022] Figure 5 This is a schematic diagram of adding samples to a sampling sample data list in the second embodiment of the present invention;

[0023] Figure 6 Schematic diagram of the encrypted sample data list in the second embodiment of the present invention (displayed in hexadecimal). DETAILED DESCRIPTION

[0024] An embodiment of the present invention provides a method for dynamic sampling and secure storage of semi-structured data, which is used to solve the technical problems in the existing technology of single sampling algorithm and insufficient flexibility when processing semi-structured data, often ignoring data security during the data sampling process, resulting in the sampling results being easily tampered with or leaked, and unreasonable storage structure of sampled data and inconvenient management.

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0026] Example 1

[0027] Figure 1 FIG. 1 is a flow chart of a method for dynamic sampling and secure storage of semi-structured data in an embodiment of the present invention. Figure 1 The embodiment of the present invention provides a method for dynamic sampling and secure storage of semi-structured data, wherein the method includes:

[0028] Step 1: Read the semi-structured data to be sampled from the data source.

[0029] Furthermore, in step 1, the storage format of the semi-structured data to be sampled is JSON, XML, or CSV.

[0030] Specifically, the overall control algorithm design for semi-structured data sampling of the present invention first requires reading the semi-structured data to be sampled, that is, reading the semi-structured data to be sampled from a specified data source (such as a database, file system, etc.). The data can be stored in a format such as JSON, XML, or CSV.

[0031] Furthermore, the structural design of the sampling data of the present invention includes two parts: basic data structure design and sampling data storage structure design to ensure the accuracy of the sampling process and the integrity of the data. Among them, the basic data structure design is specifically as follows:

[0032] The basic data structure BscDtStruct (Basic data structure) is defined as:

[0033] BscDtStruct::= D1 D2 D3 D4 D5 D6 D7 D8 D9 D10 D11 (1)

[0034] in,

[0035] D1::=<sampling data type>

[0036] D2::=<sample data file name>

[0037] D3::=<sample data size>

[0038] D4::=<URI or file path of the sample data pattern>

[0039] D5::=<Summary of sampling data>

[0040] D6::=<Special instructions block>

[0041] D61::=<Nested structure description>

[0042] D611::=<Nesting Depth>

[0043] D612::=<name or identifier of nested structure>

[0044] D613::=<Type of nested structure>

[0045] D614::=<path expression of nested structure

[0046] D615::=<Nested structure field name

[0047] D616::=<Nested structure field type

[0048] D62::=<Key field description>

[0049] D621::=<Key field name>

[0050] D622::=<path expression of key field>

[0051] D63::=<External reference description>

[0052] D631::=<external reference name>

[0053] D632::=<external reference type>

[0054] D633::=<external reference path template>

[0055] D634::=<value of external reference>

[0056] D64::<version compatibility>

[0057] D7::=<sampling range or path>

[0058] D8::=<Sampling Sample Size>

[0059] D9::=<sampling algorithm / rule>

[0060] D10::=<Sample Encryption Algorithm>

[0061] D11::=<Indicates whether the block is encrypted>.

[0062] Step 2: Determine the data type of the semi-structured data to be sampled and parse the data structure of the semi-structured data to be sampled.

[0063] Specifically, after reading the semi-structured data to be sampled, it is necessary to determine the type of the semi-structured data to be sampled and parse the structure of the semi-structured data to be sampled, that is, to parse the pattern or structure of the semi-structured data to determine the hierarchy and relationship of the data.

[0064] Step 3: Determine a first semi-structured sampling algorithm based on the data type, data structure and preset sampling requirements.

[0065] Furthermore, in the step 3, it specifically includes: when the storage format of the semi-structured data to be sampled is JSON, determining the first semi-structured sampling algorithm includes: path-based sampling: analyzing the data structure of the JSON data, determining the data path to be extracted, and determining the data that meets the path as a sample based on the path expression; key-value pair-based sampling: treating the JSON data as a set of key-value pairs, and determining the corresponding value as a sample based on the specified key or combination of keys; statistics-based sampling: if the JSON data contains a large number of similar objects or array elements, a statistical method can be used to select samples.

[0066] Furthermore, in the step 3, it specifically includes: when the storage format of the semi-structured data to be sampled is HTML, determining the first semi-structured sampling algorithm includes: sampling based on web page structure: using an HTML parser to parse the web page, building a DOM tree, and selecting specific nodes or node sets as samples based on the structure of the DOM tree; sampling based on CSS selectors: using the CSS selector to locate specific elements in the web page, and determining elements that meet the conditions as samples based on the selector expression; sampling based on web page content: performing text analysis on the web page content, extracting keywords or topics, and selecting relevant paragraphs or sentences as samples based on the distribution of keywords or topics.

[0067] Furthermore, in the step 3, it specifically includes: when the storage format of the semi-structured data to be sampled is CSV, determining the first semi-structured sampling algorithm includes: row-based sampling: selecting one or more rows in the CSV file as samples; column-based sampling: selecting one or more columns in the CSV file as samples; condition-based sampling: according to preset selection conditions, selecting rows or columns that meet the preset selection conditions as samples, wherein the preset selection conditions include value range and string matching.

[0068] Specifically, select an appropriate semi-structured sampling algorithm based on the type of semi-structured data to be sampled, the overall capacity, and specific sampling requirements.

[0069] The sampling algorithms for extracting samples from JSON data are as follows: Path-based sampling: Analyze the structure of JSON data to determine the data path to be extracted. Based on the path expression (similar to XPath but for JSON), randomly select data that matches the path as samples; Key-value pair-based sampling: Treat JSON data as a collection of key-value pairs. Based on the specified key or key combination, randomly select the corresponding value as a sample; Statistical sampling: If the JSON data contains a large number of similar objects or array elements, statistical methods (such as random sampling, stratified sampling, etc.) can be used to select samples.

[0070] The sampling algorithms for extracting web page data samples are as follows: Sampling based on web page structure: Using an HTML parser to parse the web page and construct a DOM tree. Based on the DOM tree structure, specific nodes or sets of nodes are selected as samples. Sampling based on CSS selectors: Using CSS selectors to locate specific elements on the web page. Based on the selector expression, elements that meet the conditions are randomly selected as samples. Sampling based on web page content: Text analysis is performed on the web page content to extract keywords or topics. Based on the distribution of keywords or topics, relevant paragraphs or sentences are selected as samples.

[0071] The sampling algorithms for extracting samples from CSV data are as follows: row-based sampling: randomly select one or more rows in the CSV file as samples; column-based sampling: randomly select one or more columns in the CSV file as samples; condition-based sampling: select rows or columns that meet the conditions as samples based on specified conditions (such as value range, string matching, etc.).

[0072] Step 4: Determine the total sampling amount according to the semi-structured data to be sampled, wherein the total sampling amount is an integer.

[0073] Furthermore, in step 4, it specifically includes: assuming that the total capacity is N and the sampling ratio is p, where p is a real number between 0 and 1; the calculation formula of the total sampling amount can be expressed as: in, To perform a rounding operation, ensure that the total number of samples is an integer.

[0074] Specifically, determine the total sampling amount, which should be an integer: the determination of the total sampling amount usually requires consideration of multiple factors, including the overall capacity, sampling proportion, tolerance for sampling error, etc. Assume that the overall capacity is N and the sampling proportion is p (a real number between 0 and 1).

[0075] The calculation formula for the total sampling volume can be expressed as:

[0076] in, Indicates a round-down operation to ensure that the total number of samples is an integer.

[0077] Step 5: Input the semi-structured data to be sampled read in step 1 and the total sampling amount determined in step 4 into the first semi-structured sampling algorithm, execute the first semi-structured sampling algorithm, complete sampling, and obtain sampling sample data.

[0078] Specifically, when executing the sampling algorithm, the read semi-structured data to be sampled and the calculated total sampling amount need to be input into the selected sampling algorithm first, and then the sampling algorithm is executed to complete the sampling and finally obtain the sample data.

[0079] Step 6: Determine a first encryption algorithm to encrypt the sample data;

[0080] Furthermore, in step 6, the first encryption algorithm may be a symmetric encryption algorithm or an asymmetric encryption algorithm; when the first encryption algorithm is a symmetric encryption algorithm, it is necessary to ensure that both communicating parties obtain and share the same secret key.

[0081] Furthermore, in step 6, it includes: when the first encryption algorithm is the AES algorithm, specifically including: plaintext grouping: the AES algorithm is a block encryption algorithm, and the length of each block is 128 bits. If the length of the plaintext to be encrypted is not an integer multiple of 16 bytes, padding is required to ensure that the plaintext length meets the requirement; key selection: the AES algorithm supports three key lengths of 128 bits, 192 bits and 256 bits, which correspond to the three encryption modes of AES-128, AES-192 and AES-256 respectively; key expansion: according to the selected key length, multiple rounds of keys are generated through the key expansion algorithm. ; Initial round key addition: perform an XOR operation on the initial key and the obtained plaintext group as the input of the initial encryption round; round function byte replacement: replace each byte of the input through a fixed replacement table; round function row shift: perform a row shift operation on the input state matrix; round function column confusion: perform a matrix multiplication operation on the state matrix and a fixed confusion matrix; round function round key addition: perform an XOR operation on the round key of the current round and the state matrix; the last round of the round function: omit the column confusion step, and add an XOR operation of the round key; after multiple round function processing, the encrypted ciphertext is obtained and output.

[0082] Specifically, an encryption algorithm is selected to encrypt the sample results. The specific implementation process is as follows: Based on the requirements and the characteristics of the sample data, an appropriate encryption algorithm is selected. The encryption algorithm can be symmetric (such as DES, 3DES, AES) or asymmetric (such as RSA, ECC). Considering security and performance, AES (Advanced Encryption Standard) is often selected due to its high speed and high security level.

[0083] Furthermore, the specific operation process of the AES algorithm mainly includes the following steps:

[0084] Phase 1: Preparation phase, specifically including:

[0085] (1) Plaintext Blocking: The AES algorithm is a block encryption algorithm, and each block is 128 bits (i.e., 16 bytes). If the length of the plaintext to be encrypted is not an integer multiple of 16 bytes, appropriate padding is required to ensure that the length meets the requirement.

[0086] (2) Key selection: AES supports three key lengths: 128-bit, 192-bit, and 256-bit, corresponding to the three encryption methods AES-128, AES-192, and AES-256, respectively. The key selection should be determined based on security requirements and the importance of the data.

[0087] The second stage: encryption stage, including:

[0088] (1) Key expansion: Based on the selected key length, multiple round keys (RoundKeys) are generated through the key expansion algorithm. These round keys will be used in the subsequent encryption process;

[0089] (2) Initial round key addition: The initial key (also called seed key) is XORed with the plaintext group and used as the input for the initial encryption round. This step is also called round key addition (AddRoundKey);

[0090] (3) Round Function: The core of the AES algorithm is the round function, which consists of the following four steps, which will be repeated multiple times (depending on the key length and encryption mode, the number of repetitions will also vary): Step 1: Byte Substitution (SubBytes): Each byte of the input is replaced by a fixed substitution table (S-box). This substitution is nonlinear and increases the complexity of encryption; Step 2: Row Shift (ShiftRows): Perform a row shift operation on the input state matrix (4x4 byte matrix). Specifically, the second row is shifted left by one, the third row is shifted left by two, and the fourth row is shifted left by three. This step increases the diffusion of the data; Step 3: Column Confusion (MixColumns): Perform a matrix multiplication operation (modulo 2 addition and multiplication) on the state matrix and the fixed confusion matrix. This step further enhances the diffusion and confusion effect of the data; Step 4: Round Key Addition (AddRoundKey): Perform an XOR operation on the round key of the current round and the state matrix. This step introduces changes in the key in each round.

[0091] (4) Final round processing: In the final round, the column obfuscation step is omitted, but an XOR operation of the round key is added at the last step of the round function to ensure that the decryption process can be carried out correctly.

[0092] The third stage: output ciphertext: After multiple rounds of round function processing, the encrypted ciphertext is finally obtained.

[0093] Furthermore, when choosing a symmetric encryption algorithm, it is necessary to ensure that both communicating parties can securely obtain and share the same secret key. This is achieved through a key distribution protocol.

[0094] Furthermore, the sample data is encrypted using a selected encryption algorithm and key. The encryption algorithm is integrated into the data processing flow through programming. The encrypted data is ensured to be impossible to decrypt or tamper with by unauthorized personnel.

[0095] Step 7: Obtain a sampling data object according to a preset sampling basic data structure, wherein the sampling data object includes a description block and a data block, and the data block is used to store the encrypted sampling sample data.

[0096] Furthermore, in step 7, the description block is used to store information of the sampled data, including: sampling data type, sampling data file name, sampling data size, sampling data pattern URL or file path, sampling data content summary, special description block, sampling range or path, sampling rules, sampling algorithm, sample encryption algorithm, and description block encryption algorithm.

[0097] Specifically, a sampling data object is obtained based on the designed sampling basic data structure, wherein the sampling data object includes a description block and a data block. The description block is used to store sampling description information, including: sampling data type, sampling data file name, sampling data size, sampling data pattern URL or file path, sampling data content summary, special description block, sampling range or path, sampling rules, sampling algorithm, sample encryption algorithm, and description block encryption algorithm; the data block is used to store the encrypted sampling sample result.

[0098] The sampling data storage structure SampStoreStruct (Sampling data storage structure) is defined as:

[0099] SampStoreStruct::=<Description block (SampDesc)><Data block (SampData)>

[0100] <Description block (sampDesc)>::=BscDtStruct

[0101] <data block (sampData)> = <digit> +

[0102] Among them, Figure 2 As shown, the description block (SampDesc) is used to store sampling description information, and the data block (SampData) is used to store sampling results.

[0103] Step 8: Integrate and store or transmit the sampled data objects.

[0104] Furthermore, in step 8, after the encryption of the sampled data is completed, the description block is integrated with the encrypted data block to obtain the sampled data object, and the sampled data object is stored on a storage medium.

[0105] Specifically, after the sample results are encrypted, the description block and the encrypted data block are combined to form a complete sampled data object. This object is then securely stored on a designated storage medium (such as a hard drive or cloud storage). By performing final encryption on the data description block and data block when generating the sampled data object, secure data storage is achieved. This effectively addresses the issue of prior art neglecting data security during the data sampling process.

[0106] Furthermore, the sampling algorithm and execution logic of the present invention are specifically as follows:

[0107] 1) Sampling algorithm for extracting samples from JSON data

[0108] The specific execution logic of path-based sampling is as follows:

[0109] Step 1: Parse JSON: First, use a JSON parser (such as Python's json library) to parse the JSON string into a Python dictionary (or the corresponding data structure of other programming languages);

[0110] Step 2: Determine the path: Analyze the structure of the JSON data and determine the path of the data to be extracted. The path can be expressed as a string, for example, "0.title", which means starting from the root object and accessing the value corresponding to the title key (an object);

[0111] Step 3: Generate path expressions: Generate all possible path expressions based on the structure of the JSON data. This may require recursively traversing the entire JSON object;

[0112] Step 4: Randomly select a path: randomly select one or more paths from the generated path expressions;

[0113] Step 5: Extract samples: According to the selected path, extract the corresponding data from the JSON data as samples.

[0114] The specific execution logic of key-value pair sampling is as follows:

[0115] Step 1: Parse JSON: Again, use the JSON parser to parse the JSON string into a Python dictionary;

[0116] Step 2: Determine the key: Specify the key or key combination for which you want to extract data. For example, you can specify a key list ['title', 'URL', description] to extract the title value of all book objects.

[0117] Step 3: Traverse JSON: Recursively traverse the JSON object and find all key-value pairs that match the specified key or key combination;

[0118] Step 4: Randomly select key-value pairs: Randomly select one or more key-value pairs from the found key-value pairs as samples;

[0119] Step 5: Extract samples: Return the value corresponding to the selected key-value pair as a sample.

[0120] Based on statistical sampling, the specific execution logic is as follows:

[0121] Step 1: Parse JSON: Parse the JSON string into a Python dictionary;

[0122] Step 2: Determine the statistical object: Determine the object or array element to be statistically analyzed. For example, it can be an array containing multiple book objects;

[0123] Step 3: Apply statistical methods: Draw samples from the statistical object according to the selected statistical method (such as random sampling, stratified sampling, etc.). For example, you can use Python's random.sample() function to randomly extract elements from an array;

[0124] Step 4: Draw samples: Return the drawn samples.

[0125] 2) Sampling algorithm for extracting web page data samples

[0126] Sampling based on web page structure, the specific execution logic is as follows:

[0127] Step 1: Parse the web page: Use an HTML parser (such as BeautifulSoup) to parse the web page content and build a DOM tree;

[0128] Step 2: Determine the node: According to the structure of the DOM tree, determine the node or node set from which data needs to be extracted. For example, you can specify that you want to extract all The content of the label;

[0129] Step 3: Select nodes: traverse the DOM tree and find all nodes that meet the specified conditions;

[0130] Step 4: Randomly select nodes: Randomly select one or more nodes from the found nodes as samples;

[0131] Step 5: Extract samples: Return the content or attributes of the selected node as a sample.

[0132] The specific execution logic of sampling based on CSS selectors is as follows:

[0133] Step 1: Parse the web page: Also use the HTML parser to parse the web page content;

[0134] Step 2: Determine the selector: Write a CSS selector expression to locate a specific element in the web page. For example, p.intro means to select all elements with the class name intro. element;

[0135] Step 3: Apply the selector: Use a CSS selector library (such as BeautifulSoup's select() method) to apply the selector expression and find all elements that meet the conditions;

[0136] Step 4: Randomly select elements: Randomly select one or more elements from the found elements as samples;

[0137] Step 5: Extract samples: Return the content or attributes of the selected elements as samples.

[0138] The specific execution logic of the sampling based on web page content is as follows:

[0139] Step 1: Parse the web page: parse the web page content and extract text information;

[0140] Step 2: Text Analysis: Perform text analysis on the webpage content to extract keywords or topics. This can be done using natural language processing (NLP) techniques such as TF-IDF, bag-of-words models, or topic models.

[0141] Step 3: Determine the sample area: Based on the distribution of keywords or topics, determine the paragraphs or sentences containing relevant information as the sample area;

[0142] Step 4: Randomly select sample areas: randomly select one or more sample areas from the determined sample areas as samples;

[0143] Step 5: Extract sample: Return the content of the selected sample area as a sample.

[0144] 3) Sampling algorithm for extracting samples from CSV data

[0145] The specific execution logic of row-based sampling is as follows:

[0146] Step 1: Read CSV: Use a CSV parser (such as Python's csv library) to read the CSV file;

[0147] Step 2: Determine the number of rows: Get the total number of rows in the CSV file;

[0148] Step 3: Randomly select rows: Use a random number generator to randomly select the index of one or more rows;

[0149] Step 4: Extract samples: Based on the selected index, extract the corresponding rows from the CSV file as samples.

[0150] Column-based sampling, the specific execution logic is as follows:

[0151] Step 1: Read CSV: Read the CSV file;

[0152] Step 2: Determine column names: Get the column names (headers) of the CSV file;

[0153] Step 3: Randomly select columns: Randomly select the index or column name of one or more columns;

[0154] Step 4: Extract samples: Based on the selected columns, extract the corresponding column data from the CSV file as samples.

[0155] The specific execution logic of conditional sampling is as follows:

[0156] Step 1: Read CSV: Read the CSV file;

[0157] Step 2: Determine the conditions: Specify the filter conditions, such as value range, string matching, etc. For example, you can select all rows where the value of the age column is greater than 30;

[0158] Step 3: Filter data: Filter data according to conditions to obtain rows or columns that meet the conditions;

[0159] Step 4: Randomly select samples: Randomly select one or more samples from the filtered data;

[0160] Step 5: Draw samples: Return the selected samples.

[0161] The following are three main technical problems to be solved by the present invention and their effective effects:

[0162] (1) Improving the flexibility and accuracy of semi-structured data sampling: To address the problem of single sampling algorithms and insufficient flexibility in processing semi-structured data in existing technologies, the present invention designs multiple sampling algorithms, including JSON data sampling algorithms based on paths, key-value pairs, and statistics; web data sampling algorithms based on web page structure, CSS selectors, and web page content; and CSV data sampling algorithms based on rows, columns, and conditions. These algorithms can be flexibly selected based on different data types and specific sampling requirements, thereby improving sampling accuracy and flexibility and meeting sampling requirements in different scenarios.

[0163] (2) Ensure the security and traceability of sampling data:

[0164] Existing technologies often neglect data security during data sampling, making the sampling results susceptible to tampering or leakage. This invention introduces encryption during the sampling process, encrypting the sampling results and ultimately encrypting the data description block and data block when generating the sampled data object, thus achieving secure data storage. Furthermore, the designed basic sampling data structure, including the description block and data block, makes the sampled data traceable, facilitating subsequent data analysis and processing.

[0165] (3) Optimize the storage structure and management of sampling data:

[0166] Existing technologies for sampled data storage suffer from irrational structures and inconvenient management. The present invention designs a rational sampled data storage structure, including a basic data structure and a sampled data storage structure, enabling orderly and efficient storage and management of sampled data. Furthermore, through the design of an overall control algorithm, comprehensive control and management of the sampling process is achieved, improving the efficiency and accuracy of data sampling and optimizing the storage structure and management of sampled data.

[0167] Through the design of an overall control algorithm, the present invention achieves comprehensive control and management of the sampling process. From reading the semi-structured data to be sampled, to parsing its structure, selecting an appropriate sampling algorithm, executing the sampling algorithm, encrypting the sampling results, and finally generating the sampled data object and performing the final encryption, the entire process is effectively controlled and managed. This improvement not only improves the efficiency and comprehensive control and management capabilities of data sampling, but also ensures the stability and reliability of the sampling process, making it widely applicable to fields such as big data processing, data mining, and data analysis.

[0168] Example 2

[0169] The embodiment of the present invention provides a specific implementation example of a method for dynamic sampling and secure storage of semi-structured data, which is as follows:

[0170] The semi-structured data sampled in this embodiment is a file named US_STATE_recipes.json, part of which is as follows: Figure 3 Shown:

[0171] This embodiment is divided into two parts: (1) an embodiment of the structural design of the sampling data of the present invention; and (2) an embodiment of the overall control algorithm design of the data sampling of the present invention.

[0172] (1) Structural design embodiment of sampling data of the present invention

[0173] The structural design of the sampling data of the present invention includes two parts: basic data structure design and sampling data storage structure design;

[0174] 1) Basic data structure design

[0175] The basic data structure BscDtStruct (Basic data structure) is defined as:

[0176] BscDtStruct::= D1 D2 D3 D4 D5 D6 D7 D8 D9 D10 D11 (1)

[0177] in,

[0178] D1::=<sample data type>='json'

[0179] D2::=<sample data file name>='US_STATE_recipes'

[0180] D3::=<Sampling data file size>='6123KB'

[0181] D4::=<URI or file path of sample data schema>='F:\\US_STATE_recipes\\US_STATE_recipes.json'

[0182] D5::=<Sample Data Content Summary>='This is a file of all American recipes divided by state, including recipe name, basic recipe data, preparation time, cooking time, cooking method, nutritional ingredients, etc.'

[0183] D6::=<Special instructions block>

[0184] D61::=<Nested structure description>

[0185] D611::=<Nesting Depth>='2' (Considering "0" as the outer key and its inner objects as the inner layers, the depth is 2)

[0186] D612::=<name or identifier of nested structure>='Recipe Entry'

[0187] D613::=<nested structure type>='object'

[0188] D614::=<nested structure path expression>='"$.0" (represents the value of the key "0" in the JSON object, i.e. the recipe entry)'

[0189] D615::=<Nested structure field name>='"Contient","Country_State","cuisine","title","URL","rating","total_time","prep_time","cook_time","description","ingredients","instructions","nutrients","serves"'

[0190] D616::=<Nested structure field type>='Mixed types (including strings, numbers, arrays and objects)'

[0191] D62::=<Key field description>

[0192] D621::=<Key field name>='"title","URL","description","ingredients","instructions","nutrients"'

[0193] D622::=<path expression of key field>='".0.title",".0.URL",".0.description",".0.ingredients",".0.instructions",".0.nutrients" (representing the title and URL fields in the object with key "0" in the JSON object respectively)'

[0194] D63::=<External reference description>

[0195] D631::=<External Reference Name>='RecipeLink'

[0196] D632::=<External Reference Type>='URL'

[0197] D633::=<external reference path>=''

[0198] D634::=<value of external reference>='https: / / www.allrecipes.com / recipe / ...'

[0199] D64::<Version Compatibility> = This JSON data format is compatible with general JSON parsers and has no specific version requirements.

[0200] D7::=<sampling range or path>=entire data block

[0201] D8::=<Sampling Sample Size>='367'

[0202] D9::=<Sampling algorithm / rule>='Sampling based on key-value pairs'

[0203] D10::=<Sample encryption algorithm>='AES'

[0204] D11::=<Description of whether the block is encrypted>='No'

[0205] 2) Sampling data storage structure design

[0206] The sampling data storage structure SampStoreStruct (Sampling data storage structure) is defined as:

[0207] SampStoreStruct::=<Description block (SampDesc)><Data block (SampData)>

[0208] <Description block (sampDesc)>::=BscDtStruct

[0209] <data block (sampData)> = <digit> +

[0210] (2) Overall control algorithm design of data sampling of the present invention

[0211] The overall control algorithm design embodiment of data sampling of the present invention specifically includes the following steps:

[0212] 1) Read the semi-structured data to be sampled.

[0213] In this embodiment, a file named "US_STATE_recipes.json" is selected as the data source to be sampled. This file contains American recipe data.

[0214] 2) Determine the type of the semi-structured data to be sampled and parse the structure of the semi-structured data to be sampled.

[0215] In this embodiment:

[0216] Step 1: Determine the data type: In this embodiment, the data type is a JSON file.

[0217] Step 2: Parsing the Data Structure: Next, the schema or structure of the JSON file needs to be parsed to determine the hierarchy and relationships of the data. JSON files typically contain key-value pairs, where the key represents the attribute name of the data and the value represents the corresponding data content. During the parsing process, the structure of each data entry is identified. For example, each entry contains the following fields: "Contient", "Country_State", "cuisine", "title", "URL", "rating", "total_time", "prep_time", "cook_time", "description", "ingredients", "instructions", "nutrients", "serves".

[0218] Step 3: Data hierarchy and relationships: The data hierarchy includes: Top level: a JSON object containing multiple key-value pairs. The key is a string type ("0", "1", etc.), and the value is another JSON object; Second level: For non-null values (such as the value corresponding to the key "0"), this is a detailed recipe object containing multiple fields such as "Contient", "cuisine", "title", etc. These fields are further divided into different types of data, such as strings, numbers, arrays (such as "ingredients" and "instructions"), and another JSON object (such as "nutrients"). Data relationships include: Key-value relationship: At the top level, each key (such as "0", "1", etc.) is uniquely mapped to a value (another JSON object). Field-data relationship: In the second level (recipe object), each field (such as "title", "URL", "ingredients", etc.) stores specific data related to the recipe. Nested relationships: In a recipe object, the "nutrients" field is a nested JSON object containing multiple fields about the recipe's nutritional content (e.g., "calories", "carbohydrateContent", etc.). Array relationships: The "ingredients" and "instructions" fields are arrays containing lists of strings for the recipe's ingredients and steps, respectively.

[0219] 3) Select an appropriate semi-structured sampling algorithm based on the type of semi-structured data to be sampled, the overall capacity, and specific sampling requirements.

[0220] In this embodiment, the type of semi-structured data to be sampled is a JSON file, and each data entry is an object containing a key-value pair. Considering this structural feature of the data, a semi-structured sampling algorithm based on key-value pairs is selected.

[0221] 4) Determine the total sampling volume. Note that the total sampling volume should be an integer.

[0222] The determination of the total sampling volume usually requires consideration of multiple factors, including the overall capacity, sampling ratio, tolerance for sampling error, etc.

[0223] Assume that the population capacity is N (that is, the total number of data entries in the JSON file) and the sampling proportion is p (a real number between 0 and 1).

[0224] The calculation formula for the total sampling volume can be expressed as:

[0225] in, Indicates a round-down operation to ensure that the total number of samples is an integer.

[0226] In this embodiment, the calculated JSON file contains 3672 data entries (ie, N=3672), and 10% of the samples are extracted (ie, p=0.1).

[0227] According to the calculation formula of the total sampling volume:

[0228] Therefore, the total sampling volume was determined to be 367.

[0229] 5) Execute the sampling algorithm: input the read semi-structured data to be sampled and the calculated total sampling amount into the selected sampling algorithm, execute the sampling algorithm, complete the sampling, and obtain the sample data.

[0230] In this embodiment, based on the semi-structured data to be sampled (i.e., the "US_STATE_recipes.json" file) read in the previous step and the calculated total sampling amount (3672 samples), the sampling operation is performed using the previously selected key-value pair-based semi-structured sampling algorithm.

[0231] The specific code implementation steps are as follows:

[0232] Step 1: Input: Semi-structured data to be sampled: "US_STATE_recipes.json" file, containing 3672 data entries, each of which is a JSON object containing key-value pairs. Total number of samples: 367.

[0233] Step 2: Sampling algorithm: First, initialize an empty list to store the sample data. Then, traverse the data entries in the entire "US_STATE_recipes.json" file. Next, decide whether to add the data entry to the sample list based on the preset sampling ratio (in this case, since the specific total sampling amount has been calculated, it can be regarded as a fixed quantity limit). To achieve this, use a counter to track the number of samples that have been added and stop adding when the total sampling amount is reached. To ensure the randomness of the sampling, a random number generator can be used when selecting samples. Generate a random number for each potential sample candidate, and decide whether to select the sample based on the size of the random number.

[0234] Step 3: Output data: After the sampling is completed, a list of 367 sample data will be obtained. These sample data are extracted from the original data set according to specific sampling rules (i.e., filtering and random selection based on key-value pairs). Through the above steps, the sampling algorithm is successfully executed and the required sample data is obtained. The sampling results based on the path and sample of this embodiment are as follows: Figure 4 As shown, all samples are put into a list of 367 sample data, and the results are as follows Figure 5 shown.

[0235] 6) Select an encryption algorithm to encrypt the sampling results.

[0236] In this embodiment, the AES (Advanced Encryption Standard) algorithm is selected to encrypt the sample results. The AES algorithm is a symmetric encryption algorithm that uses the same key for encryption and decryption. It is efficient and secure and is suitable for protecting the confidentiality of sensitive data. In this embodiment, the above semi-structured sampling structure list is converted into a byte string and encrypted. The encrypted data is printed (displayed in hexadecimal format). The specific results are as follows: Figure 6 shown.

[0237] 7) According to the designed sampling basic data structure, a sampling data object is obtained, wherein the sampling data object includes a description block and a data block.

[0238] The description block stores sampling descriptive information, including: sampling data type, sampling data file name, sampling data size, sampling data schema URL or file path, sampling data content summary, special description block, sampling range or path, sampling rules, sampling algorithm, sample encryption algorithm, and description block encryption algorithm. The data block stores encrypted sampling results. In this embodiment, after the sample is determined and encrypted, a sampling data object is constructed based on the designed sampling basic data structure. This object consists of two main parts: a description block and a data block.

[0239] 8) Integrate and store / transmit sample data objects.

[0240] After the sample results are encrypted, the description block is combined with the encrypted data block to form a complete sample data object. This object is then securely stored on a designated storage medium (such as a hard drive, cloud storage, etc.).

[0241] In this embodiment, the sample results are encrypted using the AES algorithm for the data block. The specific operation process is described in step 6 above. The AES algorithm ultimately generates a 23-byte, or 256-bit, key. Regarding the description block, since the description block typically does not contain sensitive information and encryption increases processing time and complexity, this embodiment does not encrypt the description block. The description block is then combined with the encrypted data block to form a complete sample data object, which is securely stored.

[0242] The above one or more technical solutions in the embodiments of the present application have at least one or more of the following technical effects: the method carefully designs the structure of the sampling data, including the basic data structure and the sampling data storage structure, to ensure the accuracy of the sampling process and the integrity of the data. In the overall control algorithm design, the method first reads the semi-structured data to be sampled, then parses its structure and selects a suitable sampling algorithm based on the data type, overall capacity and specific sampling requirements. For semi-structured data in different formats such as JSON, web pages and CSV, the method provides a variety of sampling algorithms based on paths, key-value pairs, statistics, web page structures, CSS selectors and web page content, as well as rows, columns and conditions to meet the sampling requirements in different scenarios. After determining the total sampling amount, the method executes the selected sampling algorithm and encrypts the sampling results to ensure the security of the data. Finally, based on the designed sampling basic data structure, this method generates a sampling data object containing a description block and a data block, and finally encrypts the data description block and the data block to achieve secure storage, thereby solving the technical problems in the existing technology that the sampling algorithm is single and lacks flexibility when processing semi-structured data, and the security of the data is often ignored during the data sampling process, which makes the sampling results easy to be tampered with or leaked. It achieves the technical effect of improving the flexibility and accuracy of semi-structured data sampling, ensuring the security and traceability of the sampled data, and enhancing the comprehensive control and management capabilities of data sampling, which has important practical application value.

[0243] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for dynamic sampling and secure storage of semi-structured data, characterized in that: The method comprises: Step 1: Read the semi-structured data to be sampled from the data source; Step 2: Determine the data type of the semi-structured data to be sampled and parse the data structure of the semi-structured data to be sampled; Step 3: Determine a first semi-structured sampling algorithm based on the data type, data structure, and preset sampling requirements; Step 4: determining a total sampling amount according to the semi-structured data to be sampled, wherein the total sampling amount is an integer; Step 5: Inputting the semi-structured data to be sampled read in step 1 and the total sampling amount determined in step 4 into the first semi-structured sampling algorithm, executing the first semi-structured sampling algorithm, and completing sampling to obtain sample data; Step 6: Determine a first encryption algorithm to encrypt the sample data; Step 7: Obtain a sampling data object according to a preset sampling basic data structure, wherein the sampling data object includes a description block and a data block, and the data block is used to store the encrypted sampling sample data; Step 8: Integrate and store or transmit the sampled data objects.

2. The method for dynamic sampling and secure storage of semi-structured data according to claim 1, wherein: In the step 1, the storage format of the semi-structured data to be sampled is JSON, XML, or CSV.

3. The method for dynamic sampling and secure storage of semi-structured data according to claim 2, wherein: In the step 3, it specifically includes: When the storage format of the semi-structured data to be sampled is JSON, determining the first semi-structured sampling algorithm includes: Path-based sampling: Analyze the data structure of the JSON data, determine the data path to be extracted, and determine the data that meets the path as a sample based on the path expression; Key-value pair-based sampling: The JSON data is treated as a set of key-value pairs, and the corresponding value is determined as a sample based on the specified key or key combination; Statistics-based sampling: If the JSON data contains a large number of similar objects or array elements, statistical methods can be used to select samples.

4. The method for dynamic sampling and secure storage of semi-structured data according to claim 2, wherein: In the step 3, it specifically includes: When the storage format of the to-be-sampled semi-structured data is HTML, determining the first semi-structured sampling algorithm includes: Sampling based on web page structure: using an HTML parser to parse the web page, constructing a DOM tree, and selecting specific nodes or node sets as samples based on the structure of the DOM tree; CSS selector-based sampling: using the CSS selector to locate specific elements in a web page, and determining elements that meet the conditions as samples based on the selector expression; Sampling based on web page content: Perform text analysis on the web page content, extract keywords or topics, and select relevant paragraphs or sentences as samples based on the distribution of keywords or topics.

5. The method for dynamic sampling and secure storage of semi-structured data according to claim 2, wherein: In the step 3, it specifically includes: When the storage format of the to-be-sampled semi-structured data is CSV, determining the first semi-structured sampling algorithm includes: Row-based sampling: select one or more rows in the CSV file as samples; Column-based sampling: select one or more columns in the CSV file as samples; Condition-based sampling: According to preset selection conditions, rows or columns that meet the preset selection conditions are selected as samples, wherein the preset selection conditions include value range and string matching.

6. The method for dynamic sampling and secure storage of semi-structured data according to claim 1, wherein: In said step 4, it specifically includes: Let the total capacity be N and the sampling proportion be p, where p is a real number between 0 and 1; The calculation formula of the total sampling amount can be expressed as: in, To perform a rounding operation, ensure that the total number of samples is an integer.

7. The method for dynamic sampling and secure storage of semi-structured data according to claim 1, wherein: In step 6, the first encryption algorithm may be a symmetric encryption algorithm or an asymmetric encryption algorithm; When the first encryption algorithm is a symmetric encryption algorithm, it is necessary to ensure that both communicating parties obtain and share the same secret key.

8. The method for dynamic sampling and secure storage of semi-structured data according to claim 1, wherein: In step 6, it includes: When the first encryption algorithm is the AES algorithm, the method specifically includes: Plaintext grouping: The AES algorithm is a block encryption algorithm. Each block is 128 bits long. If the length of the plaintext to be encrypted is not an integer multiple of 16 bytes, padding is required to ensure that the plaintext length meets the requirement. Key selection: The AES algorithm supports three key lengths: 128-bit, 192-bit, and 256-bit, corresponding to the three encryption modes AES-128, AES-192, and AES-256 respectively; Key expansion: Generate multiple rounds of keys using a key expansion algorithm based on the selected key length; Initial round key addition: XOR the initial key with the plaintext block and use it as the input of the initial encryption round; Round function byte replacement: each byte of the input is replaced by a fixed replacement table; Round function row shift: perform row shift operation on the input state matrix; Round function column confusion: performing matrix multiplication operation on the state matrix and the fixed confusion matrix; Round function round key addition: performing an XOR operation on the round key of the current round and the state matrix; The last round of the round function: the column obfuscation step is omitted, and the XOR operation of the round key is added; After multiple round function processing, the encrypted ciphertext is obtained and output.

9. The method for dynamic sampling and secure storage of semi-structured data according to claim 1, wherein: In step 7, the description block is used to store information of the sampled data, including: sampling data type, sampling data file name, sampling data size, sampling data pattern URL or file path, sampling data content summary, special description block, sampling range or path, sampling rules, sampling algorithm, sample encryption algorithm, and description block encryption algorithm.

10. The method for dynamic sampling and secure storage of semi-structured data according to claim 1, wherein: In step 8, after the encryption of the sample data is completed, the description block is integrated with the encrypted data block to obtain the sample data object, and the sample data object is stored on a storage medium.

Images